RMSProp

What is RMSProp?

Root Mean Square Propagation. An adaptive learning rate method developed by Geoff Hinton (in a Coursera lecture!) that divides the gradient by a running average of its recent magnitude.

Where did the term "RMSProp" come from?

Hinton's Lecture 6e.

How is "RMSProp" used today?

Precursor to Adam; still effective for RNNs.

Related Terms