Learning Rate Schedule

What is Learning Rate Schedule?

A strategy to adjust the learning rate during training (e.g., Warmup, Cosine Decay). It helps the model converge faster and avoid getting stuck in local minima.

Where did the term "Learning Rate Schedule" come from?

Critical for training stability.

How is "Learning Rate Schedule" used today?

Used in every modern training run.

Related Terms

• Adam (Adaptive Moment Estimation)
• Gradient Descent (SGD)