Gradient Descent Ladder

Who This Is For

Use this ladder only when you intentionally want one smooth-loss optimization benchmark inside an algorithms repo, not because you expect high contest ROI.

Warm-Up

• state the squared-loss objective for one-feature linear regression
• derive the gradients with respect to w and b
• explain why the sign must be -gradient

Core

• one deterministic batch gradient step
• one fixed learning rate
• one fixed epoch count
• convergence intuition on a tiny regression benchmark

Repo Anchors

• Linear Regression Gradient Descent Benchmark
• Machine Learning Algorithms
• Optimization And Duality

Stretch

• explain why SGD is not the same contract as this lane
• compare gradient descent with the perceptron update rule
• explain why feature scaling changes practical convergence speed

Compare Points

• first smooth-loss optimizer benchmark -> Linear Regression Gradient Descent Benchmark
• hard classification sibling -> Machine Learning Algorithms
• certificate / benchmark sibling -> Optimization And Duality

This ladder is intentionally tiny.

The point is not to turn the repo into a full numerical optimization course. The point is to keep one source-backed gradient -> step -> iterate lane explicit and honest.

Exit Criteria

You are ready to move on when you can:

• write the batch gradient correctly
• explain the role of alpha
• say clearly why this repo lane is fixed-step breadth, not full modern optimization coverage

External Practice

• Stanford CS229 notes on linear regression and gradient descent
• MIT Open Learning Library: Gradient Descent
• MIT nonlinear optimization notes: Gradient descent and descent lemmas

Tutorial Link

• Gradient Descent