Check your starting point
A model predicts one value for each observation. Targets accidentally have an extra trailing singleton axis. Why might mean squared error still run?
Compare your reasoning
Broadcasting can construct a pairwise residual matrix instead of one residual per observation. Assert identical prediction and target shapes, then compute one residual by hand before reducing.
If this is unfamiliar, work through Linear algebra and loss intuition before attempting the project.
Build in stages
1. Audit the objective and derivative
Work through vectors, residuals, rank and directional derivatives. Use a linear solve and a finite-difference sweep as independent checks.
2. Connect dynamics to evaluation
Predict a stable versus divergent quadratic update, inspect stable likelihood arithmetic and separate fitting, validation selection and final test reporting.
3. Make optimizer comparisons fair
Count-weight unequal batches, preserve optimizer moments and schedule position, then change feature scale while retaining the prediction contract.
Try a changed condition
Make a prediction before opening the reasoning. Keep your answer and the evidence you would collect.
For the scalar loss , predict the next error multipliers for learning rates 0.4, 1.0 and 1.1. Which curve can oscillate without converging?
Compare an approach
The update is . The factors are , and . The middle case alternates at constant magnitude; the last grows. A plotting window can hide divergence when the starting value is tiny. Check the recurrence and plot absolute error on a logarithmic scale where nonzero.
Diagnose a likely failure
Symptom: Gradient descent diverges while all values initially remain finite.
Check next: Check the update sign, loss reduction, feature scales and largest quadratic curvature before trying a different seed.
Decide what is ready
Complete regression-audit, then optimizer-audit. Keep an analytic reference, stable/unstable trajectories, a resumed optimizer comparison and an untouched final evaluation split.
Where to put your project code
Write your code in: projects/optimizer-audit/my_optimization.py (copy projects/optimizer-audit/starter/optimization.py first: cp projects/optimizer-audit/starter/optimization.py projects/optimizer-audit/my_optimization.py).
Verify your implementation: python3 projects/optimizer-audit/tests/check.py --stage all --impl projects/optimizer-audit/my_optimization.py
Run the staged project checks on your implementation in projects/optimizer-audit/my_optimization.py and record your transfer experiment results in your portfolio.
What still needs further work
Constrained, nonsmooth and large-scale stochastic optimization are not covered comprehensively. Current ranking evidence is limited to declared synthetic objectives.