ML engineering round
ML Training Loop Gradient Bug Coding Problem
Written and reviewed by Sahil Srivastav
What this interview round tests
Understanding what happens inside the training loop is what separates people who can use a model from people who can build one. When you write gradient descent by hand, a single sign error in the gradient turns learning into unlearning: the loss climbs instead of falling, and the model gets steadily worse with every step.
Interviewers reach for a from-scratch trainer because it exposes exactly this understanding. A wrong-sign regularization term, a dropped bias update, an off gradient — none of them crash, they just quietly diverge. Reading a loss curve that goes the wrong way and knowing why is the skill under test.
The scenario
You are given a small logistic-regression trainer implemented in numpy — forward pass, loss, gradient, and an update step — with a fixed random seed for reproducibility. Training does not converge; the loss increases over the injected steps.
The tests are structural and bounded: they assert the loss strictly decreases over a fixed set of steps and that the learned weights land within tolerance of a reference. Your job is to find and fix the defect in the gradient so training actually minimises the loss.
What you’ll practice
- Deriving and implementing a correct gradient for logistic regression
- How regularization enters the gradient — and its sign
- Reading a diverging loss curve and localising the cause
- Reproducible training with fixed seeds
- Bounded, structural assertions instead of exact-value checks
How to approach it
Do not trust the code — re-derive the gradient from the loss and compare it term by term. A loss that climbs instead of falling is the signature of a gradient that points the wrong way somewhere; the work is localising which term is off and confirming every parameter that should move actually does.
Because the seed is fixed and the assertions are bounded, a correct gradient will make the loss decrease monotonically and land the weights near the reference — you are aiming for correct dynamics, not a magic number.
The starter repository ships with a failing test suite and a bundled verify.sh. Reviewed reference solutions are part of Gronex Pro — this page stays spoiler-free on purpose.
FAQ
Do I need PyTorch or TensorFlow?
No. The trainer is pure numpy so the gradient is fully visible and the bug is yours to find. It runs on CPU in seconds. The concepts transfer directly to any autodiff framework.
Why test that loss decreases rather than a final accuracy?
A strictly decreasing loss over fixed steps is the direct signature of a correct gradient, and it is stable across platforms. A final accuracy would be both flakier and a weaker signal about whether the gradient itself is right.