ML engineering round
ML Metric Computation & Pooling Coding Problem
Written and reviewed by Sahil Srivastav
What this interview round tests
If your metric is wrong, everything downstream is wrong — model selection, thresholds, the go/no-go decision. Metric bugs are insidious because the number still looks like a metric: a swapped precision and recall, or a mean of per-batch scores that does not equal the metric computed over the pooled data, produces a plausible value that quietly misranks your models.
Interviewers ask this because computing metrics correctly is table stakes and surprisingly easy to botch. Averaging F1 across batches is not the same as F1 over all examples. Precision and recall are trivial to transpose. The evaluation looks fine and the conclusions are wrong.
The scenario
You are given hand-written metric functions used in an evaluation harness. One transposes precision and recall; another averages a metric per batch instead of pooling all predictions before computing it.
The tests compare your functions against scikit-learn across a range of thresholds and batch layouts. Your job is to fix the definitions so your results match the reference on every case.
What you’ll practice
- Precise definitions of precision, recall, and related metrics
- Micro vs per-batch averaging and why they differ
- Pooling predictions before computing a metric
- Threshold sweeps and how metrics move with the threshold
- Validating custom metrics against a trusted reference (sklearn)
How to approach it
Go back to the definitions rather than trusting the code. Precision and recall differ only in their denominator, so pin down which is which. And ask a sharper question about aggregation: can a ratio metric be averaged across batches at all, or does batch layout change an answer that should not depend on it?
Use scikit-learn as your oracle: match it across thresholds and batch arrangements, and any remaining discrepancy points straight at the definition that is still wrong.
The starter repository ships with a failing test suite and a bundled verify.sh. Reviewed reference solutions are part of Gronex Pro — this page stays spoiler-free on purpose.
FAQ
Why not just always use sklearn.metrics?
In production you often do, but interviews and real systems still require you to understand what the numbers mean — custom aggregation, streaming metrics, and per-group breakdowns all need the definitions. Here sklearn is the reference you validate against.
What is wrong with averaging a metric per batch?
Ratio metrics like precision and F1 are not means of per-batch values — a small batch and a large batch would count equally. Pooling all predictions and computing once gives the correct micro-averaged result.