Backend coding interview
Distributed Job Scheduler Coding Interview Problem
What this interview round tests
The distributed job scheduler is one of the most asked system design interview questions for senior backend roles — and the part that actually gets graded is deceptively narrow: when several workers race to claim the same scheduled job, exactly one must run it, and a worker that dies mid-run must not lock the job forever.
Interviewers ask it because it is the essence of distributed coordination in a form you can reason about precisely: leases with expiry instead of locks that outlive their owner, atomic claim operations instead of check-then-act races, and the uncomfortable truth that "exactly once" really means "at least once plus idempotency". These are the mechanics behind cron services, queue consumers, and every "singleton" background task in production.
The scenario
You are handed a repository that simulates a fleet of workers sharing a job store. Jobs become due, workers poll and claim them, run them, and report completion. The coordination code exists — and under concurrent workers it falls apart.
Two workers claim and execute the same job, a crashed worker leaves its job claimed forever, expired leases get honoured as if they were still valid, and re-run attempts double-execute work that already completed. The test suite hammers the claim path with racing workers; your job is to make execution single-owner and exactly-once.
What you’ll practice
- Atomic claim semantics: check-and-take as one operation, not check then take
- Leases with expiry so a dead worker’s job becomes claimable again
- Lease validation at execution and completion, not just at claim time
- Exactly-once effects: idempotent completion when attempts overlap
- Race conditions: N workers competing for one due job, exactly one winner
- State transitions for a job lifecycle: due → claimed → running → done
How to approach it
Treat the claim as the critical section: a worker must observe "due and unclaimed" and transition to "claimed by me with a lease deadline" in one atomic step, so two pollers can never both see the old state. Everything downstream trusts that claim only while the lease is alive.
Then make expiry and completion defensive: a worker finishing a job must prove it still holds a valid lease before recording the result, and a reaper reclaiming expired leases must not clobber a job that just completed. Thinking in terms of fencing — the lease as a token checked at every effect — is what separates a passing solution from one that merely looks right.
The starter repository ships with a failing test suite and a bundled verify.sh. Reviewed reference solutions are part of Gronex Pro — this page stays spoiler-free on purpose.
FAQ
Is this a system design question or a coding question?
Both — that is the point. Interviews increasingly ask you to implement the heart of the design: the atomic claim, the lease, the recovery path. This problem gives you that core as runnable code with tests that simulate racing workers and crashes.
Why are leases better than locks for distributed workers?
A lock held by a crashed process is held forever; a lease expires. The trade-off is that expiry introduces its own races — a slow worker whose lease lapsed mid-run — which is why validating the lease at completion time, not just claim time, is part of the exercise.
Do I need real infrastructure to practise this?
No. The repository simulates workers and the shared job store in-process, so every race is reproducible and the bundled test suite can grade your coordination logic deterministically in Java, Python, or C++.