Found one Java-level deadlock

What this error actually means

The JVM has done the hard part. This block appears at the bottom of a thread dump when the deadlock detector finds a cycle in the lock wait-for graph: thread A holds lock 1 and waits for lock 2 while thread B holds lock 2 and waits for lock 1. Neither will ever proceed, and nothing will break the cycle short of killing the process.

Read it as a cycle, not as a list. Each stanza names the monitor a thread is waiting for and the thread that holds it. Follow the "held by" links and you return to where you started — that loop is the bug, and the object identities in it tell you exactly which two resources were locked in opposite orders.

The structural cause is almost always the same: two code paths acquire the same pair of locks in different orders. A transfer from account X to Y locks X then Y; a simultaneous transfer from Y to X locks Y then X. The code looks symmetric and correct, and it deadlocks only when both directions run at once — which is why it passes review and fails in production.

How to diagnose it

jcmd <pid> Thread.print > dump1.txt; jcmd <pid> Thread.print > dump2.txt; diff dump1.txt dump2.txt

jcmd <pid> Thread.print -l | grep -B 5 "waiting on condition"

The fix

Impose a global lock order and make every path follow it. Derive the order from a stable, total ordering of the resources — the primary key, or `System.identityHashCode` with an explicit tie-breaker — and always acquire in that order regardless of which direction the operation logically runs. Once every path agrees on the order, a cycle is impossible by construction. This is the fix to prefer, because it removes the possibility rather than handling the symptom.

Shrink the critical section so the pair is never held at once. Often the second lock is needed only for a brief mutation that can be done after the first is released, or the whole operation can be restructured around a single lock covering both resources.

Never hold a lock while calling code you do not control. Compute inside the lock, release, then invoke callbacks and listeners outside it. This eliminates the entire class of invisible nested acquisition.

Use `tryLock` with a timeout as a safety net, not a design. Acquiring with a bound turns an eternal hang into a failed operation you can retry with backoff and, more importantly, an alert. It is the right belt-and-braces addition after ordering is fixed, and a poor substitute for fixing it.

Always release in `finally`. For explicit locks, `lock()` immediately before `try` and `unlock()` in `finally` is the only safe shape — this also fixes the leaked-lock variant where no cycle exists at all.

// Deadlocks when transfer(a,b) and transfer(b,a) run concurrently
void transfer(Account from, Account to, BigDecimal amt) {
    synchronized (from) {
        synchronized (to) { from.debit(amt); to.credit(amt); }
    }
}

// Total order on identity makes a cycle impossible
void transfer(Account from, Account to, BigDecimal amt) {
    Account first  = from.id() < to.id() ? from : to;
    Account second = first == from ? to : from;
    synchronized (first) {
        synchronized (second) { from.debit(amt); to.credit(amt); }
    }
}

How to stop it coming back

• Document the lock order for every pair of locks that can be held together, and enforce it in review
• Prefer a single coarser lock, or lock-free structures, over two fine-grained locks whose interaction you must reason about
• Write concurrency tests with mirrored arguments across many threads; that is the only shape that catches ordering bugs
• Alarm on requests exceeding a latency ceiling and dump threads automatically when it trips, so the evidence is captured while the deadlock is live
• Never synchronise on interned strings, boxed primitives, or reassignable fields

FAQ

Related

Other errors engineers hit next to this one

• CompletableFuture failed with nothing logged
• awaitTermination never returns and the JVM will not exit
• InterruptedException caught and ignored — the task can no longer be cancelled
• Two unrelated components sharing a monitor via a boxed Integer or interned String
• Cache stampede — the same expensive value built many times concurrently
• Lock convoy — throughput collapses as threads are added, with no deadlock
• ReadWriteLock writer blocked indefinitely behind a stream of readers
• Worker loop never sees the stop flag and runs forever