A ReentrantLock held forever after an exception — threads blocked with no deadlock reported

What this error actually means

There is no exception for this failure. The exception that caused it was thrown, logged, and handled somewhere upstream minutes earlier; what you are looking at now is the aftermath — a queue of threads parked in `AbstractQueuedSynchronizer.acquire` waiting for a lock whose owner has long since returned to the pool and is happily serving other requests.

The mechanism is the gap between `lock()` and `try`. `ReentrantLock` does not release on scope exit, on thread death, or on stack unwinding — it is an object with a counter and an owner field, and the only thing that decrements it is an `unlock()` call. If anything throws after the acquire and before the `finally` that releases, the counter stays above zero and the owner stays set to a thread that no longer knows it holds anything. Because reentrant locks track an owner rather than a stack frame, there is nothing in the runtime that can notice the mismatch.

The tell in the dump is the absence of a cycle. The JVM deadlock detector reports cycles in the wait-for graph, and a leaked lock is not a cycle — it is a single unreachable release. So `Found one Java-level deadlock` never appears, and the second giveaway is that the blocked threads name a lock whose *owner thread is nowhere in the dump waiting for anything*, or whose `Locked ownable synchronizers` section attributes it to a thread that is idle or gone.

This is also why the failure looks like a slow leak of capacity rather than an outage. One code path throws once per hour; each occurrence permanently removes the ability of any thread to enter that critical section. The endpoint that dies is not the endpoint that threw.

How to diagnose it

jcmd <pid> Thread.print -l > dump.txt

grep -n "0x000000070f3c9a80" dump.txt

jcmd <pid> Thread.print -l > d1.txt; jcmd <pid> Thread.print -l > d2.txt; diff d1.txt d2.txt

log.info("locked={} holdCount={} queued={}", lock.isLocked(), lock.getHoldCount(), lock.getQueueLength());

The fix

Adopt one acquire shape and never deviate: `lock()` on the line immediately before `try`, with nothing between them, and `unlock()` as the first statement of `finally`. Nothing between the acquire and the `try` is the whole point — that gap is the only place this bug can live.

For `tryLock`, branch on the result and release only inside the branch that acquired. The release must be governed by the same boolean that the acquire returned, never by reaching a particular line.

Remove early returns from critical sections, or restructure so the guard runs before the acquire. Computing the decision outside the lock and entering it only to mutate usually shortens the critical section as a side benefit, which helps contention too.

Never hold a lock across a network call, a disk write, or anything else that can block without a bound. If the data the call needs is under the lock, copy it out, release, call, then re-acquire to apply the result — and re-validate, because the state can have changed while you were outside.

Where the critical section is genuinely simple, delete the lock. A `ConcurrentHashMap.compute` or an `AtomicReference.updateAndGet` performs the read-modify-write atomically with no release to forget. Code that cannot leak a lock is better than code that releases correctly.

As a safety net once the shape is fixed, use `tryLock(timeout)` at the outermost acquisition so a held lock degrades into a failed request with an alert instead of an unbounded hang. Treat this as instrumentation, not as the fix.

// Leaks the lock: anything throwing before `try` skips the finally entirely
lock.lock();
validate(request);                 // throws -> lock held forever
try {
    stock.put(sku, remaining);
} finally {
    lock.unlock();
}

// Also leaks: the guard returns past the release
lock.lock();
try {
    if (stock.isEmpty()) return false;   // fine
    ...
} finally { lock.unlock(); }

// Correct: nothing between acquire and try; conditional acquire, conditional release
validate(request);                 // outside the critical section
if (!lock.tryLock(200, TimeUnit.MILLISECONDS)) {
    throw new ReservationBusyException(sku);
}
try {
    return stock.reserve(sku, qty);
} finally {
    lock.unlock();
}

How to stop it coming back

• Make "nothing between `lock()` and `try`" a review rule; it is mechanically checkable and catches every instance of this bug
• Enable an error-prone or SpotBugs rule for unreleased locks in CI rather than relying on human attention
• Write a test per critical section that forces the exception path and then asserts a subsequent call still completes within a timeout
• Alarm on `lock.getQueueLength()` sustained above zero — waiters queuing while throughput is normal is the earliest signal
• Prefer atomic collection operations over explicit locks wherever the critical section is a single read-modify-write
• Capture a thread dump automatically when a request exceeds a latency ceiling, so the evidence exists without needing to catch it live

FAQ

Related

Other errors engineers hit next to this one

• ERR_STREAM_PREMATURE_CLOSE during an upload
• Process exits before asynchronous writes finish
• ERR_MODULE_NOT_FOUND during ESM/CommonJS migration
• Unbounded JSON body blocks the loop or exhausts memory
• Undici / fetch connections remain occupied
• ERR_UNHANDLED_REJECTION in a worker thread
• command not found in a script that works interactively
• Permission denied when executing a script