Python

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out

Written and reviewed by Sahil Srivastav

SQLAlchemyConnection poolProduction incident
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/20/3o7r)

What this error actually means

This is a queue timeout inside your process. A thread or task asked the engine for a connection, all `pool_size + max_overflow` slots were already checked out, and the caller waited `pool_timeout` seconds before giving up. PostgreSQL never saw the request. Database CPU can be flat at 5% while this fills your logs.

Read the numbers in the message as capacity: size 5 plus overflow 10 means fifteen concurrent connections is the hard ceiling for this engine, in this process. Multiply by worker count to get what the database actually sees — four Gunicorn workers with that configuration can demand sixty connections, which is most of a default `max_connections` of 100 before anything else connects.

Behind the timeout there are only two states, and they need opposite fixes. Either fifteen connections are genuinely busy executing queries, or connections have been checked out and never returned. A leak has a signature: the pool never recovers when traffic stops, and a restart fixes it for a predictable number of requests. Saturation tracks load and clears at the trough.

Causes, most common first

  1. 1A session created outside a scope that closes it. The most common cause by a wide margin. `Session()` called in a helper, a cached property, a signal handler or a background thread, with no `close()` on the failure path. Every exception permanently consumes one slot, so the outage arrives long after the requests that caused it.
  2. 2A generator or streaming response holding a session open. A FastAPI or Flask endpoint that yields rows from a query keeps the connection checked out until the client finishes reading. A slow client, or a client that disconnects without the server noticing, pins a slot for as long as the socket lives.
  3. 3Network or CPU work inside the session scope. Fetching rows, then calling a payment API, then writing — all inside one `with Session()` block. The slot is held for the duration of the HTTP call, so any latency spike downstream drains the pool. This is saturation created by scope, not by traffic.
  4. 4Pool sized per process but reasoned about per service. The configuration looks modest until you multiply by workers, threads, replicas and Celery processes. Often the pool is large enough to exhaust `max_connections` at the database and small enough to time out in the application — the worst of both.
  5. 5A pool inherited across fork, or shared with asyncio incorrectly. An engine created before `fork()` gives every worker handles to the same sockets. In async code, using a sync engine inside `async def`, or sharing an `AsyncEngine` across event loops, produces checkouts that are never cleanly returned.

When you see it

  • Checked-out count sits at `pool_size + max_overflow` even with zero requests in flight
  • A restart fixes it completely, then it returns after roughly the same volume of traffic
  • `pg_stat_activity` shows sessions `idle in transaction` rather than `active`
  • Latency on every endpoint degrades together, because they all queue on the same pool
  • It began after adding a background task, a streaming endpoint, or an async view to a sync codebase

How to diagnose it

Step 1

Print the pool status, repeatedly

This is the decisive measurement. Watch it as traffic drains: checked-out connections that never fall to zero prove a leak and rule out capacity entirely.

print(engine.pool.status())
# Pool size: 5  Connections in pool: 0  Current Overflow: 10  Current Checked out connections: 15

Step 2

Capture the stack at checkout time

SQLAlchemy exposes pool events. Recording a stack for every checkout, and clearing it on checkin, gives you the exact line that borrowed the connection that was never returned — the equivalent of HikariCP leak detection.

from sqlalchemy import event
import traceback

@event.listens_for(engine, "checkout")
def _on_checkout(dbapi_con, con_record, con_proxy):
    con_record.info["stack"] = traceback.format_stack()

Step 3

Ask the database what the sessions are doing

Rows in `idle in transaction` mean a lifecycle bug in your code — the connection is checked out, the transaction is open, and nothing is running. Rows in `active` with long durations mean real work and a query problem.

SELECT state, count(*), max(now() - state_change) AS longest
FROM pg_stat_activity
WHERE datname = current_database()
GROUP BY state ORDER BY 2 DESC;

Step 4

Reproduce the leak deliberately

Fire `pool_size + max_overflow` requests at the endpoint you suspect and force each one to raise. If the next healthy request times out, you have proven the exception path never returns the session.

The fix

Make every session a scope with an unconditional close. `with Session(engine) as session:` releases the connection on every exit path including exceptions; in FastAPI the equivalent is a dependency that yields and closes in a `finally`. Never construct a `Session` and rely on garbage collection or on reaching the end of a function — the whole failure mode is paths that do not reach the end.

Commit or roll back before the connection is returned. A session closed with an open transaction returns a connection PostgreSQL reports as `idle in transaction`, which holds a snapshot and possibly row locks while counting as available. `session.begin()` as a context manager makes the boundary explicit and correct on both paths.

Shrink what happens inside the scope. Read what you need, close, do the HTTP call or the CPU-heavy transformation, then open a new scope to write. A connection is expensive shared state, not a convenience to hold for the life of a request handler.

Size from the database backwards. Decide the total connections the database can serve, subtract headroom for migrations, admin sessions and replicas, then divide by the number of processes to get `pool_size`. Keep `max_overflow` small and deliberate — a large overflow mostly delays the timeout while adding load the database must track. If you need many more clients than the database can hold, that is what PgBouncer in transaction mode is for.

For background work, give Celery and cron its own engine with its own small pool, so a slow batch job cannot starve request handling. Shared pools couple unrelated failure domains.

# Leaks a slot on every exception between here and close()
session = Session(engine)
rows = session.execute(stmt).all()      # raises -> slot gone until restart
session.close()

# Scope closes on every path; transaction boundary is explicit
with Session(engine) as session:
    with session.begin():
        rows = session.execute(stmt).scalars().all()
        payload = [serialise(r) for r in rows]

# HTTP call happens after the connection is back in the pool
response = client.post(url, json=payload)

How to stop it coming back

  • Export `engine.pool.status()` as metrics and alarm on checked-out connections staying high while request concurrency is low — that combination is only ever a leak
  • Install the checkout-stack pool event permanently in staging; it costs nothing and names the offending line before production does
  • Add a test that forces the failure path `pool_size + max_overflow + 1` times and asserts the service still serves traffic
  • Set `statement_timeout` at the database so one pathological query cannot hold a slot indefinitely
  • Review every `Session(` construction outside a dependency or context manager — that is where these bugs live

Practise this failure in a real repository

Gronex ships this failure as a runnable repository: a small pool, an exception path that never returns its connection, and sessions handed back while still inside an open transaction. The test suite asserts the lifecycle invariant rather than the happy path, so raising `pool_size` does not make it pass.

FAQ

Should I raise pool_size and max_overflow?

Only once `pg_stat_activity` shows the sessions genuinely active. If they are idle in transaction you are buying a longer runway to the same outage while adding connections the database has to track. And every increase multiplies by worker count, so this is the fastest route from a pool timeout to `FATAL: sorry, too many clients already`.

Why does the error appear long after the request that caused it?

A leak consumes one slot per failure while the pool keeps serving from the remaining slots. The timeout only appears when the last slot is gone, which can be hundreds of requests later. That lag is why the wrong endpoint almost always gets blamed first.

Does closing the session roll back an open transaction?

Closing a session releases the connection and the pool issues a rollback on return, so uncommitted work is discarded — which is safe but silent. Code that expected the write to land sees no error at all. Make the commit explicit so the outcome is in your code rather than in pool behaviour.

Is NullPool a reasonable fix?

It removes the timeout by connecting per use, which trades a visible queue for per-request connection setup and unbounded concurrent connections against the database. It is the right choice with an external pooler like PgBouncer in front, and the wrong choice as a way to avoid fixing a session lifecycle.

Related

psycopg2: connection already closedDetachedInstanceError: not bound to a SessionPostgreSQL: too many clients alreadyDatabase engineering interview questions

Other errors engineers hit next to this one

Full error and symptom index →