Distributed systems

Idempotency key reused with a different request body

Written and reviewed by Sahil Srivastav

Distributed systemsAPIsCorrectness

409 Conflict: idempotency key 7f3a was already used with a different request fingerprint

What this error actually means

An idempotency key names one attempted operation. Reusing it with another amount, customer, or resource is not a retry; it is an ambiguous request. Returning the first result would silently apply the wrong intent, while accepting the second breaks deduplication.

The server should persist a canonical request fingerprint with the key and compare it before returning the stored result. Canonicalisation must make equivalent JSON representations equal while keeping semantically meaningful differences visible.

A 409 is a safety response. The client must generate a new key for a new operation and reuse the old one only to recover an unknown outcome.

Causes, most common first

1Client reuses a key for a new business action. A UI or job retries with a stale key.
2Fingerprint uses raw JSON. Equivalent serialization differs by ordering or whitespace.
3Key scope is too broad or too short. A key collides across tenants or after retention expires.

When you see it

A retry receives 409 after a client reused a key
Whitespace or field-order differences produce unexpected conflicts
Duplicate charges appear when keys are generated per attempt
The idempotency record stores a result but no request hash

How to diagnose it

Step 1

Compare canonical fingerprints

Inspect tenant, endpoint, key, and stored digest; never log secrets or full payment data.

SELECT tenant_id, endpoint, idempotency_key, request_hash, status FROM idempotency_keys WHERE idempotency_key = :key

Step 2

Replay equivalent JSON

Test field order, omitted defaults, and numeric representations against the canonicalizer.

Step 3

Trace key generation

Find whether the key is created per business action or per HTTP attempt.

The fix

Scope keys by tenant and operation endpoint.

Canonicalise the validated request and store a cryptographic digest with the key.

Return the original result for the same digest; return 409 for a different digest.

Generate a new key for a new operation, never as a reaction to a timeout on the old one.

Retain records for at least the maximum retry and reconciliation window.

digest = sha256(canonical_json(validated_body)).hexdigest()
record = store.claim(key, tenant, digest)
if record and record.digest != digest: raise Conflict('key reused')

How to stop it coming back

Document key lifetime and scope
Test lost responses and reordered JSON
Alert on conflict rate
Never hash unvalidated or secret-bearing logs
Provide client libraries that persist keys across retries

Practise production debugging in a real repository

Reading about a failure and reproducing one are different skills. Gronex ships broken backend repositories with failing test suites that encode the real invariant, so you debug from evidence instead of memorising symptoms.

FAQ

Should the server accept the second body?

No. It cannot know whether the first operation committed, so accepting the second makes the key unsafe.

Can clients hash the body?

They can send a fingerprint, but the server must compute its own from validated canonical data.

What status should conflict use?

409 clearly indicates the key is bound to another request intent; include a safe explanation and no sensitive payload.

Exactly-once versus effectively-once

Other errors engineers hit next to this one

Full error and symptom index →