API design
Webhook design: interview questions and practical design
Webhook design delivers state changes to another system over an unreliable boundary, so authenticity, retries, ordering, and duplicate effects must be part of the contract.
Written and reviewed by Sahil Srivastav
What it actually is
Webhook design delivers state changes to another system over an unreliable boundary, so authenticity, retries, ordering, and duplicate effects must be part of the contract.
The receiver may be down, slow, or deployed between attempts. At-least-once delivery is safer than silently dropping an event, but it requires an idempotent consumer.
The useful interview answer is precise about the boundary: Give every event a stable id, type, version, occurred-at time, and resource reference. The receiver stores the id before applying an effect or uses a guarded transaction.
Why it matters in production
The receiver may be down, slow, or deployed between attempts. At-least-once delivery is safer than silently dropping an event, but it requires an idempotent consumer.
A webhook is an integration API: signature rotation, event versioning, replay, and observability determine whether users can recover from a missed delivery.
How it works
Event identity and payload
Give every event a stable id, type, version, occurred-at time, and resource reference. The receiver stores the id before applying an effect or uses a guarded transaction.
Signing and replay defence
Sign the raw body with a rotating secret and include a timestamp. The receiver verifies the signature before parsing and rejects stale timestamps within a configured tolerance.
Retry and delivery state
Use bounded exponential backoff, record attempts and responses, and provide a dead-letter or replay path. A timeout is not evidence that the receiver did nothing.
Implementing it
Design a signed event and receiver deduplication table.
Replay an event after the consumer commits but before acknowledgement.
Document ordering guarantees and how a consumer reconciles missing events.
Interview questions and how to answer them
How do you make webhook delivery reliable?
Persist the event before sending, retry with a bounded policy, track delivery state, expose replay, and make consumers idempotent by event id.
Can webhooks guarantee ordering?
Only within a defined scope and transport design. If ordering matters, include sequence numbers and have consumers detect gaps rather than assuming network order.
How should signatures be verified?
Use the raw request body, timestamp and key id from headers, constant-time comparison, and a rotation window. Parse only after authentication succeeds.
Answers that lose the round
- Assuming a 2xx acknowledgement means the business effect completed later.
- Signing parsed JSON instead of the exact transmitted bytes.
- Retrying forever without a dead-letter or operator signal.
- Embedding secrets or mutable state that makes events impossible to replay safely.
FAQ
Should a webhook sender retry 4xx?
Usually retry only responses that indicate a transient receiver or transport problem. A 400 needs correction or replay after the receiver is fixed, not rapid automatic retries.
What if the receiver processes then times out?
The sender may retry; the receiver must deduplicate and return the same success outcome once it recognises the event.
How do I handle a missed event?
Offer replay by id or time range and a reconciliation endpoint that lets consumers compare current resource state.