Emmanuel DonkorCOMPUTER SCIENCE
Menu
← WritingNOTE

HTTP · RELIABILITY · DISTRIBUTED SYSTEMS

Idempotency is a boundary, not a delivery guarantee

KEY TERMS idempotency keys / request fingerprints / PostgreSQL uniqueness constraints / transactional outbox

A duplicate request is not an edge case. It is what happens when the network stops being polite.

A client sends POST /orders. The server creates the order, but the response disappears on the way back. From the client’s side, there are two believable stories: the server never saw the request, or it created the order just before the connection died. Retrying is reasonable. Creating a second order is not.

That is the job an idempotency key can do well. It gives one logical request an identity that survives a retry. It does not make every operation downstream of that request happen once. That distinction is the whole point.

First, make the request recognisable

An Idempotency-Key is supplied by the client and lets the server recognise a later attempt as a possible retry. The IETF document describing the header says a key must be unique and must not be reused with a different payload. It also leaves the resource owner responsible for the key’s lifecycle and any expiry policy. The document is an Internet-Draft, not a final RFC, which is worth remembering when people describe the header as a universal API rule. Read the draft.

On the first request, a service can store the key, a fingerprint of the meaningful request data, and the result it produced. On a later matching request, it can return that result instead of doing the work again.

That is a narrow promise, but it is useful: this request identity has already produced this outcome here.

The key alone is not enough

Treating the key as the whole story creates an ugly failure mode. Imagine that a client accidentally reuses the same key for two different payloads:

Idempotency-Key: 7b3...
POST /payments { "amount": 20 }

Idempotency-Key: 7b3...
POST /payments { "amount": 200 }

The second request is not a retry. It is a conflict. Returning the first response without noticing the changed amount is misleading; processing the second one is worse. A payload fingerprint gives the service something to compare. If the key matches but the meaningful request data does not, the correct response is to reject the ambiguity rather than guess.

The fingerprint does not need to be mystical. It can be a stable hash of a canonical request body, or of the fields that define the logical operation. What matters is that the API documents what it considers the same request.

The race lives below the HTTP handler

The easy implementation looks like this: look up the key; if there is no row, create one; otherwise replay the old response.

That fails when two copies arrive at almost the same time. Both handlers can look up the key before either has saved it. Both then believe they are first.

This is where the database must own the invariant. A unique constraint on the stored idempotency key makes the second insert fail instead of silently creating a second record. PostgreSQL describes a unique constraint as enforcing uniqueness across the rows covered by the constrained columns; it also creates a unique B-tree index for that constraint. PostgreSQL constraints.

The application still has work to do after that failure: it should reload the stored record and decide whether the matching request is a replay or a payload conflict. But the database constraint is what prevents two optimistic HTTP handlers from both winning.

The boundary ends at the local commit

Suppose the service has now committed an order and stored the idempotency record. It still needs to notify another system. The next line might publish a message, call a payment provider, or send an email.

This is where a lot of “exactly once” language becomes too casual.

The database can atomically commit its own rows. It cannot, by itself, atomically commit a row and confirm that an independent broker has accepted a message. If the process crashes after the database commit but before the publish, the order exists and the notification does not. If it crashes after the broker accepts the message but before the service records that success, a retry may publish a duplicate.

A transactional outbox fixes the first gap cleanly. The service writes the business record and a pending event row in the same database transaction. A separate dispatcher forwards committed event rows later. That prevents the service from claiming a database change without retaining the event that should follow it. It does not mean a consumer will only ever see the event once. AWS’s description of the pattern makes the same distinction: standard queue delivery can repeat an event, so downstream handling must be idempotent too. Transactional outbox pattern.

“Exactly once” is not one switch

There is no single header or table column that turns a distributed workflow into exactly-once processing. There are several boundaries, each with its own question:

An idempotency key answers the first question. A unique constraint helps with the second. An outbox addresses the third. The rest still belongs to delivery and consumer design.

Calling all of that “idempotency” hides the interesting parts. The useful habit is to name the boundary being protected and the failure that still sits outside it.

A practical checklist

For a create-style endpoint that may be retried:

None of this is glamorous. That is partly why it works. A good idempotency design does not promise that failures disappear; it leaves fewer places for a retry to quietly turn into new work.

Related implementation

R01: duplicate-safe workflow service is a local implementation exploring request replay, a uniqueness race, and an outbox retry boundary. Its recorded results are separate from the sources and argument in this article.