Exactly-once delivery in practice

The impossibility, stated precisely

A sender that transmits a message and receives no acknowledgement cannot distinguish three cases: the message never arrived, it arrived and was processed but the acknowledgement was lost, or it is still in flight. No protocol resolves this, because any acknowledgement is itself a message that can be lost. The sender must choose to resend (at-least-once) or not (at-most-once). There is no third option at the delivery layer.

What vendors mean when they say exactly-once

• Kafka transactions coordinate consumer offsets and producer writes atomically *within Kafka*. Read a message, write a derived message, commit the offset - all or nothing. Genuine, and bounded to Kafka's own storage.
• Stream processors checkpoint operator state and input positions together, so a restart resumes consistently. Again genuine, and bounded to the processor's state store.
• Neither extends to your side effects. The moment the consumer calls a payment API or writes to an unrelated database, the transaction boundary ends and at-least-once resumes.

The dual-write problem

The common mistake: write to the database, then publish an event. A crash between the two leaves the database updated and no event published, or the reverse if the order is swapped. There is no ordering of two independent writes that is crash-safe, which is why this needs a structural fix rather than careful coding.

The outbox pattern

• Write the business change and a row in an `outbox` table in the same local transaction. One commit, so they cannot diverge.
• A separate relay reads unpublished outbox rows and publishes them to the broker.
• The relay marks rows published after the broker acknowledges - and if it crashes between publishing and marking, it republishes. That is at-least-once, by design.
• Consumers deduplicate on the event ID, which converts at-least-once delivery into exactly-once effect.

Change-data-capture is the same idea with the database's replication log as the outbox, which avoids a separate table at the cost of operational complexity.

What to build, concretely

• Assume duplicates everywhere; design consumers so reprocessing is a no-op.
• Give every event a stable ID at creation, not at publish time.
• Make the dedup check a unique constraint, committed with the effect.
• Where the effect is external, use that provider's idempotency key - most payment APIs offer one precisely for this.
• Monitor for duplicates rather than assuming they are absent; a counter of suppressed duplicates tells you the mechanism is working.

The implementation detail of keys and windows is in idempotency and deduplication, and the broker specifics in Kafka interview questions.

Where CoPilot Interview fits

Disclosure: we build an interview assistant, so treat this as a disclosed pitch. CoPilot Interview is a native Windows and macOS app that puts suggested answers on your own screen in about two seconds, across coding, system design and behavioral rounds, with a permanent free tier that needs no credit card.

Frequently asked questions

Is exactly-once delivery possible?

Not across a network boundary. A sender receiving no acknowledgement cannot tell whether the message was lost, processed with a lost acknowledgement, or still in flight, and any acknowledgement can itself be lost. The choice is at-least-once or at-most-once; exactly-once effect is achieved above the delivery layer.

What does Kafka's exactly-once actually guarantee?

Atomic coordination of consumer offsets and producer writes within Kafka itself - read, transform, write and commit as one unit. It does not extend to side effects outside Kafka, so a consumer calling an external API is back to at-least-once and must be idempotent there.

What is the dual-write problem?

Writing to a database and then publishing an event as two independent operations, where a crash in between leaves them inconsistent in one direction or the other. No ordering of the two is crash-safe, which is why the outbox pattern commits the business change and the event row in a single local transaction.

How does the outbox pattern work?

The business change and an outbox row are written in one local transaction, so they cannot diverge. A separate relay publishes unpublished rows to the broker and marks them afterwards, republishing if it crashes in between. Consumers deduplicate on event ID, turning that at-least-once delivery into exactly-once effect.