Skip to content

Messaging idempotency

An application assigns one stable hierarchical ID to each logical command. Durable Messaging uses that ID throughout admission, retries, completion, diagnostics, and recovery. The receiving inbox’s retained completion record recognizes repeated submissions of that command, from any grain.

For the commit protocol, see Messaging. For complete grain examples, see Practical recipes.

IdentityOwner and scopePurpose
Envelope MessageIdExact key in the receiving grain’s inboxCoalesces pending repeats and recognizes completed commands during retention.
Envelope MessageIdExact key in the sending grain’s outboxBinds one pending outgoing message to its destination, subject, and body.
Workflow rootThe application’s protocolGroups related command IDs and deterministic reply IDs.
External idempotency keyThe external system’s account or API namespaceRecognizes a retried external side effect using the canonical command ID.

Derive a command ID from stable tenant, entity, and action identifiers before sending. Every retry and reconstructed submission uses the original ID. Namespace independent producers explicitly. A forwarding grain preserves the original command ID. A new workflow command uses a distinct child or sibling ID; fan-out derives one stable recipient child per destination.

One key identifies one immutable command: destination, subject, and body retain their original meaning. A genuinely new reservation, authorized compensation, or revision uses a new key. Protocol upgrades preserve the identity of an existing command even when producers and consumers are deployed at different times.

For deliveries, the inbox answers a tracked identity from its pending or completed record, whatever the delivered SentAt, including after a receiver clock rollback. An untracked identity whose send time is at or below the admission floor receives Expired. Above the floor, the inbox bounds an untracked identity’s SentAt by the receiver’s Durable Jobs time plus DeduplicationWindow; equality is admitted, and a later time throws ArgumentOutOfRangeException with no staged change or sink offer. The sender retries within its original budget, so diagnose clock skew or an incorrectly stamped direct caller.

A grain’s messages to itself use its local staging time, or the first time above the admission floor after a backward clock jump. The latter remains just beyond the floor for compaction, even when it exceeds the delivery-time bound.

Each receiver checks the exact MessageId, which identifies one command at that receiver regardless of sender or subject. A parent and its child are independent commands. A pending duplicate refers to the already-accepted work and carries the same subject, body bytes, and DeliverAt time; a pending repeat with a different subject, body, or DeliverAt fails. A completed duplicate returns Duplicate while its completion record is retained. Pending and completed records retain the ID and the SentAt which the inbox accepted. A repeat keeps that accepted time, and the record is durable before the receiver answers Duplicate or repeats its recorded rejection. Applications uphold the immutable command contract; the completion fact acknowledges the original execution.

A handler stages its business changes and outgoing messages, then calls Complete, which stages the inbox removal and completion record. One journal write persists them together, and replay restores the business effects and completion record together. Effects therefore apply once per logical send. A handler can execute more than once for the same ID, for example after a failed attempt or activation loss, so its preparation must tolerate repetition.

The sending journal write records SentAt from the Durable Jobs clock. Every attempt and recovery uses that same time, including after backpressure, and the retry budget ends at SentAt + OutboxRetryTimeout. Configure OutboxRetryTimeout shorter than DeduplicationWindow. Each inbox journals a non-decreasing admission floor. The retention period DeduplicationWindow controls how far behind the observed Durable Jobs time the floor advances. Completed records become eligible for removal once their accepted SentAt is at or below that durable floor. Snapshot acknowledgement and recovery reclaim them. A delayed delivery with that original send time then receives Expired before any sink or handler runs. Forward clock jumps can expire sends early; backward jumps can retain history longer. Budget history by processing rate and retention. Application result-query retention serves business query and audit needs independently.

Expired means that earlier acceptance is unknown: an acknowledgement may have been lost before the receiver compacted its completion record. Reconcile business state using the original ID before authorizing another command. Automatic outbox retries and recovery preserve the original SentAt; direct delivery callers likewise reuse it. A new outbox submission after retirement receives a new staging time. A fresh submission after an identity’s retention ends can be processed again when its new send time is admitted; reconcile the original outcome before authorizing it.

The outbox coalesces equivalent messages that share a pending message ID, whether staged or durable. Equivalent messages have the same destination, subject, payload bytes, and DeliverAt time; the original message and its headers are kept. Reusing a pending ID for different content throws and leaves the original message in place.

A message a grain sends to itself is keyed only in that grain’s inbox. Send applies the receiver rules synchronously: a pending repeat with the same subject, body bytes, and DeliverAt time coalesces, a retained completed ID is a duplicate, and a pending repeat with different content throws. A local send receives a staging time which the inbox admits, including after a clock rollback. A local repeat of a tracked identity keeps the record’s accepted send time.

MessageId identifies a message by wrapping a HierarchicalKey. Construct it with new MessageId(key) or Create, and read the key through Key. Message IDs compare, hash, parse, and format through that key.

HierarchicalKey is an immutable value type wrapping its canonical escaped path. Equality, ==, !=, and hashing compare that path ordinally and case-sensitively. The key provides segment-aware navigation through GetParent, IsParentOf, IsChildOf, IsAncestorOf, and a segment enumerator. default(HierarchicalKey) is the unset value, exposed by IsDefault; envelope validation requires a set ID.

A useful hierarchy is:

tenants/{tenant}/orders/{order}
inventory/{sku}/reserve
inventory/{sku}/release
payment/charge
payment/refund/{refund}
shipment/create

The root groups a workflow; each leaf identifies one effect. Retried requests keep the same leaf. Assign a stable attempt/revision segment once when authorizing a new effect. Format numeric components invariantly and normalize business identifiers before construction. Key comparison preserves their ordinal spelling.

Create accepts literal segments, and CreateChildKey appends one literal segment. Both escape slash and backslash characters exactly once. Construct payment/charge using separate "payment" and "charge" arguments. Parse reads an already escaped path, and formatting emits that canonical representation.

The following helper constructs command IDs using literal tenant and SKU identifiers:

public static class OrderOperationKeys
{
public static HierarchicalKey Order(string tenantId, Guid orderId)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tenantId);
if (orderId == Guid.Empty)
{
throw new ArgumentException("An order ID must be nonempty.", nameof(orderId));
}
return HierarchicalKey.Create("tenants", tenantId, "orders", orderId.ToString("N"));
}
public static MessageId Reservation(HierarchicalKey order, string sku)
{
ArgumentException.ThrowIfNullOrWhiteSpace(sku);
return new MessageId(order.CreateChildKey("inventory").CreateChildKey(sku).CreateChildKey("reserve"));
}
public static MessageId Charge(HierarchicalKey order) =>
new MessageId(order.CreateChildKey("payment").CreateChildKey("charge"));
}

The SKU widget/blue occupies one literal segment:

internal static class HierarchyExample
{
internal static (HierarchicalKey Order, HierarchicalKey Step) Create()
{
var order = HierarchicalKey.Create("orders", "42");
var step = order.CreateChildKey("inventory")
.CreateChildKey("widget/blue")
.CreateChildKey("reserve");
// orders/42/inventory/widget\/blue/reserve
return (order, step);
}
}

Use the same canonical identity across producers, inboxes, logs, result queries, and external provider keys. IsAncestorOf matches at segment boundaries: orders/42 groups orders/42/payment, while orders/420/payment belongs to another order. A key is its own ancestor; IsChildOf tests exactly one additional segment. Deduplication always compares the full key.

Trusted ingress derives the authorized tenant/entity root and checks the claimed command key, subject, and destination against it before accepting work. A key expresses identity; authorization comes from that trusted application boundary. Sending and delivery limit a message ID’s canonical path to 1,024 UTF-8 bytes and a subject to 256 UTF-8 bytes.

A handler can stage a durable reply whose ID is the command’s stable result child. The inventory recipe computes its reservation decision and next stock locally. Typed SendReply encodes and stages the reply, then the method updates stock and completes synchronously. A shortage is a completed rejection with unchanged stock. The originally committed outbox intent supplies result delivery within its configured policy. A resubmission acknowledges the retained completion fact and preserves that original execution.

An inbox Duplicate acknowledges pending or completed command identity. Obtain the business result from the durable reply, or query business state keyed by the same command ID. A distinct query command can request a result after the original reply’s delivery policy ends.

Store reservation records when lookup, release, or expiry needs them, payment results for status and reconciliation, and workflow phases for progress queries. These are business state with explicit query/audit retention. Commit them together with local effects, outgoing intent, and inbox completion. The payment recipe exposes a result query by the original command ID.

An external payment, email, or HTTP operation has its own commit boundary. An activation can fail after that operation succeeds and before its result is journaled. The payment recipe calls its provider during local preparation with context.Message.Envelope.MessageId.ToString(). The provider enforces request consistency and returns its original result on retries with that key.

Specify the provider account/tenant namespace, key format, retention, conflict behavior, and outcome-query API as part of that integration. Retain provider idempotency facts for every retry or replay which can reach it. For a provider’s key-length restriction, an adapter can deterministically hash the canonical ID and retain the original command ID locally for correlation. Reconcile an ambiguous response or retry with the original key. An API with an uncertain outcome requires its reconciliation workflow before another effect is authorized.

Once the result is known, observe cancellation, send the typed reply, and finish the journaled result and Complete() without awaiting. Every asynchronous provider call precedes the first shared mutation.

Give each participant or step its own command ID under the workflow root. One live outbox key maps to one destination, so notifications use stable recipient-specific children. A coordinator records submitted steps and aggregates correlated outcomes in its own journal. Participants commit their effects and deterministic replies atomically; the coordinator commits each received outcome and next-step intent together.

Use a durable step status such as Requested, Succeeded, Rejected, or Compensated with the participant outcome. A rejection can emit separately keyed release or refund commands for completed steps. Each participant completes that compensation under its own exact key. This gives the workflow an explicit forward and recovery path across independently committed grain journals.

OperationApplication policy
Reserve stock or charge a paymentPreserve the command ID through retries and retain its inbox completion fact for the supported horizon.
Replace a projection with a complete snapshotApply only a newer domain version; complete older snapshots as superseded.
Apply every incremental eventPersist the expected sequence and buffer gaps before applying contiguous events.
Intentionally reprocess with corrected logicAuthorize a new revision key and record its relationship to the original outcome.
Replay an inbox dead letterReconcile effects keyed by the original ID, then authorize a new revision key, such as a child of the original; a retained completion answers Duplicate, and an untracked old send time answers Expired.

Message identity recognizes completed commands; a domain version or sequence establishes order. The projection recipe demonstrates a complete-snapshot policy.