Brokers provide at-least-once delivery: "exactly-once" doesn't really exist in practice.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | // ====================================================================== // Q: A message is delivered twice. Or out of order. What happens? // // Brokers provide at-least-once delivery: "exactly-once" doesn't really exist in // practice. So the consumer must be idempotent. // - Natural idempotency where possible: "set status = Shipped" or an upsert, // instead of "add 1". // - Ordering: the queue doesn't guarantee order when there are multiple // consumers or retries. Solutions: Service Bus sessions (FIFO per SessionId = // OrderId), a Kafka partition key, or a version / sequence number in the // message, where the consumer ignores anything older than what it already has. // - Dead-letter queue: alert when it isn't empty, a tool for inspection and // replay, and a human who looks at it. // - Keep messages small: IDs and the necessary fields, not whole entities. For // large payloads: the claim-check pattern (blob storage plus a link in the // message). // ====================================================================== // Idempotent consumer with an inbox table: // unique index (MessageId, Consumer), in the same transaction as the change public async Task HandleAsync(OrderPlaced e, string messageId, CancellationToken ct) { if (await db.ProcessedMessages.AnyAsync(m => m.MessageId == messageId && m.Consumer == "erp-sync", ct)) return; // already processed await using var tx = await db.Database.BeginTransactionAsync(ct); db.ProcessedMessages.Add(new ProcessedMessage(messageId, "erp-sync", DateTime.UtcNow)); db.ErpExports.Add(new ErpExport(e.OrderId, ErpExportStatus.Pending)); try { await db.SaveChangesAsync(ct); await tx.CommitAsync(ct); } catch (DbUpdateException ex) when (ex.IsUniqueViolation()) // our own extension (e.g. PostgreSQL 23505) { // two instances got the same message concurrently: the unique index keeps only one } } |