Idempotency: The #1 Rule of Safe Integrations Teams Ignore

Idempotency: the #1 rule of safe integrations that 90% of teams ignore

Most integration outages don’t start with downtime. They start with duplicate requests.

Same payload. Same endpoint. Sent twice.

And suddenly you have:

• double orders,
• duplicated payments,
• inventory chaos,
• angry business teams,
• and a Slack channel that won’t stop blinking.

This article is about why idempotency is the most ignored integration rule, and how real systems fail without it.

The disaster nobody expects

Most teams don’t wake up one morning and say:

“Let’s design a non-idempotent integration.”

They ship something that works — until reality interferes.

Let’s look at real cases.

Case #1: Double order creation (eCommerce → ERP)

Context: Adobe Commerce → ERP (SAP / Infor / Epicor-style system)

Flow:

• POST /orders
• ERP returns 200 OK

What actually happened:

1. Magento sends POST /orders
2. ERP processes the order successfully
3. Network hiccup happens after processing but before response is received
4. Magento retries the request
5. ERP creates the same order again

Result:

• Same business order, different internal IDs
• Inventory reserved twice
• Finance sees duplicate revenue
• Manual cleanup required inside ERP

❗ The integration worked perfectly — twice.

No errors. No exceptions. Just silent duplication.

Case #2: Payment captured twice

Context: eCommerce → Payment Provider → Webhook

Endpoint:

POST /payment-confirmed

What went wrong:

1. Payment provider sends webhook
2. Your system processes payment
3. Temporary DB issue causes 500 response
4. Provider retries webhook (as designed)
5. Payment is captured again

Impact:

• Customer charged twice
• Refunds and chargebacks
• Support tickets
• Trust permanently damaged

Root cause: 👉 Webhook handler was not idempotent.

Case #3: When “at-least-once” events burned the warehouse

Context: ERP → Message Broker → eCommerce (event-driven inventory updates)

The ERP publishes inventory update events. The broker guarantees at-least-once delivery.

Sounds safe.

It isn’t.

The event

{
  "eventId": "evt-739182",
  "sku": "ABC-123",
  "available": 5,
  "timestamp": "2025-01-14T10:22:31Z"
}

Consumer logic:

“When an event arrives — update inventory.”

No event tracking. No idempotency.

What actually happened

1. ERP publishes inventory event
2. Consumer receives it
3. Inventory updates correctly
4. Consumer crashes before ACK
5. Broker re-delivers the same event
6. Inventory update runs again

No error. Same event. Same payload.

The real damage

The inventory update triggered:

• Re-indexing
• Cache invalidation
• Backorder recalculations
• Stock alert notifications

Because the same event was processed twice:

• Customers received duplicate “Back in stock” emails
• Indexers ran unnecessarily
• Storefront slowed down under peak load

Nothing was wrong.

Everything was duplicated.

Why this was dangerous

• No negative stock
• No financial mismatch
• No alerts

Just:

• Extra load
• Confused business users
• Hard-to-explain side effects

This kind of bug survives QA and only shows up in production.

Why retries are unavoidable

Many teams believe:

“We don’t retry requests, so duplicates won’t happen.”

That’s a myth.

Retries come from:

• HTTP clients
• API gateways
• Load balancers
• Message brokers
• Webhook providers

If your integration assumes exactly-once delivery, it is already broken.

What idempotency actually means

Idempotency means:

Processing the same request multiple times produces the same result as processing it once.

Not “almost the same”. Not “should be fine”.

Exactly the same.

How to implement idempotency (real patterns)

1. Idempotency keys

POST /orders
Idempotency-Key: 8f3a9c2e-41f1-4c4b-a9f9-0e12d4c8b111

Server rules:

• Store key + result
• If key already exists → return previous response
• Never re-run business logic

Used by Stripe, PayPal, and every serious payment API.

2. Natural business keys

A very common real-world example is payments and transactions.

When a customer submits a payment, you usually create two entities:

• an Order
• a Transaction

The transaction has a unique transaction number generated by the payment system.

This number already represents a single real-world action: "capture this payment".

How idempotency works here

1. When the transaction is first submitted, it is created with a status like:
   • initiated
   • processing
2. The transaction number is stored and enforced as unique
3. Payment webhooks include this transaction number in metadata (most platforms support this)
4. If the webhook is delivered again, the system finds the existing transaction
5. The handler exits safely without creating or capturing anything again

Result:

• Only one capture is possible
• Duplicate webhooks become harmless replays
• No double charges

Rules

• Use business-generated identifiers (transactionNumber, externalOrderId, erpDocumentId)
• Enforce uniqueness at the database level
• Treat duplicates as replays, not errors
• Return a safe success response

Never rely only on:

“Check if it exists, then insert.”

That logic fails under concurrency.

Two requests can pass the check at the same time. Only the database can arbitrate truth.

Rule of thumb: if a business key represents a real-world transaction, the system must allow it to be completed only once — no matter how many times the request or webhook arrives.

3. Idempotent handlers

This is where most event-driven systems break — not because of bugs, but because of delivery guarantees.

In enterprise systems, message brokers such as Kafka, RabbitMQ, SQS almost always provide at-least-once delivery. This guarantees that a message will be delivered, but it also means the same message can be delivered more than once.

A typical message handler effectively runs like this:

read message
begin database transaction
update business data
commit transaction
acknowledge message

The dangerous gap is between commit and acknowledge.

If the handler crashes, the process restarts, or the broker loses the ACK, the broker will redeliver the same message — even though the database transaction already succeeded.

That’s how the same business logic can execute multiple times for a single message.

Why naive handlers fail

Handlers are often written as state mutations instead of state replacements.

Bad (non-idempotent):

quantity = quantity - 5

If this message is delivered twice:

• the database transaction runs twice
• the same business effect is applied twice
• the system slowly drifts into an incorrect state

No exception is thrown. No error is logged.

The system is simply wrong.

What idempotent handlers look like

Good (idempotent):

quantity = 5

This handler describes the desired end state, not the transition.

If the message is replayed:

• the database ends up in the same state
• side effects are not multiplied
• the system remains correct

Making handlers idempotent in practice

There are two common strategies:

1. Track processed message IDs

• Store messageId / eventId in the database
• Enforce uniqueness (unique index or conditional write)
• If the ID already exists → treat the message as a replay and exit safely

This can be implemented:

• in a dedicated PROCESSED_MESSAGES table (SQL)
• inside the business entity itself (common for NoSQL)

2. Make updates naturally idempotent

• Prefer state replacement over mutation
• Use absolute values instead of deltas
• Store the highest processed event version per aggregate

Rule of thumb: assume every message will be delivered more than once.

If processing the same message twice changes the outcome, the handler is not production-safe.

A rule worth remembering

If your integration cannot safely process the same request twice, it is not production-ready.

Idempotency is not an optimization. It is a safety guarantee.

Key takeaways

• Duplicate requests are normal
• Retries are unavoidable
• Idempotency must be designed intentionally
• Payments, orders, and inventory must be idempotent
• If you ignore this, production will teach you — painfully

In the next article, we’ll talk about retry patterns that don’t destroy systems.

Because retries without idempotency are a loaded gun.