What bank and payment integrations taught me about resilience

Most of my time at Işıksoft goes into B2B e-commerce — and a big slice of that is talking to systems I don’t control: banks, payment gateways, ERPs. If there’s one thing those integrations beat into you, it’s that the happy path is the smallest part of the job.

Money can’t be retried blindly

The first rule: a network timeout does not mean the charge failed. The bank may have processed it and your request just died on the way back. So you can never naively retry a payment.

The fix is idempotency keys — a unique token you generate per attempt and send with the request. If you retry with the same key, the provider returns the original result instead of charging twice.

async function charge(orderId: string, amount: number) {
  const key = idempotencyKey(orderId); // stable per order attempt
  return gateway.charge({ amount, idempotencyKey: key });
}

On your side, store the key and the outcome before you call out, not after. If the process crashes mid-flight, the record tells you what state you were in.

Webhooks are the source of truth, not the response

The synchronous response tells you what the gateway thinks right now. The webhook tells you what actually settled. I treat the HTTP response as a hint and the webhook as the truth — and I always:

• Verify the signature. An unsigned “payment succeeded” callback is just a stranger asking you to ship goods for free.
• Make the handler idempotent. Providers retry webhooks. Receiving the same event three times must not create three orders.
• Return 200 fast, process async. Do the heavy work in a queue, not inside the webhook request.

Reconcile, don’t trust

Even with all that, states drift. A nightly reconciliation job that compares your records against the provider’s settlement report catches the cases your real-time flow missed. It’s boring code that saves you from very un-boring phone calls.

None of this is glamorous. But payment code is the one place where “it works on my machine” is genuinely dangerous — and getting the failure paths right is the whole job.