Migrating the American Express Payment Network, Twice

themafia

This is cool, but a part of me has to be flip, and humanity is kind of hilarious to me. All of this amazing infrastructure and hard work to essentially just very carefully and accurately subtract one number from another in a way that makes auditing easy.

e-dant

A fascinating en dash, used as an em dash, followed by two em dashes. Hmmm.

alberth

Re: transition to micro services (from monolith). I’m surprised a network so sensitive to latency (as are payment networks), was able to achieve their latency SLAs with micro services. Maybe Amex being a closed-loop network helps with latency?

mitchellh

Cmd+F "Kubernetes". Oh Jesus Christ.

3x35r22m4u

Norton Antivirus for Android doesn't allow me to access the site. Hmm.

ivlozada

The GTR design is smart — minimal protocol understanding, just enough to route. The temptation in payment systems is always to push more logic to the edge. Keeping the router thin is what made the canary ollback trivial. The shadow traffic approach is the real insight here. You can't test a payment system with synthetic data — the edge cases are in the real transaction patterns. Replaying production traffic against a parallel system and diffing the results is the only way to know if you've achieved parity. Their closing lesson — "patience and discipline, success is measured in reliability" — applies to any system where transactions are irreversible.