What uptime means when your client retries

Two numbers, measured on the same twenty-three models on the same afternoon, 2026-08-31. Seventy-five per cent of first attempts came back with an answer. Ninety-three per cent came back within three attempts. Both are true, and a status page that publishes only one of them is misleading in one direction or the other.

Which one describes your users depends on a default you probably never changed.

Your SDK already retries

The official OpenAI and Anthropic client libraries retry failed requests automatically. Both ship a default of two retries — three attempts in total — with exponential backoff, and both apply it to connection errors, 408, 409, 429 and 5xx responses.

You can see the setting in each library’s own documentation: the OpenAI Python SDK and the Anthropic Python SDK both expose max_retries and both default it to 2. The equivalent Node libraries do the same.

Almost nobody changes it. Which means that for almost every application built on these APIs, a single refused request is not an error the user ever sees — it is a retry, a short pause, and an answer. The failure exists in the provider’s logs and nowhere in the customer’s experience.

What the gap actually was

Measured across the full catalogue on 2026-08-31, while the platform was shedding load under real traffic:

Eighteen points. The refusals were overwhelmingly 503 with an overload message, arriving in under half a second, and clearing on the next attempt. One case we traced by hand: a model returned 503, and the identical request two seconds later returned 200.

Publish only the 75% and you tell customers the service fails one call in four, which no user of a standard client would recognise. Publish only the 93% and you have quietly written off the load-shedding as though it did not happen. So publish both, side by side, and label them: what the platform did on first contact, and what a correctly built client ended up with.

What to retry, and what never to

Retrying is not free of judgement. The distinction that matters is whether the condition can clear.

The second half of that table is the half people get wrong, and getting it wrong is worse than not retrying at all. Retrying a 401 three times turns the customer’s own typo into three seconds of apparent instability, and they will report it as your service being flaky. The same is true of 404: on an OpenAI-compatible endpoint that is very often a base URL with one path segment too many, which is a configuration mistake that retries can only disguise.

One more that looks transient and is not: a request that hangs. A model holding the connection open until your timeout expires will usually do exactly the same thing on the retry. Three attempts against a hanging endpoint is three timeouts — minutes, not seconds — and at the end of it you know nothing you did not know after the first.

Retrying is not the same as hiding

There is a version of this argument that becomes an excuse, and it is worth naming. If a platform refuses one call in four and you report only the retry-inclusive number, you have used a real technique to bury a real problem. The refusals are costing your users latency even when the retry succeeds, and if the rate climbs the retries stop covering it.

Three habits keep the technique honest:

1. 01Record both outcomes on every sample. Not one derived from the other — the first-attempt result and the final result, stored separately, so neither can be reconstructed away later.
2. 02Say when a retry was needed. An API playground that answers on the third attempt should say so. "It worked" and "it worked eventually" are different facts about a service.
3. 03Bound it. Three attempts is the SDK default and therefore the honest ceiling. A probe that retries ten times is measuring its own persistence.

The counting rule follows from this: a refusal your probe provoked is not a data point, and a refusal a customer would also have met is. If your monitor sends traffic no customer sends, you cannot tell those apart — which is a different failure, covered in the article on probes that measure themselves.

What to tell customers

If you are reselling or relaying model access, the practical advice to hand your users is short:

• Use the official SDK and leave max_retries alone. You get correct backoff and correct classification for free.
• If you are calling with raw curl or fetch, you have no retries at all. Add two, with roughly one and two seconds of backoff, for connection errors and 5xx only.
• Set a client timeout well above the model’s tail. Some models answer in two seconds when streaming and thirty when not — a short timeout turns that into an error your retries cannot fix.
• Never retry a 4xx other than 429. If a request is wrong, it is wrong three times.

And the number to look at on any provider’s status page is whichever one they are willing to show you next to the other. A single availability figure with no indication of how many attempts it allows is not comparable to anything.

Where to go next

• AI API relays, explained — the layer where retries, routing and rate limits actually happen.
• When your probe becomes the outage — the three ways a monitor measures itself, and the one test that separates your load from their fault.
• What “OpenAI-compatible” actually means — including the base-URL mistake behind most of the 404s above.

Both of our numbers are on the status page: the headline figure allows retries, because that is what your client does, and the first-attempt figure sits beside it wherever the two differ.