When your uptime probe becomes the outage

On 2026-08-31 our own status page reported that most of a twenty-three model catalogue was unstable. Customers were sending requests to those same models and getting answers. The page was not lying — it was faithfully reporting what its probe measured. The probe was measuring itself.

This happened three separate times, with three different causes, and each time the first instinct was to go looking for a broken upstream. Each time the answer was in our own code. If you are building a monitor for an API you do not control, these are the three ways it will lie to you, in the order we met them.

Catalogue probed

23 chat models

Peak false failure rate

~65% of models shown unstable

Actual cause, all three times

the probe’s own request pattern

Measured

2026-08-30 to 2026-08-31

One: concurrency against a single model

The first version probed models in parallel, which is the obvious way to keep a sweep short. It reported six models as intermittently failing.

Measured on 2026-08-30, one model at a time:

Then the same test with the four concurrent requests spread across four different models: twelve of twelve. The ceiling was per-model, not global. Our probe had been sending two requests to each model within milliseconds of each other — a chat completion and a streaming call — and roughly a quarter of the second ones were being refused.

The lesson generalises past this one platform. A rate limiter is a real part of the service, but it is scoped to something: a key, a model, an account, a route. Until you know what it is scoped to, a parallel probe is measuring the limiter and calling it availability.

Two: a timeout shorter than the upstream’s own tail

The second failure looked completely different and was the same category of error. A model was published as 25% available. It was answering every request; the probe was hanging up before the answers arrived.

The timeout was 20 seconds. That model’s median response was 16. Raising the cutoff to 45 seconds fixed that model and immediately broke two others, and the recorded samples showed why. Their non-streaming latency was bimodal — measured on 2026-08-30, four consecutive calls with max_tokens: 8 took 4.0s, 11.9s, 40.3s and 19.0s — while streaming time-to-first-byte on the identical prompt stayed between 1.4s and 2.6s every time.

The completion was waiting on reasoning the caller never sees. Capping max_tokens does not shorten it, because the tokens being generated are not the ones being counted.

That is a real and useful fact about those models. It is a fact about latency, and a cutoff below the upstream’s own tail converts it into a fiction about uptime. The two belong in different columns, and a monitor that cannot keep them apart will keep publishing slow as down.

Three: the sweep that was one long burst

The third one is the one that produced the worst page. The probe woke once an hour and walked the whole catalogue — twenty-three models, two requests each, four hundred milliseconds apart. From the platform’s side that is not twenty-three small checks. It is one continuous stream of roughly forty-six inference requests from a single key, followed by fifty-seven minutes of silence.

No real client behaves that way, and the platform answered the pattern with 503 system_cpu_overloaded. The page published the result as most of the catalogue being unstable, while every customer sending occasional requests got answers.

What made this one expensive was the diagnosis. We tested the same request from four different entry points — the storefront domain, the platform directly, and two separate regional hosts — and reproduced the 503 at all four. That looked conclusive, and it proved nothing at all, because every test used the same key and the limit follows the key rather than the hostname.

How to tell your load from their outage

Four checks, in the order that costs least:

1. 01One request, after real silence. The single most informative test, and the one we kept skipping because it feels too simple to be worth running.
2. 02Non-inference paths. Ask for the model catalogue, or make a deliberately unauthenticated call and read the 401. Measured on 2026-08-31 those answered in 0.27–0.45s while the inference path was returning 503, which located the problem in model capacity rather than in the platform’s web tier.
3. 03Go quiet and re-test. If stopping your own traffic for two minutes changes nothing, your traffic was not the trigger. If it changes everything, it was.
4. 04Check what a normal caller does. Our sweep sent forty-six consecutive requests. A customer sends one, thinks, sends another. If the two patterns produce different results, you are not measuring what your customers experience.

Two things that are not outages

Beyond traffic shape, a monitor needs to refuse to attribute two other kinds of failure to the thing it is probing.

Your own billing. Our probe key ran out of credit and every model in the catalogue turned red at once. The response was an HTTP 403 with a quota message — the platform was working perfectly and declining to serve *us*. A refusal to bill is a fact about your wallet. It now writes no samples at all and puts a banner on the page saying the probe has no credit, which is a different sentence from twenty-three models being down.

Requests that were shed rather than failed. A 503 that says the system is overloaded is the platform protecting itself, and it usually clears immediately. Whether that belongs in your availability number depends entirely on whether your probe caused it — which brings the whole argument back to traffic shape.

Once the probe stopped bursting, we started counting these again. A refusal a customer would also have met is a real failure, and hiding it behind a friendlier category would have been the same mistake in the other direction.

Stamp the samples with the method

The practical safeguard that made all of this recoverable is one integer on every stored sample: the version of the measurement method that produced it.

When you fix a probe, the history taken under the broken version is not merely old — it answers a different question. Averaging six months of "did one request succeed" together with "did a client that retries get an answer" produces a number that is neither. So each sample carries a version, and raising it discards everything below it.

That is expensive: the page goes back to "measuring" and takes an hour or more to refill. We paid it three times in two days, and each time it was cheaper than leaving contaminated readings to colour the page. It also forces the question every time — *does this change what the number means, or only how precisely it is measured?* — which is a question worth being made to answer.

What we would do differently from the start

• Find the rate limiter before writing the sweep. Measure what it is scoped to — key, model, route — because that determines where concurrency is free and where it is poison.
• Set the timeout from the upstream’s measured tail, not from what feels reasonable. And measure streaming first-byte separately, because the two numbers diverge and only one of them is what a client timeout races.
• Look like a normal caller. Small slices with real gaps beat one hourly sweep, even when the hourly sweep sends fewer requests in total.
• Version the samples on day one. Retrofitting a version stamp means the first bump discards history you cannot attribute either way.
• Write down what each state is allowed to claim, and hold the summary to the same floor as the rows. Ours published an average uptime across models that had not yet cleared the sample threshold the table itself enforced.

None of this is exotic. It is the ordinary discipline of not letting your instrument participate in the thing it measures — and it is easy to skip, because a monitor that reports failures looks like it is working.

Where to go next

• AI API relays, explained — what sits between your request and the model, and why a relay has state worth monitoring at all.
• Canary-testing an AI API relay — the companion method: not whether a model answered, but whether the model that answered was the one you asked for.
• What uptime means when your client retries — why single-attempt availability describes an experience almost none of your users have.

Our own measurements are on the status page, including the models currently below their usual availability. The point of this article is that those numbers are only worth reading because we spent two days establishing that they are not measurements of ourselves.