AI API relays, explained: how they work and how to audit one

An AI API relay is a server that stands between your code and a model provider. You point your SDK at the relay instead of at OpenAI or Anthropic, send the same request you would have sent anyway, and the relay forwards it upstream, gets the answer, and passes it back. That is the whole idea. Everything that distinguishes one relay from another — the price, the reliability, and the risk — lives in what happens during that round trip.

This guide covers what a relay actually does, the four kinds you will meet, and the checks that separate one worth using from one worth avoiding. It is written by people who run one, which is a conflict of interest and also the reason we can show you the internals rather than describe them.

The mechanics, in one request

A relay is an HTTP server that speaks the same protocol as the provider it forwards to. In practice that means it accepts a request shaped like this one and returns a response shaped like the provider’s.

curl https://proxystation.co/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"hi"}]}'

Four things happen between that request leaving your machine and the answer arriving:

1. 01Authentication. The relay checks its own key, not the provider’s. Your key is issued by the relay and is meaningless upstream.
2. 02Routing. The relay decides which upstream credential and which upstream endpoint will serve this request. One model name can have several possible routes; which one answers is a policy decision, and it is the part most worth understanding.
3. 03Forwarding. The request is re-signed with the relay’s own provider credential and sent on. The provider sees the relay, not you.
4. 04Accounting. The response comes back, the relay reads the token counts out of it, prices them, and debits your balance.

Step three is why relays exist commercially: the relay buys capacity at whatever rate it can get and resells it. Step two is why they differ from each other. Step four is where the numbers you are comparing come from.

Four kinds of relay

The category’s own vocabulary, from the community-maintained index of relay stations, sorts them by what sits upstream:

The distinction is not academic. A reverse relay is serving you a consumer session dressed as an API, which is why it can be so cheap and why it can disappear without notice. An official relay is reselling capacity it actually paid for, so its price floor is the provider’s wholesale rate and its failure mode is a business failing rather than an endpoint vanishing.

One model name, several routes

The single most surprising thing about relays, to people meeting them for the first time, is that a model name is not a destination. It is a label that maps to one or more upstream channels, and the relay picks between them per request.

On this station’s catalogue, read on 2026-08-30: of 53 models, 29 have exactly one route and 24 have more than one — seventeen with two, four with three, three with four. That is public data; the route count is published per model in the catalogue API.

When there is more than one route, something has to choose. This station’s routing policy is ordered stability first, then time-to-first-token, then authenticity, then price. A relay that ordered those differently — price first, say — would be making a defensible but very different promise, and you would feel the difference in which model actually answers.

This matters because it is measurable and it is measurably wrong elsewhere. A study of 28 relays found that 45.83% of endpoints served a model that did not match the one requested. That is not a rounding error; it is close to half. How routing works, and what a route count tells you goes through the mechanism.

How the price comparison is built

Every relay advertises a discount, and a discount is a ratio between two numbers: what the relay charges and what the provider charges. The first number the relay knows exactly. The second one it copies from somewhere.

That second number is the soft part of every comparison you will read, including ours. The providers publish it themselves — OpenAI, Anthropic, Google and xAI each maintain a page — so it is checkable, and checking it is the single most useful thing a sceptical reader can do.

We audited our own catalogue’s list-price column against those pages on 2026-08-27 and found it wrong in both directions: one row overstating our discount, one understating it. We wrote up exactly what was wrong and how we found it, because a comparison whose denominator nobody has checked is not a comparison.

Models in the catalogue

53 (read 2026-08-30)

Carrying a list-price column

44 of 53

Hand-verified against provider docs

10

Median discount, comparable models

87%

The gap between 44 and 10 is the honest shape of this. Forty-four rows carry a number the upstream software supplied; ten have been read by a person against the provider’s own page and carry the date they were read. Only those ten are used where we make a public claim, and each expires sixty days after it was checked so that no figure stays on the page unre-read.

There is also a units trap sitting underneath all of this, and it is worth knowing before you compare anything: some sources quote per thousand tokens and others per million. Mixing them silently multiplies or divides by a thousand, and the result looks like a spectacular deal rather than a mistake. That error, and how to spot it.

And list prices themselves carry conditions — context thresholds, cached input rates, batch tiers, peak and off-peak windows. xAI, for one, bills a prompt that reaches 200,000 tokens at a higher rate for the entire request. Reading a provider’s list price correctly covers the conditions that a single headline number hides.

How can it be cheaper at all?

This is the question worth being suspicious about, and a relay that does not answer it is asking for trust it has not earned. There are only a few honest answers, and they are not equally reassuring.

The first two are the benign explanations, the third and fourth are the ones a canary test and a route inspection are for, and the fifth is why a large prepaid balance is a bad idea regardless of how good the price is. Any relay quoting a headline discount should be able to say which of these rows it is standing on.

Our own answer is the first row, and the number to check it against is our own catalogue: read on 2026-08-30, the median discount across the 40 models with a comparable list price was 87%, the shallowest was 31%, and the models where the gap is widest are also the ones with the most upstream competition. A uniform discount across every model in a catalogue would be the suspicious pattern — it would mean the number was decided rather than derived.

Token accounting: who counts, and against what

Your bill is token count times price. The price is published; the count comes from whoever is doing the counting. On a relay, that is the relay — it reads the usage block out of the provider’s response and prices it.

This is a genuine trust surface and it is rarely discussed. Two things make it checkable. The provider returns a usage object in every non-streaming response, so you can compare what you were billed against what the provider said was used. And a relay that publishes per-request logs lets you reconcile a period rather than a request. If neither is available, the accounting is unverifiable, which is worth knowing before you route production traffic through it.

Streaming, timeouts and what a relay adds

A relay sits in the path, so it adds latency — one extra network hop, plus whatever the routing decision costs. In practice the hop is small next to model inference time, but it is not nothing, and it is the reason time-to-first-token appears in routing policies at all.

Streaming is where implementations diverge most. A faithful relay forwards server-sent events as they arrive and preserves the chunk shape; a lazy one buffers the whole response and replays it, which looks identical to a client that only reads the final text and completely different to one rendering tokens as they arrive. If your product streams to a user, test that specifically — it is the compatibility detail most likely to be quietly missing, and one of several covered in what OpenAI-compatible has to mean.

What can actually go wrong

The category has a documented failure record, and pretending otherwise would make everything else here less believable. An audit of 428 relay stations found nine injecting malicious code into responses and one stealing funds outright. Those are small percentages of a large number, and they are also the reason the community index carries risk flags — no-entity, cheap-trap, ran-away, reverse — rather than just prices.

The realistic failure modes, roughly in order of how often they happen:

• Silent substitution. You ask for a frontier model and get a cheaper one. Costs you quality, invisibly, until you test for it.
• Prepaid balance loss. The operator stops trading with your credit on the books. This is the one that actually costs money, and it is why the billing model you choose is a risk decision and not just an accounting one.
• Quiet degradation. Rate limits, queueing and timeouts that were not there last month.
• Data handling. Your prompts pass through a third party. For regulated or confidential workloads that is a compliance question, not a preference.

How to audit one

You cannot read a relay’s routing table, and its marketing page is not evidence. What you can do is send requests and look carefully at what comes back. Four checks, in increasing order of effort:

1. 01Ask it what it serves. GET /v1/models on an OpenAI-compatible relay lists the models your key can reach. Compare that with the marketing page; a gap between them is informative.
2. 02Run a canary. Send a prompt whose answer differs between models and see which answer comes back. This is the only check that tests the claim that actually matters. The method, with a script.
3. 03Check the denominator. Take three models, look up the provider’s own published price, and recompute the advertised discount yourself. If it does not reconcile, the comparison was built on an unchecked column.
4. 04Read the terms and look for a legal entity. Who are you actually extending credit to? A relay with no identifiable company behind it is a relay whose failure has no address.

That last check is the uncomfortable one for us to publish, and we are publishing it anyway: this site does not currently name a registered legal entity in its terms. We think you should weigh that, and we would rather you heard it here than found it yourself and wondered what else was unstated. A dedicated guide to the trust checklist is planned once that is resolved.

When a relay is the right answer

A relay makes sense when you want one integration across several providers, when you want to spend below list price, and when the workload is not sensitive enough to make a third-party hop a problem. It makes less sense when you need a contractual relationship with the model provider, when the data is regulated, or when your volume is large enough that the provider will negotiate with you directly.

Practically, the integration cost is close to zero either way, because the surface is standardised — as far as it goes. Changing a base URL is a one-line edit. That cuts both ways: it is also a one-line edit to leave, which is the best protection you have.

The rest of this series

Each of these goes deeper on one part of the above, and each links back here. The numbers in them are ours, dated, and checkable against the providers’ own pages.