Which AI gateway should a small team actually pick?

The honest answer to which gateway a small team should pick is that it depends on one question you have probably not asked yet: what is the actual problem? These tools look interchangeable from the outside and they are not. One optimizes for catalog size, one for self-hosting, one for observability depth, one for breadth of media type. Picking the wrong axis is how teams end up migrating twice.

We build one of them, so read this accordingly. What follows tries to be the comparison we would want to read if we were choosing, which means saying plainly where the others are better.

First, do you need one at all?

If you call one provider and have no plan to add a second, you do not need a gateway. You would be adding a network hop, a vendor and a failure mode in exchange for benefits you are not currently using. Call the provider directly.

The case begins at the second provider and becomes obvious somewhere around the fourth. What tips it is rarely the code. Talking to four providers is a small abstraction that any competent engineer writes in an afternoon. What accumulates is everything around the code: four accounts, four invoices arriving on different days, four sets of rate limits with different tier systems, four credentials in your secrets manager, and no single place that answers "what did we spend on models last month".

That is the moment a gateway starts paying for itself, and it is worth naming because teams often adopt one too early, then conclude gateways are overhead.

The five questions that actually decide it

1. Is breadth of catalog the deciding factor?

If your requirement is "we want access to as many models as possible, including obscure ones", the answer is OpenRouter. They carry 300+ models, which is a larger catalog than ours at 130+. We are not going to pretend otherwise, and a comparison where the author's own product wins every row is not credible to anyone who checks.

Where breadth is the whole requirement, pick on breadth.

2. Does the gateway have to run inside your own infrastructure?

If yes, the answer is LiteLLM. It is open-source, self-hosted and MIT-licensed, covering 100+ providers. No managed service can compete with "it runs on your hardware, you can read every line, and there is no third party in the request path".

Be clear-eyed about what you are accepting. Self-hosting means you operate it. Deployments, version upgrades, provider credential rotation, capacity when traffic spikes, and an on-call rotation for the case where the gateway itself is the outage. For a team with platform engineers, that is routine. For a team of four shipping a product, it is a second product.

The question to ask internally is whether self-hosting is a requirement or a preference. If a compliance obligation says traffic cannot leave your network, it is a requirement and the decision is made. If it is a preference for control, price the operational work honestly before committing.

3. Is your real problem observability?

Sometimes the pain that sends a team looking for a gateway is not routing at all. It is not knowing which feature burned the budget, which prompts are slow, or which version of a prompt produced last week's output.

If that is the problem, Portkey is stronger than we are on observability, prompt management and semantic caching. Those are core parts of their product and secondary parts of ours. Helicone is observability-first and is not primarily a gateway, which makes it complementary to whatever you route through rather than an alternative to it.

Buying a gateway to solve an observability problem, or the reverse, is the most common category error in this space. A gateway sits in the request path and decides where a call goes. An observability tool records what happened. Both are useful and they are not substitutes.

4. Do you need more than text?

This is where we are different, and it is the honest reason to choose us.

Text, image, video and audio models sit behind the same key and the same balance, alongside web context tools for scraping, crawling and structured extraction. That means one account, one invoice and one spend number covering a workflow that generates copy, renders an image, produces a clip and pulls a page for context.

If your product only calls text models, this advantage does not apply to you and you should weigh the other four questions instead. If your product touches three or four modalities, assembling that from separate providers is where the account sprawl gets genuinely painful, and it is the case we built for.

5. Does an agent need to operate without a human?

A narrower question, but decisive when it applies. If an agent has to obtain its own credential and keep itself funded with no person in the loop, that needs the provider to expose registration and payment as API calls rather than as web forms.

We expose both. POST /agent/register returns a working key with no dashboard and no card, and POST /agent/fund returns a USDC deposit address on Base so an agent can top up its own balance from its own wallet. There is also a hosted MCP server so an agent already speaking MCP can list models, check its balance and call the gateway as tools.

If that is not your architecture, ignore this question entirely.

Side by side, on the axes that differ

Rows are deliberately limited to differences we can state accurately. A comparison table with twenty rows of checkmarks is usually padded with features everyone has, and the padding hides the three rows that would actually change your decision.

What is genuinely the same everywhere

Worth saying so you do not spend evaluation time on it.

All of these speak the OpenAI request format. Switching between them is a base URL and a key, plus the model string, which never ports because every platform names models its own way. That is true of us and of every serious alternative, and it is the single best argument for adopting any of them: the switching cost stays low, including the cost of switching away.

All of them do some form of failover. The details differ and the marketing language is nearly identical, so read the specifics rather than the claim. On our side, each provider has a circuit breaker and models carry a routing priority that defines the order rails are tried. When a rail is unhealthy the gateway skips it. That is a configured priority plus a health gate, not a system that automatically re-ranks providers from live scores, and we would rather be precise about that than let the word "intelligent" do the work. We wrote up the mechanics in how AI API failover works.

All of them let you cap spend somehow. Ours is per key: a daily limit, a monthly limit, a model allowlist and an expiry date, set per key rather than per account, which is what makes separate keys per environment useful. The authentication docs cover the fields.

The three things to check before you commit

Price transparency in the unit you actually reason in. Some platforms quote a credit currency that sits between what you spend and what you owe. If you cannot answer "what did that specific call cost in dollars" without a conversion table, cost attribution stays approximate forever. We return the exact USD cost of every call in the X-ApexApi-Cost header, to eight decimal places, including on streaming responses, and we price in real dollars with no invented currency. Whichever way you go, check this early. It is hard to retrofit.

What happens to a parameter the platform does not support. Three behaviors exist: reject with a 400, drop it silently, or forward it upstream. Silent dropping is the one that costs you a day, because your output changes and nothing errors. Test it with a parameter you actually use.

Whether the endpoints you call are covered. Compatibility almost always means chat completions. It frequently does not include embeddings, and almost never includes Assistants, Batch or fine-tuning. If your code calls client.embeddings.create(), find out now rather than during the migration. We do not have an embeddings endpoint, for example, and that is better to learn from a paragraph than from a stack trace. The detail is in how to use any AI model through an OpenAI-compatible API.

A decision you can make in ten minutes

Widest possible catalog, nothing else matters: OpenRouter.

Must run in your own infrastructure, and you have someone to operate it: LiteLLM.

The pain is observability and prompt management: Portkey, or Helicone alongside whatever routes your traffic.

More than one modality, or an agent that has to run itself: us.

One provider today, no plans for a second: none of the above, call the provider directly and revisit when you add the second.

If you want the longer per-competitor breakdowns, ApexApi versus OpenRouter, versus LiteLLM, versus Portkey and versus Helicone go row by row. If you would rather just try it, the quickstart is a base URL and a key.