How to use any AI model through an OpenAI-compatible API

You built against OpenAI's API and it works. Now you want to try Claude Opus 5 or a cheaper Gemini model on the same workload, and the prospect of rewriting the integration is enough to stop you looking.

You do not have to rewrite anything. Change the base URL, change the API key, change the model string. Your SDK calls, your message arrays, your retry logic and your streaming loop all stay exactly as they are.

That is the promise of an OpenAI-compatible API, and for the common path it holds. What follows is how the switch works, then the part that matters more: which things port cleanly, which ones quietly do not, and how to test a provider's compatibility claim in fifteen minutes instead of finding the gap in production.

What "OpenAI-compatible" actually means

There is no certification body, no standards organization and no formal protocol. OpenAI compatibility is a convention that emerged because the OpenAI Python and Node SDKs became the default AI client libraries, and every other provider wanted access to that installed base. Match the request and response shape and you inherit every tool, framework and internal script already written against those SDKs. That distribution effect, not any technical merit of the format, is why dozens of providers now ship a compatible endpoint.

The surface the SDK actually depends on is narrower than most people assume. Three things:

1. The endpoint path. POST /v1/chat/completions, appended to whatever base URL you configure.
2. Bearer token authentication. An Authorization: Bearer <key> header. No OAuth handshake, no custom scheme.
3. The JSON contract. A model string and a messages array going in. A choices array with a message and a finish_reason coming back, plus a usage object. Streaming arrives as server-sent events carrying delta fragments and terminating with [DONE].

Everything beyond those three is optional territory. Two providers can both be honestly described as OpenAI-compatible while supporting very different subsets. The useful mental model is OpenAI-shaped input routed into provider-native semantics, not a clone of every OpenAI feature.

The switch itself

The result first: three values change, nothing else does.

# Before
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# After
client = OpenAI(
    api_key=os.environ["APEXAPI_KEY"],
    base_url="https://api.apexapi.dev/v1",
)

response = client.chat.completions.create(
    model="anthropic/claude-opus-5",
    messages=[{"role": "user", "content": "Summarize this changelog."}],
)
print(response.choices[0].message.content)

The Node pattern is identical:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.APEXAPI_KEY,
  baseURL: "https://api.apexapi.dev/v1",
});

const response = await client.chat.completions.create({
  model: "google/gemini-3.6-flash",
  messages: [{ role: "user", content: "Summarize this changelog." }],
});

Notice what did not change. No new SDK, no new client class, no rewrite of how messages are built, no per-provider branching. The SDK never cared which company was on the other end. It cared about the URL it posts to and the JSON it gets back.

This is also the argument for keeping the model string in configuration rather than hardcoded at the call site. If the model name is a constant in one place, switching later is a config edit. If it is a literal in forty call sites, it is a refactor.

What ports cleanly, and what does not

The first column is what you are relying on. The second is how reliably it survives a provider switch.

That last-but-one row is worth expanding, because the failure modes are opposite and both are documented. Groq returns a 400 if you send logprobs, logit_bias, top_logprobs or messages[].name, and requires n to equal 1. Google's Gemini compatibility layer takes the other approach: it maps reasoning_effort onto its own thinking budgets and states plainly that any parameter not on its supported list is silently ignored.

Same category of field, opposite behavior. The 400 is the friendlier outcome, because it tells you immediately. Silent dropping is the one that costs a day, since your seed for reproducibility or your logit_bias for output control disappears without a trace and the only symptom is that the output is different from what you expected. A 200 response is not the same thing as correct behavior.

Tools cause the second most common surprise. The request validates, the call returns 200, and the model replies with prose where your code expected a tool_calls array. That is not an HTTP error, so error monitoring stays quiet and the failure surfaces in your parser. Before moving a tool-using workload, test the tool call specifically rather than testing that the endpoint responds.

Where compatibility ends

Five gaps are worth checking before you commit, because none of them announce themselves.

Endpoint coverage is narrower than the phrase suggests. Most compatible providers implement chat completions and a model list. Fewer implement embeddings, and almost none implement Assistants, Batch, Realtime or fine-tuning. If your application calls client.embeddings.create() anywhere, that call needs its own home. This is the single most common reason a migration that looked like a one-line change turns into a two-day change.

Token counting differs even when the shapes match. Different tokenizers produce different counts for the same string. A prompt that fits a context window on one model overflows on another, and cost estimates shift even at identical per-token prices. Treat usage as authoritative per provider rather than assuming a local count transfers.

Error bodies are only loosely standardized. The {"error": {...}} envelope is broadly consistent. The code and type values inside it are not, and neither are the status codes for cases OpenAI does not have. Anything that parses error codes to decide whether to retry deserves a look before the cutover.

Rate limits are not standardized at all. Most providers enforce requests per minute and tokens per minute tied to account tiers, but the tiers, the thresholds and the headers that report them differ by provider. Whatever backoff logic you tuned against one provider's limits is not calibrated for the next one, and the published numbers change often enough that it is worth reading them at migration time rather than trusting a figure in an article.

Streaming edge cases surface last and hurt most. Whether a final chunk carries usage, whether the stream terminates with [DONE], and how the connection behaves when the upstream provider dies mid-generation all vary. A broken stream frequently produces no error code at all, just a connection that closes early, which is a failure that status-code monitoring never sees. We covered that specific problem in how AI API failover works.

Model identifiers never port, and that is fine

Every provider names models its own way and there is no standard for it. gpt-5.6-luna means something on one platform and nothing on another, and the same underlying model reached through two different infrastructure rails may carry two different names.

On ApexApi the slug is always maker/model, where the maker is whoever created the model rather than whichever rail delivers it. anthropic/claude-opus-5 is the Anthropic model whether it is served directly or through a cloud rail, because the delivery path is our routing problem and not something your code should encode. When a slug changes we keep the old one as an alias, so an integration written a year ago keeps resolving.

Since model names are the only thing guaranteed not to transfer, GET /v1/models is the first call to make against any new provider. It returns the same list shape OpenAI returns, so you can enumerate real identifiers programmatically instead of reading a docs page.

One provider, or a gateway

Both are reasonable, and the right answer depends on how many models you expect to touch.

A first-party provider gives you one compatible endpoint for that company's own models. You get native feature access, the lowest latency floor, and new capabilities the day they ship rather than whenever an intermediary adds support. If you need one provider's models at maximum fidelity, go direct. Nothing in this article argues otherwise.

The cost shows up at the fourth or fifth provider. Five accounts, five invoices, five keys in your secrets manager, five sets of rate limits, and five different answers to "what did we spend last month". A gateway collapses that into one endpoint, one key and one balance, and the same three-value switch reaches every model it carries.

Being straight about the trade, since technical readers check: OpenRouter carries 300+ models, a larger catalog than our 130+. If raw model count is the deciding factor, that is the honest answer. Portkey is stronger than us on observability, prompt management and semantic caching. LiteLLM covers 100+ providers as open-source, self-hosted, MIT-licensed software, which is the right pick when infrastructure control matters more than a managed service. Helicone is observability-first and complements a gateway rather than replacing one.

Where we are different is breadth of modality on one account. Text, image, video and audio models sit behind the same key and the same balance, alongside web context tools for scraping, crawling and structured extraction, plus a hosted MCP server and autonomous USDC funding for agents that need to top themselves up. If you need more than text inference from a single billing relationship, that combination is the reason to look at us.

How this works on ApexApi, precisely

We are an OpenAI-compatible gateway, so it is worth being exact about the edges rather than leaving you to find them.

POST /v1/chat/completions accepts three different request shapes on the same path. The OpenAI chat completions body is the primary one. We also detect an OpenAI Responses-shaped body and an Anthropic Messages-shaped body and convert both internally, so a client built against either format works without a translation layer of your own. Worth stating plainly: those are body shapes accepted on the chat completions path, not separate /v1/responses and /v1/messages routes. A client that posts to those paths by URL will not find them.

Unknown parameters are forwarded to the upstream provider rather than rejected. Our schema validates the fields we need to reason about for billing and limits, and passes everything else through untouched. tools and tool_choice are deliberately in that pass-through set, because different clients send slightly different tool shapes and a gateway that validates them strictly breaks compatibility for no benefit. The provider is the schema authority, not us.

That holds on every rail whose upstream API is itself OpenAI-shaped. Anthropic's Messages API is not, so on Claude models the request is rebuilt rather than forwarded, and a few OpenAI fields have nothing to be rebuilt into. We took the position argued above, that a silent drop is the worst of the three options, and split those fields by consequence. response_format and an n above 1 change the shape of what comes back, so their loss would surface as a bug inside your code rather than ours: those return a 400 naming the field in error.param. seed, logprobs, logit_bias and the two penalties only affect sampling, so the request runs and the response carries an x-apexapi-unsupported-params header listing exactly what was not applied. Gemini sits in between: it has native equivalents for response_format, seed and both penalties, so those are applied rather than announced. The full matrix is in the API reference.

What we do validate, and therefore what can return a 400 that OpenAI would not: temperature between 0 and 2, top_p between 0 and 1, the two penalties between -2 and 2, and n capped at a small ceiling because each extra completion multiplies the output cost we hold against your balance. File content parts are accepted as base64 PDF data URIs up to 2MB, with a bounded number per request.

Two OpenAI endpoints we do not have: there is no embeddings endpoint, and no Assistants, Batch or fine-tuning surface. If you need embeddings, leave that call pointed where it is today.

One status code OpenAI does not have: an empty balance returns 402. If your retry logic treats every 4xx the same, special-case that one, because retrying will not help and rate-limit backoff is the wrong response to it.

Two things that are ours rather than part of the convention. Every response tells you the exact cost of that call in USD, to eight decimal places. On a normal response that is the X-ApexApi-Cost header. On a streaming response it cannot be a header, because headers are sent before the first token and the cost is not known until the last one, so it arrives instead as one final SSE frame with an empty choices array carrying usage and an x_apexapi object with the cost and your remaining balance. That frame is sent before data: [DONE]. The official SDKs skip a chunk with no choices; a parser of your own that reads choices[0] needs a length check. And a key can carry a model allowlist, which means a model that exists in the catalog can still be refused for a specific key. That is a deliberate scoping feature rather than a compatibility gap, but it explains why a slug that works with one key can fail with another.

On failover, one precise claim rather than a broad one: each provider has a circuit breaker, and models carry a routing priority that defines the order rails are tried. When a rail is unhealthy the gateway skips it, with no configuration in your codebase. That is a configured priority plus a health gate, not a system that re-ranks providers automatically from live scores.

Verify it yourself in fifteen minutes

Never take a compatibility claim from a landing page. Five checks against your own workload settle it.

List the models. Confirm the response has the OpenAI list shape and that the model you want is in it under the identifier you expect.

curl -s https://api.apexapi.dev/v1/models \
  -H "Authorization: Bearer $APEXAPI_KEY" | head -40

Send one non-streaming completion. Confirm choices[0].message.content, finish_reason and usage all arrive in the shape your code reads.

curl -s https://api.apexapi.dev/v1/chat/completions \
  -H "Authorization: Bearer $APEXAPI_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "anthropic/claude-opus-5",
    "messages": [{"role": "user", "content": "Reply with the word ok."}]
  }'

Send the same request with "stream": true. Watch for delta chunks, a terminating [DONE], and whether a final frame carries usage. If your cost accounting reads that frame, this is the check that matters.

Exercise the one advanced feature you actually depend on. Tools, JSON output, image input, whichever it is. Not all three if you only use one, and definitely not none of them on the grounds that a basic completion worked.

Send a deliberately broken request and read the error body. An unknown model, or a malformed message array. You are looking at the status code and the code and type values, because that is what your retry and alerting logic branches on at 3am.

If all five pass on your own traffic, the switch is the three-value change it promises to be. If one fails, you found it now instead of after the cutover.

The compatibility is real, the details are yours

An OpenAI-compatible API is genuine infrastructure leverage. It turns model choice into a configuration value and removes the per-provider branching that otherwise accumulates in every codebase talking to more than one model. It is the closest thing this ecosystem has to a universal adapter.

It is not a guarantee that everything works. It is a guarantee about one request shape on one endpoint, and the distance between that and your whole application is the five gaps above. Read the compatibility claim as a starting point, then spend fifteen minutes testing the parts you depend on.

Read the quickstart to make your first call against the gateway, or browse the catalog to see the identifiers you would be switching between.