Portkey Alternative: What to Look For & Top Options
When teams evaluate an LLM gateway, Portkey almost always appears on the shortlist — and for good reason. But search traffic for "portkey alternative" is growing, and that growth reflects something real: teams that have looked carefully at Portkey and decided they want a different trade-off. This guide covers why teams look for an alternative to Portkey, what to evaluate, a fair assessment of Portkey itself, and how a zero-markup BYOK gateway like flo2 fits the picture. For broader context, see our best LLM gateway comparison.
Why teams look for a Portkey alternative
The reasons are rarely complaints about Portkey being bad — they are about fit. Different teams have different priorities, and a platform that covers everything is not always the right choice for a team that needs one thing done simply.
Feature breadth versus simplicity
Portkey is expansive: unified provider access, routing and fallbacks, load balancing, caching, observability, guardrails, prompt management, and team governance. For a platform team standardizing LLM infrastructure across many applications, that breadth is the value. For a small team that wants smart routing and transparent cost accounting, it can mean learning and maintaining a lot of surface area that will never be used. The cost of a broad platform is not just the subscription — it is the cognitive load of everything you carry but do not touch.
Pricing and BYOK
Some hosted gateway products sit in the token path and add a per-token margin. Whether a given Portkey tier does this is worth verifying directly at portkey.ai — not in a third-party article, including this one. Separately, some teams want to hold their own provider relationships so that spend history, rate-limit capacity, and enterprise agreements live with OpenAI or Anthropic directly rather than through an intermediary. A bring-your-own-key (BYOK) architecture with zero markup preserves that.
Specific routing needs
Teams sometimes find that a particular pattern — AI racing, A/B testing with a model judge, or granular per-call cost attribution — either requires a tier they do not need otherwise, or is handled differently than expected. When the capability that matters most is not a primary feature of a broad platform, a more focused alternative can be a better fit.
What to evaluate in any Portkey alternative
Before comparing names, get clear on what you actually need. Our article on the Portkey AI gateway overview walks through the full feature set as a baseline; these are the dimensions that determine fit across the whole category:
- Provider coverage. Does it reach OpenAI, Anthropic, Gemini, Groq, Cerebras, Mistral, DeepInfra, and xAI — plus whatever open-weight hosts you rely on?
- Routing, fallback, and racing. Cost-aware or latency-aware routing, automatic failover, and optional racing (parallel calls, take the fastest result) are the core justification for a gateway.
- Caching. Opt-in response caching cuts repeat-call spend and latency. The key word is opt-in: you need per-call control, not a blanket policy.
- Cost accounting. The bar is true per-call cost in real dollars — reconcilable against provider invoices, not aggregate token tallies.
- Guardrails and observability. If input/output safety policies and deep tracing are central requirements, weight these heavily. Not every alternative matches Portkey's depth here.
- OpenAI + Anthropic compatibility. Drop-in compatibility with both SDKs means switching is a base-URL and key change, not a rewrite.
- Hosted vs self-host. Self-hosting gives you control but means you own deployment, scaling, security, and maintenance. Hosted trades that for faster time-to-first-call and no infrastructure to babysit.
- Pricing model. Subscription, per-token markup, or BYOK zero-markup? Each suits a different profile. BYOK preserves the provider relationship and gives you the most cost transparency; it also means you manage your own keys.
A fair note on Portkey
The honest assessment: Portkey is a strong, full-featured AI gateway and for many teams it is the right choice. Its open-source core gives you a self-hostable routing engine you can audit and run inside your own infrastructure. Its hosted platform wraps that in observability, prompt management, guardrails, and team governance — tooling that a platform team standardizing across many applications genuinely needs. The fact that alternatives exist does not mean Portkey falls short; it means the LLM gateway category has matured enough that several credible approaches coexist, each with a different trade-off.
Evaluation criteria: Portkey vs alternatives
| Criterion | Portkey | flo2 (BYOK zero-markup) |
|---|---|---|
| Provider coverage | Large catalog, many providers | OpenAI, Anthropic, Gemini, Groq, Cerebras, DeepInfra, Mistral, xAI, OpenRouter |
| Routing & fallback | Yes — routing, retries, load balancing | Yes — smart routing, fallback chains |
| AI racing | Check current docs | Yes — parallel calls, return fastest result |
| A/B testing | Check current docs | Yes — A/B + model judge for evidence-based model selection |
| Caching | Yes | Yes — opt-in response caching |
| Cost accounting | Yes, via observability dashboard | True per-call dollar figures at provider list prices |
| Observability & tracing | Strong — a primary feature | Cost and routing focused; less depth |
| Guardrails | Yes — built-in | Not a current focus |
| Prompt management | Yes — first-class feature | Not included |
| OpenAI + Anthropic compat. | Yes — OpenAI-compatible | Yes — compatible with both APIs |
| Hosted vs self-host | Open-source self-host + hosted platform | Hosted; no infra to manage |
| Pricing model | Hosted tiers; verify at portkey.ai | BYOK; zero token markup; free during beta |
flo2: a zero-markup BYOK option
flo2 takes a deliberately narrower position than Portkey. Where Portkey aims to be the full control plane for a platform team — routing plus observability plus guardrails plus prompt management — flo2 focuses on routing economics for developer teams that want to minimize cost and maximize routing intelligence, without operating their own infrastructure or adding a reseller margin.
The model is BYOK: you connect your own keys for OpenAI, Anthropic, Gemini, Groq, Cerebras, DeepInfra, Mistral, xAI, and OpenRouter. You pay each provider directly at published prices. flo2 adds zero token markup. In return, you get one key compatible with both the OpenAI and Anthropic APIs, plus:
- Smart routing — sends each request to the cheapest or fastest qualifying model.
- Fallback chains — transparently fails over when a provider errors or rate-limits.
- AI racing — fires several models in parallel and returns the fastest acceptable response.
- A/B testing with a model judge — scores outputs on real traffic so model selection is evidence-based.
- Opt-in caching — cuts repeat-call spend where it is safe to reuse a prior answer.
- True per-call cost accounting — real dollars per request and per model, not aggregate token tallies.
The honest trade-off: flo2 does not have Portkey's observability depth, guardrails, or prompt management. If those are central to your requirements, Portkey — or another platform built around them — is the right answer. If your bottleneck is cost transparency and routing intelligence with no token markup, flo2 is free during beta and takes a base-URL change to try.