Back to catalog
Portkey AI Gateway
MITPortkey AI Gateway replaces proprietary routing proxies with an open-source, self-hosted control plane for over 1,600 models and automated guardrails.
Open-source alternative to — compare it against:
Overview
Vercel AI Gateway requires prepaid credits to unlock custom model routing and bring-your-own-key access, sending inference traffic through proprietary infrastructure. Portkey AI Gateway eliminates platform markups, allowing engineering teams to run an open-source proxy across hundreds of language models directly inside their own networks.
Why Portkey AI Gateway instead of Vercel AI Gateway
Vercel AI Gateway pairs well with frontends on Vercel, providing unified routing, response streaming, and tracing inside the Next.js workflow. However, custom API keys and full model catalogs require upfront platform credits and specific account tiers. When backend services, pipelines, or agent loops run outside Vercel on AWS or bare metal, routing requests through an external hosting platform introduces unnecessary latency, network hops, and billing dependencies.
Portkey AI Gateway eliminates intermediary platform tolls. Deployed as a container, Node.js process, or Cloudflare Worker in your virtual private cloud, you pay only for base compute and direct provider usage. Teams handling millions of tokens through smaller models or local vLLM instances can distribute traffic across multiple keys, trigger automatic fallbacks during provider outages, and cache responses without per-seat platform markups. In exchange, you own deployment, updates, and service monitoring.
What it does well
- OpenAI-compatible endpoints: Routes calls across more than 1,600 language, vision, and audio models through standard APIs, letting developers switch providers by changing request headers.
- Automated failover and retries: Executes exponential backoff retries and cascades failed requests to backup models or providers during upstream outages.
- Multi-key load balancing: Distributes high-throughput traffic across multiple vendor API keys using weighted allocations to prevent provider rate limits.
- Lightweight deployment: Deploys as a self-contained container, Node.js daemon, or edge worker without database requirements, fitting cleanly into existing container orchestration stacks.
- Integrated request guardrails: Validates user prompts and model responses against JSON schemas, keyword deny lists, and compliance rules before returning payloads.
- Local token telemetry: Surfaces request latencies, error frequencies, and token expenditures locally, calculating expenses via the open-source Portkey Models pricing dataset.