Provider routing
Requests are dynamically routed based on policy
Incoming request
Relay Gateway
Cost
Latency
Policy
Selected provider
OpenAI · Anthropic · Gemini
cost-optimized
latency-optimized
weighted
failover-chain
Active providers
OpenAI
api.openai.com
Healthy
Requests
1.2M
Latency
412ms
Spend
$8,420
Routing weight58%
Enabled
Anthropic
api.anthropic.com
Healthy
Requests
562k
Latency
488ms
Spend
$3,910
Routing weight27%
Enabled
Gemini
api.gemini.com
Degraded
Requests
210k
Latency
612ms
Spend
$1,180
Routing weight12%
Enabled
Experimental / Hidden providers
opt-in
DeepSeek
api.deepseek.com
Hidden
Requests
—
Latency
—
Spend
—
Routing weight0%
Enabled
Groq
api.groq.com
Hidden
Requests
—
Latency
—
Spend
—
Routing weight0%
Enabled