provider
object in the request body.
Load Balancing
For each model in your request, 1Router.com’s default behavior is to load balance requests across providers with the following strategy:- Prioritize providers that have not seen significant outages in the last 10 seconds.
- For the stable providers, select the lowest-cost candidates and choose one weighted by price.
- Use the remaining providers as fallbacks.
Example Scenario
Let’s say:- Provider A charges $1/million tokens.
- Provider B charges $2/million tokens.
- Provider C charges $3/million tokens.
- Provider B recently experienced outages.
- Your request is 3x more likely to be routed first to Provider A than to Provider C.
- If Provider A fails, Provider C will be tried next.
- If both providers fail, Provider B will be tried last.
Custom Routing
You can customize which providers 1Router.com will prioritize for your request using theorder
field. The router will follow the priority list for the model you’re using. If you don’t specify this field, the router will load balance across the top providers to maximize uptime.
1Router.com will attempt the providers in your specified order and proceed to other providers if none are operational. To restrict routing strictly to your preferred providers, you can disable fallbacks.