Route around failure
Keep a surface answering through a provider outage, spread load across a pool by cost or latency, and change behaviour safely on a stable endpoint. Field-level reference for every block here is in Resilience and caching → and Routing and variants →.
| Guide | What you will achieve |
|---|---|
| Fail over to a backup provider on error | Add ordered backup providers to a surface, so a request retries against a backup instead of failing outright when the primary provider errors. |
| Balance traffic across providers by cost or latency | Distribute requests across a weighted pool of providers, selecting members by weighted round-robin, lowest observed latency, or lowest price. |
| Roll out a model with a canary split | Create a surface variant and route a small percentage of live traffic to it, so you can watch its cost, latency, and rejection figures before promoting it to every caller. |
| Route requests to a capable model by attachment type | Add the Router element so requests carrying an image, audio file, document, or realtime voice session go to a variant whose model and Inputs elements accept them. |
| Compare or summarise answers across models | Send every request to several models at once, then return each answer side by side or consolidate them into one with a merger model. |
Was this page helpful?
Glad to hear it! Please tell us how we can improve more.
Sorry to hear that. Please tell us how we can improve.
Thank you for sharing your feedback so we can improve your experience.