Route around failure
Keep a surface answering through a provider outage, spread load across a pool by cost or latency, and change behaviour safely on a stable endpoint. Field-level reference for every block here is in Resilience and caching → and Routing and variants →.
| Guide | What you will achieve |
|---|---|
| Fail over to a backup provider on error | Add ordered backup providers to a surface, so a request retries against a backup instead of failing outright when the primary provider errors. |
| Balance traffic across providers by cost or latency | Distribute requests across a weighted pool of providers, selecting members by weighted round-robin, lowest observed latency, or lowest price. |
| Roll out a model with a canary split | Create a surface variant and route a small percentage of live traffic to it, so you can watch its cost, latency, and rejection figures before promoting it to every caller. |
| Route requests to a capable model by attachment type | Add the Router element so requests carrying an image, audio file, document, or realtime voice session go to a variant whose model and Inputs elements accept them. |
| Compare or summarise answers across models | Send every request to several models at once, then return each answer side by side or consolidate them into one with a merger model. |
Was this page helpful?
Glad to hear it. Thanks for letting us know.
Sorry to hear that. Thanks for letting us know.