Routing and variants

See Variants and progressive rollout → for the conceptual model. This page is the field-level reference for the Decider, the Router element’s routing rules, canary, and the variant catalogue itself.

Decider

The Decider block routes a request across candidate upstreams based on content, rather than a fixed provider. Routing becomes a runtime decision instead of a fixed wire: a surface can send a coding question to one model and a creative-writing question to another, or fan a request out to several models at once for comparison, all without the caller knowing which upstream actually answered.

Set it with the Mode dropdown on the LLM node’s sidebar. Any choice other than LLM Direct adds a routing node to the canvas, labelled Decider, Comparer, or Summariser to match the mode, and the LLM node’s sidebar then shows Open decider…. Configure the routing node with Configure Decider, which opens the Decider Configuration editor.

• Decide LLM based on content gives a single best-fit answer: a small, fast router LLM inspects the request and picks one candidate to actually serve it.
• Compare output from multiple LLMs and Summarise output from multiple LLMs send every candidate the same request. Summarise merges the results into one answer, while Compare hands a person every answer to compare. Every candidate is billed, and a streaming request gets a buffered response.
• In Decide LLM based on content mode, keep the candidate list to a handful of clearly distinct options (e.g. “fast/cheap model” vs. “high-reasoning model”). More candidates give the router finer-grained choices, but also more room to misclassify a request.
• Order candidates with the option you’d want as a safe fallback listed first, since the first candidate is the fallback while Default candidate (fallback) is left at First candidate.

Decider fields:

Each candidate:

Merger LLM

Shown in Summarise output from multiple LLMs mode. The merger produces one clean answer that continues through the normal Jury and guardrail pipeline.

Validation: an enabled Decider needs at least one candidate with a unique, non-empty candidate id; a selected default candidate must be one of the candidates; Decide LLM based on content needs a Router LLM; Summarise output from multiple LLMs needs a Merger LLM.

Routing rules

Routing rules send a request to a variant by matching a request header, a top-level body field, or an attachment type. They live on the Router element, which you drag from the Inputs palette group onto the edge after the Access Point. The variant is chosen before authentication, policy, and guardrails run, so the chosen variant’s own model, Inputs elements, and guardrails govern the rest of the request.

Select the Router element to see its Enforce routing switch and a count of its rules, and use Configure routing rules… to open the full-page editor, where Add routing rule adds a row. Rules apply while Enforce routing is on, and are kept when you turn it off. Routing rules belong to the base surface, not to a variant.

• Header is the most explicit signal, usable when the caller (or a proxy in front of them) can set a custom header.
• Body field keys off something already present in the request body, like a caller-supplied tag.
• Attachment type gives deterministic, LLM-free routing to a variant whose model accepts that input: for example, sending any request carrying an image straight to a variant with a vision-capable model and an Image File element.

Match kinds:

How a chat request picks its variant:

1. An explicit $alias in the route wins first.
2. Otherwise, the first matching rule wins and sends the request to its variant.
3. A request that matches no rule falls through to the canary split, and then to the surface’s default: the base surface, or the variant promoted to default.

A realtime session uses the first Realtime voice (WebSocket) rule that names a variant. Otherwise it goes to the surface’s default.

Validation: while Enforce routing is on, every rule must name a variant that exists on the surface, and a Header or Body field rule needs both a name and a value.

Canary

Canary implements an A/B traffic split across a surface’s variants, letting you prove a new model or variant is safe with real traffic before committing to it fully. Instead of an all-or-nothing cutover, a small percentage of requests see the new option while the rest keep hitting your known-good baseline. It’s set in the Canary / A-B traffic split section of the Surface Configuration editor, and belongs to the base surface, not to a variant.

• Start small (5–10% is a reasonable first split) and watch the Monitoring tab’s error rate, latency, and cost for that variant against the baseline.
• Once its telemetry looks comparable or better, increase the split in steps rather than jumping straight to an even or majority share, so a regression only ever affects a small slice of traffic before you notice it.

Turning the canary split off keeps the stored percentages, so turning it back on restores the previous split.

Validation: an enabled canary needs a positive total weight, and every named variant must exist on the surface.

Variants

Each surface carries a catalogue of named variants, edited on the Variants panel. A variant is addressed at {route}$alias/.... The base surface is the default until you select Default on a variant, which promotes that variant to receive every request that no $alias, routing rule, or canary split has placed.

Each variant:

Edit a variant’s own settings by switching to it with the variant switcher on the canvas and editing its own copies of the surface’s elements. A variant keeps its own copy of the settings it covers, so a later change to the base surface doesn’t flow into it. Its identity and routing fields (id, name, listener, route, metrics and log settings, and integrations) stay those of the base surface, so a variant is reachable at the same base route and follows the same metrics and log settings.

A variant keeps its own copy of these settings:

Related

• Guardrails: The provider/model/endpoint target shape shared by Decider candidates, Judge, and Jury.
• Resilience and caching: Failover and load-balancing targets, which also use that shape.
• Variants and progressive rollout: The conceptual model for routing rules, canary, and variants.
• Inputs and modalities: How the Router sends each attachment type to a variant whose model accepts it.
• LLM drift detection: How a replay evaluation’s target surface or variant compares to routing rules and canary as a way to evaluate a change.