# Routing and variants

> Field reference for the Decider (content-based routing and Mirror mode), attribute-based routing rules, canary splits, and the surface variant catalogue.

See [Variants and progressive rollout →](/products/affinidi-trust-fabric/agent-stream/concepts/variants-and-rollout.md) for the conceptual model. This page is the field-level reference for the Decider, routing rules, canary, and the variant catalogue itself.

## Decider

The Decider block routes a request across candidate upstreams based on content, rather than a fixed provider. Routing becomes a runtime decision instead of a fixed wire: a surface can send a coding question to one model and a creative-writing question to another, or fan a request out to several models at once for comparison, all without the caller knowing which upstream actually answered.

- route, the default, gives a single best-fit answer: a small, fast router LLM inspects the request and picks one candidate to actually serve it.

- mirror sends multiple models the same request, then either merges the results into one answer (merge mode) or hands a human every answer to compare verbatim (collect mode). It’s capped at 10 candidates, since every candidate is a fully parallel call and cost and latency scale with the candidate count.

- In route mode, keep the candidate list to a handful of clearly distinct options (e.g. “fast/cheap model” vs. “high-reasoning model”). More candidates give the router finer-grained choices, but also more room to misclassify a request.

- Order candidates with the option you’d want as a safe fallback listed first, since it becomes the implicit default when Default Candidate (Fallback) is left unset.

| Field | What it does | Default |
| Enabled | Master switch. When disabled, the surface uses its base provider unchanged. The dashboard has no standalone toggle for this field. It’s implied by the Mode dropdown (below): choosing “LLM Direct” clears the decider, and any other choice enables it. | Off |
| Strategy | Dispatch strategy (below). Set in the dashboard via the same Mode dropdown, whose options read “Decide LLM based on content” (route), “Compare output from multiple LLMs” (mirror + collect), and “Summarise output from multiple LLMs” (mirror + merge). There is no control literally labeled “Strategy”. | Decide LLM based on content (route) |
| Router LLM | The router LLM that makes the routing decision (typically small and fast). Required by the route strategy; unused by mirror, which fans out to every candidate instead. Matches the dashboard’s Router LLM section heading. | Not set |
| Routing Instructions | System prompt steering the routing decision. The candidate menu is appended automatically. Labeled Routing instructions in the dashboard, not “System Prompt”. | Empty |
| Candidates | Candidate upstreams. Must be non-empty when enabled. Matches the dashboard’s Candidates (N) list label. | None added |
| Default Candidate (Fallback) | Candidate used when the router returns an unknown id or fails. When unset, the first candidate is the fallback. Route strategy only. Labeled Default candidate (fallback) in the dashboard. | Not set |
| Mirror | Mirror-strategy aggregation settings. Consulted only when the strategy is mirror. No dashboard control is literally labeled “Mirror”. Its settings (below) surface once Mode is set to “Compare output from multiple LLMs” or “Summarise output from multiple LLMs”. | Not set |
| Disable Thinking | Disable chain-of-thought for a reasoning router model, as a toggle. Matches the dashboard’s Disable thinking switch. | Off |

Strategies:

| Value | Behaviour |
| route (default) | The router LLM inspects the request and selects a single candidate, whose provider serves the request: the classic content-based router. |
| mirror | The request is fanned out to every candidate concurrently (no router LLM runs), then aggregated per the mirror mode below. Capped at 10 candidates to bound per-request egress, latency, and cost. |

Each candidate:

| Field | What it does |
| Candidate ID | Stable identifier the router returns to select this candidate; must be unique and non-empty. Labeled Candidate id in the dashboard. |
| Name | Human-readable; shown in the router menu. |
| Description | Fed into the router menu so the model can reason about which upstream to pick. |
| LLM Configuration | Provider/model/endpoint for the main call when this candidate wins. Dashboard section headed LLM Configuration, not “LLM Target”. |

### Mirror aggregation

Aggregation settings, only consulted in the mirror strategy. merge produces one clean answer that continues through the normal Jury/guard pipeline; collect returns every candidate’s answer verbatim so you can choose yourself, skipping the merger and Jury entirely.

| Field | What it does | Default |
| Mode | Aggregation applied to the fanned-out answers. Set via the same top-level Mode dropdown described under [Decider](#decider). The option chosen there (“Compare output from multiple LLMs” or “Summarise output from multiple LLMs”) sets both the strategy and this field together. | Merge |
| Merger LLM | The LLM that synthesizes candidate answers into one. Required in merge mode; ignored in collect mode. Matches the dashboard’s Merger LLM section heading. | Not set |
| Merge Instructions | System prompt steering the merge; falls back to a built-in synthesis instruction when empty. Candidate answers are appended automatically. Labeled Merge instructions in the dashboard, not “Merge System Prompt”. | Empty |
| Disable Thinking | Disable chain-of-thought for a reasoning merger model, as a toggle. Matches the dashboard’s Disable thinking switch. | Off |

Aggregation modes:

| Value | Behaviour |
| merge (default) | A dedicated merger LLM synthesizes every candidate answer into one best answer, which then continues through the Jury/guard pipeline as normal. |
| collect | Every candidate answer is returned verbatim so a human can compare and choose. No merger and no Jury run; the response carries all answers. |

Validation: an enabled Decider must have at least one candidate with a unique, non-empty Candidate ID; a set Default Candidate (Fallback) must reference a real candidate; the route strategy requires a Router LLM; mirror in merge Mode requires a Merger LLM; mirror candidate count is capped at 10.

## Routing rules

Routing rules (base config only, not variant-overridable) implement attribute-based routing, gated by a separate enabled switch (default on). Rules are evaluated in order (first match wins), only when the caller did not explicitly address a variant via $alias, and they take precedence over a [canary](#canary) split. When the switch is off, the rules are retained (so the surface round-trips and can be re-enabled without rework) but never evaluated.

- header is the most explicit signal, usable when the caller (or a proxy in front of them) can set a custom header.

- body_field keys off something already present in the request body, like a caller-supplied tag.

- modality gives deterministic, LLM-free routing to a modality-capable variant: for example, sending any request carrying an image straight to a vision-capable variant without needing an LLM to infer that from content.

Each rule: Match (a match condition; the dashboard shows this as an unlabeled row, a kind selector plus its name/value fields, rather than a field captioned “Match”), Target (the dashboard row is prefixed “→ variant” rather than captioned “Target”).

Match kinds:

| Kind | Fields | Matches when |
| header | Name, Value | A request header (matched case-insensitively) equals the value. |
| body_field | Field, Value | A top-level request-body string field equals the value. |
| modality | Modality | The request carries a given modality: text, image, audio, document (detected from the payload on the chat path), or realtime (a transport signal, matched only on the /v1/realtime path). Enables deterministic, LLM-free routing to a modality-capable variant. |

Target: Variant (alias to route to; preferred when set; picked from the dashboard’s “→ variant” dropdown), or an inline provider target (used only when no variant is set). At least one of the two must be present. The dashboard’s routing-rule editor does not expose the inline-target option; it can only be set through the API, and the UI dropdown is disabled with an “(inline target)” note when one is present.

Validation: a header/body-field rule requires both a non-empty name/field and value; a modality rule must name one of the five recognized modalities; a rule’s target must resolve to either a known variant alias or a complete inline provider target.

## Canary

Canary (base config only) implements an A/B traffic split across a surface’s variants, letting you prove a new model or variant is safe with real traffic before committing to it fully. Instead of an all-or-nothing cutover, a small percentage of requests see the new option while the rest keep hitting your known-good baseline.

- Start small (5–10% is a reasonable first split) and watch the Monitoring tab’s error rate, latency, and cost for that variant against the baseline.

- Once its telemetry looks comparable or better, increase the split in steps rather than jumping straight to an even or majority share, so a regression only ever affects a small slice of traffic before you notice it.

| Field | What it does | Default |
| Enable Canary Split | Master switch. When enabled and a request does not explicitly address a variant, the gateway picks a target by weight. Labeled Enable canary split in the dashboard, not “Enabled”. | Off |
| Splits | Weighted routing targets. Weights are relative, normalized against their sum. The dashboard shows no field captioned “Splits”. Instead, one row per target (base surface plus each variant) with a percentage input; entering a value rebalances the others so they always total 100%. | None added |

Each split: Variant (alias to route to; empty/absent routes to the base surface; shown in the dashboard as the row’s plain-text label, “Base surface” or the variant’s alias, not an editable field), Weight (relative weight, so this target’s probability is its share of the total; shown in the dashboard as a percentage input with no “Weight” caption, constrained to sum to 100% rather than an arbitrary relative weight).

Validation: an enabled canary requires at least one split with a positive total weight, and every named variant must be a known alias.

## Variants

Each surface carries a catalogue of named variants plus a default variant naming which one an alias-less request resolves to (when unset, alias-less requests resolve to the bare base surface). A variant is addressed at {route}$alias/....

Each variant:

| Field | What it does |
| Alias | URL-safe alias used in {route}$alias/.... Must be lowercase letters, digits, and dashes, 1–32 characters. |
| Name | Human-readable name shown in the dashboard. |
| Description | Free-text description. |
| Enabled | Toggle, on by default. A disabled variant is still resolvable for UI display, but the proxy answers a request that selects it with HTTP 503. |
| Overrides | The sparse delta over the base surface (below). No single dashboard field is captioned “Overrides”. A variant’s overrides are set implicitly by switching to it via the variants widget on the canvas and editing its own copies of the surface’s elements. |

A variant’s overrides never touch the surface’s identity (id, name, listener, route, metrics/log toggles, integrations), since those define where the variant is reachable and how it is observed. Overrides built in the dashboard always use frozen-snapshot semantics: once you finish editing a variant, its overrides are locked in and won’t later pick up changes made to the base surface.

The dashboard’s variant editor only ever writes a specific subset of the blocks a variant can technically override. These are dashboard-reachable:

| Overridable block | What it overrides |
| Pipe Type | The upstream: provider, model, endpoint, and API key. |
| Override Policy | LLM parameter override policy. |
| Prompt Guard, Judge, Jury | The corresponding guardrail block. |
| Rate Limit | The corresponding resilience block. |
| Cost Tracking | Cost-tracking enabled state and price overrides. |
| OPA Policy | The surface’s Rego policy attachment. |
| Realtime, Audio Input, Image Input, Document Ingestion | The corresponding modality gate: lets one variant front realtime voice while siblings stay chat-only, for example. |
| Canvas | Per-variant override of the dashboard’s opaque canvas blob. |

The remaining blocks are real, backend-supported override fields, but the dashboard’s variant editor never writes any of them: a dashboard-built variant always resolves these as empty, regardless of what the base surface has configured, and there is currently no dashboard control to set them per variant.

| Backend-only override (not settable from the dashboard) | What it overrides |
| Expert Witnesses | Setting this wholesale-replaces the base witness list (an empty list clears all witnesses); leaving it unset inherits the base list. |
| Usage Alerts | The breach and anomaly alert configuration. |
| Response Cache | The response cache. |
| Source Authentication | The surface’s caller-authentication method. If your base surface relies on it, do not assume a dashboard-built variant inherits it. |
| Forward Caller Token | Leaving it unset inherits the base; setting it explicitly sets identity pass-through for this variant. |
| Request Timeout Seconds | HTTP request timeout. |
| Circuit Breaker | The circuit-breaker configuration. |
| Retry Policy | The retry-with-backoff configuration. |

Sparse, the default, suits a variant that should keep picking up base-surface changes over time: leaving a field unset means “inherit from base.” Frozen snapshot is for a variant that must not silently inherit later edits to the base surface, even edits that clear a field.

| Merge mode | Semantics |
| Sparse (default, legacy) | Leaving an overridable field unset means “inherit from base”; setting it means “use this value.” A variant that only sets its upstream provider swaps that and inherits everything else. |
| Frozen snapshot | Every overridable field is wholesale-replaced from the override, including fields explicitly cleared (which override the base’s value with nothing). Non-optional base fields (like the upstream provider and request timeout) still fall back to base when the override leaves them unset. Use this when the variant must not silently inherit later edits to the base surface. |

## Related

- [Guardrails](/products/affinidi-trust-fabric/agent-stream/reference/surfaces/guardrails.md): The provider/model/endpoint target shape shared by Decider candidates, Judge, and Jury.

- [Resilience and caching](/products/affinidi-trust-fabric/agent-stream/reference/surfaces/resilience-and-caching.md): Failover and load-balancing targets, which also use that shape.

- [Variants and progressive rollout](/products/affinidi-trust-fabric/agent-stream/concepts/variants-and-rollout.md): The conceptual model for routing rules, canary, and variants.

- [LLM drift detection](/products/affinidi-trust-fabric/agent-stream/reference/observability/llm-drift.md): How a replay evaluation’s target surface or variant compares to routing rules and canary as a way to evaluate a change.