Cap spend per pipeline stage

This guide replaces or supplements a single cumulative usage limit with a per-stage limit on the LLM call, Judge, and Jury, so each stage is capped against its own budget. For the conceptual model, see Cost and usage governance →.

A cumulative limit, such as the one added in Add guardrails and a budget, caps a surface’s total spend across every tracked stage. Without a per-stage limit, a Jury running several jurors on every response can quietly consume most of that shared cap, leaving no budget headroom for the LLM calls themselves once the Jury’s own spend is subtracted.

Use this guide when:

• Your surface runs a Judge or Jury stage that could plausibly account for a large share of total spend.
• You need to answer “which stage is driving cost” with a hard ceiling, not just a chart.
• A single juror’s model is materially more expensive than the rest of the jury and needs its own ceiling.

You do not need per-stage limits if your surface runs only the LLM call, with no Judge, Jury, or Decider; a cumulative limit already covers the same spend.

Prerequisites

• An active LLM Surface. Per-stage limits are most useful once Judge and Jury or a Decider is already configured.
• Dashboard access with permission to edit surfaces.

Steps

Add a usage limit on the LLM stage

Under SURFACES in the dashboard sidebar, select LLM, open your surface, and select its LLM node. Turn on Enforce a usage limit and set Monthly budget (USD), Monthly token cap, or both, for this stage alone. Because you set it from this node’s panel, the limit applies to the LLM stage only.

Add a separate limit on the Judge stage

If your surface has a Judge, select its node, turn on Enforce a usage limit, and set its own Monthly budget (USD), Monthly token cap, or both. This limit is tracked independently of the LLM stage’s limit.

Add a limit on an individual juror

Select a juror’s own panel and turn on Enforce a usage limit there. The dashboard scopes a Jury usage limit to one juror at a time. A juror-scoped limit targets that juror by its position in the roster and does not affect the other jurors.

Decide whether to keep a cumulative limit as a backstop

A surface-level cumulative limit still caps total spend across every stage, in addition to any per-stage limits. Keep one configured if you want an overall ceiling even when individual stages have headroom left, or remove it if per-stage limits alone give you the coverage you need. The Decider has no per-stage limit of its own, so the surface-wide limit is what caps its spend.

To set it, select the Surface node, select Configure Surface Features, and under Surface-wide usage limit turn on Enforce surface-wide usage limit. Then set Monthly budget (USD), Monthly token cap, or both. You can turn this control on once the surface has a Judge, Jury, or Decider.

Save the surface

Select the save button (Save changes) under Manage. The limits apply from the next request. Before any stage runs, Agent Stream checks the limit of every stage the request could run. Once any one stage’s cap is reached, every request to the surface is refused until you raise that cap or it resets at the start of the next calendar month (UTC).

Confirm

Replace <YOUR_APPLIANCE_HOST> and <YOUR_SURFACE_ROUTE> with your surface’s values. Set a low Monthly budget (USD) on the Judge stage, such as 0.01, so you can see the limit trigger without waiting for real usage to accumulate.

Test 1: a request after the Judge stage’s cap is reached returns 429 before the LLM stage is charged

curl -k -X POST "https://<YOUR_APPLIANCE_HOST><YOUR_SURFACE_ROUTE>/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "user", "content": "Say hello." }
    ]
  }'

The -k flag disables TLS certificate verification. Use this for local testing only. Remove it in production.

Send the request until the Judge stage’s month-to-date spend has reached its cap. The request whose spend crosses the cap still completes. Expected: the next request returns HTTP 429 with the error code rate_limit_exceeded and a message naming the scope stage:judge, before any stage is called. The blocked request adds no tokens to the Main LLM card on the surface’s Monitoring tab.

Test 2: the Monitoring tab returns a separate cost for each stage

Open the surface’s Monitoring tab. The Total Cost card breaks spend for the selected window down by LLM, Judge, Jury, and Decider, and the Main LLM, Judge, and Jury cards each show that stage’s tokens. Together they confirm each stage is metered separately rather than in one pool.

Troubleshooting

Next steps

• Alert on budget breaches and cost spikes: Get notified before a stage’s spend reaches a hard cap, not just after.
• Cache repeated responses to cut cost and latency: Avoid provider spend altogether on repeated prompts.
• Cost and usage governance: Why usage limits and breach alerts are two different mechanisms.

Related

• Cost and usage limits reference: Every usage-limit field, including scope and stage values.
• Teams and attribution: How spend rolls up by team and member, on top of these per-stage caps.
• Block and re-review calls with Judge and Jury: Configure the Judge and jurors this guide puts a budget on.