Cost and usage limits

See Cost and usage governance → for the conceptual model. This page is the field reference for the four related blocks on a surface.

Rate limiting

Rate limiting throttles requests before they reach the provider, stopping a burst of traffic (a caller retry loop, a runaway script, an unexpected spike) from reaching it at all, and protecting both your own request budget and the provider’s own rate limits. There is no separate enabled toggle: dragging the Rate Limit element onto the canvas is what turns it on, and removing the element turns it off. Requests are counted per one-minute window.

Cost tracking

The LLM, Judge, Jury, and Decider stages always count both spend and tokens. They are the only metered stages: Expert Witnesses and Prompt Guard are not cost stages. Cost tracking has no dashboard toggle.

By default, a call is priced from the model catalogue on the Models page, and the LLM node’s sidebar shows the selected model’s catalogue price per 1K input and output tokens. To price an upstream yourself, declare a price in USD per million tokens in the upstream_pricing field of the surface’s Config JSON view. Setting a price needs the surface pricing permission, which defaults to Administrator. See Providers: token pricing resolution → for the full order.

Usage limits

Usage limits are the per-stage spend/token guardrail mechanism: a hard ceiling that blocks (HTTP 429) further calls for its scope once spend or tokens have reached a monthly cap, enforced pre-call, unlike usage alerts (below), which only notify. The request that crosses the cap goes through, and the requests after it are blocked. A surface usage limit answers with HTTP 429 and the error code rate_limit_exceeded. Reach for this when a runaway surface, stage, or juror must not be allowed to keep spending past a ceiling you’ve set.

You don’t pick a scope or stage directly: the dashboard sets both automatically from where you configure the limit. Set it in the Surface-wide usage limit section of the Surface Configuration editor and it covers the whole surface; set it on a specific LLM, Judge, or juror panel and it covers only that stage. The surface-wide section is available once the surface has an enabled Judge, Jury, or Decider; with only the LLM, set the limit on the LLM node instead. There is no usage-limit control on the Decider panel or on the Jury panel as a whole: only an individual juror’s own panel exposes one, capping that juror alone, not the jury stage together.

• Use the surface-wide limit for a blanket ceiling across everything the surface does; set it on a specific LLM, Judge, or juror panel when only that one stage needs its own cap.
• To pick a number for Monthly budget (USD), look at the surface’s own Monitoring-tab cost chart (or, for a caller-facing surface, historical x-agent-stream-cost-usd totals from response headers) for a recent typical month, then add headroom above that for expected growth. This is a hard, request-blocking ceiling, so leave enough margin that a normal usage spike doesn’t trip it.
• Source Monthly token cap the same way, from recent monthly token totals on the Monitoring tab.

Validation: at least one of Monthly budget (USD) and Monthly token cap must be set, and a budget can’t be negative.

Usage alerts

Usage alerts send a notification through an integration when a threshold or anomaly condition is crossed, and let the request continue. It’s a deliberately separate mechanism from usage limits, so you get notified with time to react instead of finding out only when requests start failing. Alerts are set in the Breach & anomaly alerts section of the Surface Configuration editor. Its Enable breach & anomaly alerts switch starts off; turn it on to add alerts. When it’s off, existing alerts are kept but don’t fire.

• Cost per request and Tokens per request each evaluate one single request, not cumulative spend: they fire the moment any one execution’s own cost or token count exceeds the threshold. They are not an early-warning proxy for a monthly usage limit; a threshold set as a fraction of a monthly budget will rarely, if ever, fire, since a single request’s cost is normally far below a monthly ceiling. Use them to catch one unexpectedly expensive call, for example an oversized prompt or an unbounded completion.
• Error rate and Cost spike catch a problem a flat ceiling wouldn’t: a sudden jump in failures, or in per-request cost relative to the surface’s own recent baseline, rather than cumulative spend.

Each alert:

Alert types and the corresponding threshold meaning:

The value in brackets is the alert type an integration receives in its ALERT_TYPE template variable.

Related

• Providers: token pricing resolution: How catalogue and override pricing combine to price a call.
• Teams and attribution: The team attribution block on a surface, and how spend rolls up per member and team.
• Cost and usage governance concept: Why usage limits and breach alerts are two different mechanisms, and how appliance tier limits differ from both.