Stop teams eating the budget
When every caller shares one provider key, model spend is a single number that nobody can break down. One runaway integration or a single busy team can consume the budget, and you often only find out when the invoice arrives. Without per-caller attribution, there is no way to cap usage in advance or attribute it to the department that incurred it.
What you can do
- Set monthly USD budgets and token caps per member, team, or surface so that a runaway integration is stopped before it produces a large bill.
- Set per-minute request and token limits per member, and a requests-per-minute ceiling on a surface, so that one caller’s burst of traffic is throttled before it exhausts shared capacity.
- Attribute every token and dollar to a member and their teams so that you can answer who spent what, on which model, and charge AI spend back to the team that incurred it.
- Route cheaper requests to smaller models and reserve frontier models for the hard ones so that you right-size the cost of every call.
- Get alerted on cost spikes and threshold breaches so that a sudden anomaly reaches the people who can act, not just a dashboard nobody is watching.
How it works
On a surface with team attribution on, each caller resolves to a Member, or to the built-in anonymous Member when it can’t be matched. A Member acts as a virtual key carrying a model allow-list, per-minute request and token limits, and monthly ceilings. Members are grouped into Teams, and once a surface is set to contribute to team quotas, spend and tokens are attributed to both the Member and its Team across every eligible surface. Each Team’s Usage tab shows what it has spent.
On each request the Team Gate resolves the caller, checks the model allow-list, then enforces per-minute limits and monthly ceilings before the provider is called. Teams carry monthly ceilings only, while per-minute limits sit on each Member and, through the Rate Limit element, on the surface as a whole. Cost, tokens, and latency are metered per stage, so the LLM call, the Judge, the Jury, and the Decider each show their own share rather than one combined figure. With Mode set to Decide LLM based on content, the Decider reads each request and sends simple ones to a cheaper model and complex ones to a frontier model.
Usage limits and breach alerts are separate mechanisms: a limit is a hard ceiling, while a breach alert only notifies. Once spend or tokens reach a monthly ceiling, further requests are rejected with a 429 before they reach the provider, and the request that crosses the ceiling is itself allowed through. A request over a member’s per-minute limit is also rejected with a 429, and an access or model-allow-list failure returns a 403.
Related
- Cost and usage governance: How per-stage metering, monthly ceilings, and anomaly alerts turn spend into something you can cap.
- Cost and attribution: How members and teams act as virtual keys and attribute spend.
- Cap spend and rate per member: Step-by-step configuration of per-member budgets, token caps, and rate limits.
- Alert on budget breaches and cost spikes: How to route threshold and anomaly notifications to your existing integrations.
Glad to hear it. Thanks for letting us know.
Sorry to hear that. Thanks for letting us know.
Still stuck? Ask the community on Discord or contact us.