Control cost and usage
Go beyond the single cumulative budget from Add guardrails and a budget: cap spend per pipeline stage, get notified before a breach becomes a bill shock, and avoid provider cost altogether on repeated prompts. Field-level reference for every block here is in Cost and usage limits →.
| Guide | What you will achieve |
|---|---|
| Cap spend per pipeline stage | Set a separate monthly budget or token cap on the LLM call, Judge, and Jury stages, so one stage’s spend cannot exhaust a shared cumulative limit. |
| Alert on budget breaches and cost spikes | Route a cost, token, error-rate, or cost-spike alert to Slack, email, or a webhook, so a person is notified without the request being blocked. |
| Cache repeated responses to cut cost and latency | Serve an identical or near-duplicate request from an in-memory cache instead of calling the provider again. |
Was this page helpful?
Glad to hear it! Please tell us how we can improve more.
Sorry to hear that. Please tell us how we can improve.
Thank you for sharing your feedback so we can improve your experience.