Guides
These guides cover configuration and operational tasks in Agent Stream, organised by the same groups shown in the sidebar. Each builds on the basics from Get started.
Layer content safety
Go beyond the built-in Prompt Guard patterns from Add guardrails and a budget: preserve context in masked data, catch free-form PII a regex cannot express, and layer a pre-call and post-call review on top of pattern matching.
| Guide | What you will achieve |
|---|---|
| Pseudonymize PII with NER IDs | Replace matched PII with stable NER ID pseudonyms instead of a static placeholder, so a model can still reason over which values belong together, then restore the real values in the response. |
| Detect free-form PII with AI matching | Add a model-backed PII detector to a surface so names, addresses, and other free-form values are caught even when no regex pattern matches them. |
| Block and re-review calls with Judge and Jury | Add a Judge pre-check that stops an unsafe request before the model is called, and a Jury post-review that can approve, block, or regenerate the response before it returns. |
Control cost and usage
Go beyond the single cumulative budget from Add guardrails and a budget: cap spend per pipeline stage, get notified before a breach becomes a bill shock, and avoid provider cost altogether on repeated prompts.
| Guide | What you will achieve |
|---|---|
| Cap spend per pipeline stage | Set a separate monthly budget or token cap on the LLM call, Judge, and Jury stages, so one stage’s spend cannot exhaust a shared cumulative limit. |
| Alert on budget breaches and cost spikes | Route a cost, token, error-rate, or cost-spike alert to Slack, email, or a webhook, so a person is notified without the request being blocked. |
| Cache repeated responses to cut cost and latency | Serve an identical or near-duplicate request from an in-memory cache instead of calling the provider again. |
Route around failure
Keep a surface answering through a provider outage, spread load across a pool by cost or latency, and change behaviour safely on a stable endpoint.
| Guide | What you will achieve |
|---|---|
| Fail over to a backup provider on error | Add ordered backup providers to a surface, so a request retries against a backup instead of failing outright when the primary provider errors. |
| Balance traffic across providers by cost or latency | Distribute requests across a weighted pool of providers, selecting members by weighted round-robin, lowest observed latency, or lowest catalogue cost. |
| Roll out a new model with a canary split | Create a surface variant and route a small percentage of live traffic to it, so you can watch its cost, latency, and rejection figures before promoting it to every caller. |
Secure the appliance
Protect who can call a surface, who can administer the appliance itself, and the credentials it holds on your behalf.
| Guide | What you will achieve |
|---|---|
| Validate bearer tokens on a surface | Create a JWT verification strategy and attach it to a surface so only callers with a valid, correctly issued token can reach it. |
| Rotate provider credentials without downtime | Update a stored secret’s value in place so every surface referencing it picks up the new credential on its next request, with no surface reconfiguration. |
| Limit dashboard actions with RBAC roles | Assign the administrator, poweruser, or user role to a dashboard account so it can only perform the actions its job requires. |
Apply OPA policies
Decide not just whether a caller is authenticated, but whether that specific caller, claim, or credential is allowed to make this request at all.
| Guide | What you will achieve |
|---|---|
| Control access to a surface with an OPA policy | Write a Rego policy definition and attach it to a surface, so requests are allowed or denied based on claims, request attributes, and organisational rules. |
| Roll out an OPA policy globally with monitor-only mode | Enforce one policy across every surface of a type, watching what it would deny before it can block any real traffic. |
| Verify caller credentials with VP evidence | Gate a policy on a cryptographically verified issuer and claims, not just a bearer token, without re-implementing verification yourself. |
Attribute usage across teams and members
Turn a surface’s shared spend into per-caller and per-team accountability: know who is responsible for a call, and cap what an individual caller can spend.
| Guide | What you will achieve |
|---|---|
| Attribute usage to members and teams | Enable a surface’s team-attribution block and configure caller identity sources, so every call is attributed to a member and a team. |
| Cap spend and rate per member | Set a member’s monthly budget, per-minute rate caps, and model allow-list, so each caller behaves like its own virtual key. |
Glad to hear it! Please tell us how we can improve more.
Sorry to hear that. Please tell us how we can improve.
Thank you for sharing your feedback so we can improve your experience.