# Overview > What Agent Stream is, how it governs model execution across providers with routing, guardrails, budgets, and observability, and how to adopt it progressively. ## Agent Stream runtime architecture ## What is Agent Stream? Agent Stream is part of the [Affinidi Trust Fabric](https://www.affinidi.com/trust-fabric) suite of products, services, and tools for identifying, managing, governing, auditing, and controlling AI-agent and AI-adjacent traffic across organisations and clouds. Technically, Agent Stream is an intercepting proxy for governing LLM service traffic. It sits between your agents and the models, APIs, and other services they depend on, applying policy evaluation, guardrails, cost and usage governance, and observability to every call. Your applications keep their existing model-call patterns; point them at Agent Stream instead of a provider endpoint. As organisations move from pilot agents to production workloads, execution spreads across providers with different limits, cost profiles, and failure modes. Agent Stream gives you one control point for deciding which provider handles a request, what a call may send and receive, what it may cost, and how every execution is attributed. ## Where Agent Stream sits An application or the agent inside it calls an Agent Stream endpoint rather than a provider directly. Agent Stream can inspect, validate, transform, meter, cache, and route that request to one provider or across a provider pool. At every stage, it applies the configured budgets, allow-lists, guardrails, and policy, then records the result for operational review and audit. This works for a single team routing one application to one LLM provider. It also works for multi-provider, multi-tenant deployments where many teams, agents, and end users share a governed model estate under central safety controls and budgets. ## Govern model execution without rewrites Agent Stream provides a single OpenAI-compatible interface for multi-provider LLM access. It combines AI-specific controls with the resilience features needed to operate critical model traffic: - Safety and policy: layered guardrails and content safety, LLM-as-judge and LLM-as-jury review, prompt templates, and policies derived from identity-provider integrations such as Entra and Okta. - Routing and performance: intelligent content-based routing, parallel model runs for comparison and aggregation, exact and semantic response caching, weighted and latency-based load balancing, retries, circuit breakers, and cross-provider failover. - Cost and operations: per-team and per-member cost attribution, budgets and quotas, secrets and API key management, rate limiting, real-time monitoring, and enterprise integrations. ## Two surface types A surface is the configuration and runtime unit for a governed connection through Agent Stream. The LLM Surface is the primary surface type: it fronts one LLM provider and carries the full pipeline of guardrails, routing, resilience, cost governance, and observability. The complementary IDE Surface aggregates one or more LLM Surfaces into a governed model-discovery menu for IDE clients and OpenAI-compatible agents. | Surface type | What it fronts | | LLM Surface | One configured model provider: OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Cohere, Mistral, Together AI, Replicate, Hugging Face, Ollama, a set of OpenAI-compatible presets (Groq, xAI, OpenRouter, Fireworks, Cerebras, DeepSeek, Moonshot, SambaNova, Hyperbolic, NVIDIA NIM, Nebius), Microsoft 365 Copilot, or any custom OpenAI-compatible endpoint. | | IDE Surface | A governed, per-caller-filtered model menu that aggregates one or more LLM Surfaces behind a single OpenAI-compatible endpoint, for editors such as VS Code and GitHub Copilot Chat. | See [Surfaces →](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md) for how the two types relate, and [IDE Surface →](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md#ide-surface) for the editor-facing model menu. Use the browser-based management interface to configure and operate surfaces. You can compose a surface interactively on its canvas, for example by adding a Prompt Guard to redact or block sensitive patterns, an Expert Witness to consult a safety classifier, and a Judge or Jury for reasoning-based review. Surface configuration is also available as JSON for developer-oriented workflows. ## Key capabilities Every capability below attaches to a surface and composes with the others. Each links to its full concept page. ### Govern every model call [ Guardrails → How Prompt Guard, Expert Witnesses, Judge, and Jury layer content safety and PII protection around a call. ](/products/affinidi-trust-fabric/agent-stream/concepts/guardrails.md) [ PII protection and NER IDs → How Prompt Guard detects sensitive data with pattern and AI matching, and pseudonymizes it with NER IDs instead of destroying it with static redaction. ](/products/affinidi-trust-fabric/agent-stream/concepts/pii-protection.md) [ Policies → How gateway-level and surface-level Rego policy decide whether a request is allowed. ](/products/affinidi-trust-fabric/agent-stream/concepts/opa-policies.md) [ Cost and usage governance → How Agent Stream meters cost and tokens per stage and enforces budgets before they are exceeded. ](/products/affinidi-trust-fabric/agent-stream/concepts/cost-and-usage-governance.md) [ Resilience and caching → How failover, load balancing, and response caching keep calls fast and available. ](/products/affinidi-trust-fabric/agent-stream/concepts/resilience-and-caching.md) ### Attribute usage and secure access [ Cost and attribution → How members and teams apply virtual-key controls, attribute spend and usage, and enforce ceilings for quota governance. ](/products/affinidi-trust-fabric/agent-stream/concepts/teams-and-attribution.md) [ Identity → How Agent Stream distinguishes independent identity concerns, authenticated identities, OAuth identifiers, dual tokens, and identity hash peppers. ](/products/affinidi-trust-fabric/agent-stream/concepts/security-and-access-control.md) [ Governance records → How a signed, tamper-evident Verifiable Credential and hash chain prove what entered a surface, how it was processed, and what left. ](/products/affinidi-trust-fabric/agent-stream/concepts/governance-records.md) [ Variants and progressive rollout → How surface variants, routing rules, and canary splits let you change behaviour safely on a stable endpoint. ](/products/affinidi-trust-fabric/agent-stream/concepts/variants-and-rollout.md) ### Extend governed access to your organisation [ IDE Surface → How a governed, per-caller-filtered model menu lets editors and OpenAI-compatible agents consume approved models under corporate sign-in. ](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md#ide-surface) [ Observability → What telemetry Agent Stream produces, what the dashboard shows, and how to export it to your own stack. ](/products/affinidi-trust-fabric/agent-stream/concepts/observability.md) [ Trust-fabric integration → How gateway-to-gateway DIDComm, the model marketplace, and x402 payments put Agent Stream on the trust fabric. ](/products/affinidi-trust-fabric/agent-stream/concepts/trust-fabric-integration.md) ## Built on the Trust Fabric Agent Stream shares its appliance foundation with [Agent Gateway](/products/affinidi-trust-fabric/agent-gateway/overview.md), including the underlying identity, policy, and governance model. An operator familiar with one appliance can use the same concepts in the other. Connecting Agent Stream with Agent Gateway installations, Trust Registries, and other Fabric-native services creates secure, auditable, and verifiable interactions between agents, people, and services across organisational trust boundaries. ## Deploy and observe Agent Stream integrates with enterprise observability and delivery systems through OpenTelemetry metrics, logs, and traces; native Langfuse exports for LLM traces and generations; and a Prometheus scrape endpoint. It can also deliver events to email, Slack, generic webhooks, and streaming systems including Kafka, Kinesis, Pulsar, and Redis Streams. For identity, access, and federation in depth, see [Security and access control →](/products/affinidi-trust-fabric/agent-stream/concepts/security-and-access-control.md), [OPA policies →](/products/affinidi-trust-fabric/agent-stream/concepts/opa-policies.md), and [Trust-fabric integration →](/products/affinidi-trust-fabric/agent-stream/concepts/trust-fabric-integration.md). ## Adopt it progressively Confirm pass-through Point a caller at an Agent Stream endpoint and confirm traffic flows through unchanged. Add your first LLM Surface Add credential injection, a budget cap, and guardrails. Add resilience Add routing and failover across providers. Watch it run Monitor executions, cost, and guardrail outcomes from the dashboard. [ Explore Concepts → Architecture, the request pipeline, guardrails, cost governance, and how Agent Stream extends to editors and Microsoft 365 Copilot. ](/products/affinidi-trust-fabric/agent-stream/concepts.md) [ Surfaces → How the LLM Surface and IDE Surface represent one governed connection to a provider or a governed model menu. ](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md) [ Pipeline and stages → The named stages every request passes through, and why an unconfigured stage adds no latency. ](/products/affinidi-trust-fabric/agent-stream/concepts/pipeline-and-stages.md)