# Architecture > How Agent Stream sits between your applications and the LLM providers and services they call, and how it is deployed as a single, hot-reloadable appliance. When every application and agent embeds its own provider keys, retry logic, and safety checks, cost, guardrails, and reliability drift out of sync across the organisation, and no single place shows what any of them are actually doing. Agent Stream sits between your AI applications and agents and the LLM providers and external services they depend on. It gives you one place to apply guardrails, cost governance, resilience, and observability across every model call, without rebuilding those controls in every application. [Surfaces →](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md) ## Where Agent Stream sits At a high level, Agent Stream acts as a control layer between: - your applications, agents, and editor integrations. - the LLM providers and external services they call out to. - the guardrails, budgets, and policies your organisation needs to enforce. Instead of embedding provider keys, retry logic, and safety checks into each application, you centralise those responsibilities in Agent Stream and point your existing OpenAI-compatible client at a surface route instead of the provider itself. ## How the appliance is organised into four layers For most operators, the architecture is easiest to understand in four layers, sitting between callers and the providers they reach. Callers Applications, agents, editors request Affinidi-hosted appliance 01 Entry layer LLM Surface · IDE Surface 02 Governance layer Source auth · Rate limit · Team gate · OPA policy · Guardrails 03 Routing layer Decider · Variants · Failover · Cache 04 Operations layer Metering · Governance records · Telemetry governed call Providers Models and external services Control plane Dashboard · RBAC · Secrets · Alerts · Integrations Each box in the diagram is a concept covered elsewhere: - Entry: [Surfaces →](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md). - Governance: [Security and access control →](/products/affinidi-trust-fabric/agent-stream/concepts/security-and-access-control.md) and [OPA policies →](/products/affinidi-trust-fabric/agent-stream/concepts/opa-policies.md). - Routing: [Variants and progressive rollout →](/products/affinidi-trust-fabric/agent-stream/concepts/variants-and-rollout.md) and [Resilience and caching →](/products/affinidi-trust-fabric/agent-stream/concepts/resilience-and-caching.md). - Operations: [Cost and usage governance →](/products/affinidi-trust-fabric/agent-stream/concepts/cost-and-usage-governance.md), [Governance records →](/products/affinidi-trust-fabric/agent-stream/concepts/governance-records.md), and [Observability →](/products/affinidi-trust-fabric/agent-stream/concepts/observability.md). ## How live traffic and configuration stay isolated The four layers above handle every request that rides through a surface, its data plane. Configuring the appliance itself runs on a separate control plane, so a caller can never reach an administrative function, and an operator’s session can never see live traffic without the right permission. - Dashboard and RBAC. Every configuration change, surface, policy, secret, or role, goes through the same dashboard, gated by an ordered role ladder: administrator, poweruser, and user. Sensitive mutations, editing a surface, publishing a policy, or managing secrets and roles, default to administrator-only, so a missing or incomplete role assignment fails toward the more restrictive setting there; other areas, such as pipes, notifications, departments, and payments, default to broader user-level editing instead. - Secrets store. Provider API keys and other credentials resolve from an encrypted secrets store and are referenced by ID from a surface, never embedded directly in configuration. - Alerts and integrations. Breach alerts, budget events, and lifecycle notifications route to email, Slack, webhooks, or a streaming bus, independently of the request path itself, so wiring up an alert never touches the data plane. Reading is tiered the same way writing is: aggregate, per-surface figures sit behind one permission tier, per-member and per-team breakdowns behind a broader one, and governance-record content behind its own dedicated permission, since a record can contain the raw prompt and completion text. See [Security and access control →](/products/affinidi-trust-fabric/agent-stream/concepts/security-and-access-control.md) for the full role ladder. ## Traffic Agent Stream governs Every call that rides through a surface passes through the same governed pipeline: request guards, routing, the provider call, response guards, metering, and telemetry. See [Pipeline and stages →](/products/affinidi-trust-fabric/agent-stream/concepts/pipeline-and-stages.md) for the full stage order. - LLM Surface traffic: chat, embeddings, moderation, rerank, image generation, audio, batch, and file calls to a configured provider. - IDE Surface traffic: model discovery and dispatch calls from editors and OpenAI-compatible agents, which Agent Stream forwards to the appropriate LLM Surface. Anything not configured on a surface is simply skipped, so a minimal surface behaves as a thin, low-latency proxy, and a fully configured one behaves as a governed, guard-railed, multi-model router, using the same request path either way. ## Request lifecycle At the customer level, every call follows the same straightforward sequence, regardless of how many stages a surface has switched on. See [Pipeline and stages →](/products/affinidi-trust-fabric/agent-stream/concepts/pipeline-and-stages.md) for the full, named stage order. - Receive the request. Agent Stream accepts the call at the surface’s route, in place of the provider’s own endpoint. - Authenticate and gate the caller. The caller’s JWT, API key, or mTLS identity is verified, then checked against gateway policy, team quota, and rate limits. - Apply guardrails and choose a route. Prompt Guard, expert witnesses, and the Judge inspect the request while the surface’s routing mode selects a model, or fans out to several, all before any provider is called. See [Pipeline and stages →](/products/affinidi-trust-fabric/agent-stream/concepts/pipeline-and-stages.md) for the Direct, Decider, Comparer, and Summariser modes. - Call the provider. The request reaches the selected provider, with retry, circuit breaking, failover, or load balancing applied as configured. - Record what happened. Agent Stream meters cost and tokens, emits traces and metrics, and, if enabled, signs a governance record of the whole interaction. ## Deployment and operations Agent Stream runs as a fully managed appliance hosted by Affinidi, so a team manages behaviour centrally through the dashboard instead of redeploying every client and service individually. In practice, that means you can: - edit a surface live from the dashboard and see the change apply immediately, since configuration is authoritative in an in-memory cache and only mirrored to disk, not the other way round. - export configuration and governance data from the dashboard with PII redaction and encryption applied, for your own backup or audit records. Your appliance is also provisioned with entity-count limits tied to your commercial tier. See [Cost and usage governance →](/products/affinidi-trust-fabric/agent-stream/concepts/cost-and-usage-governance.md) for how these limits work. ## What this means for your organisation At a business level, Agent Stream gives you a repeatable architecture for governed AI spend and safety. Instead of treating each integration as a special case, you adopt one control point that makes cost, guardrails, and observability consistent across every team calling a model. That is especially valuable when you are: - scaling from one team’s LLM integration to many teams calling many providers. - consolidating scattered provider keys and ad-hoc safety checks into one place. - rolling out governed model access to a whole engineering organisation through their editors. ## Related - [Surfaces](/products/affinidi-trust-fabric/agent-stream/concepts/surfaces.md): The LLM Surface and IDE Surface routing model. - [Pipeline and stages](/products/affinidi-trust-fabric/agent-stream/concepts/pipeline-and-stages.md): The named stages a request passes through. - [Security and access control](/products/affinidi-trust-fabric/agent-stream/concepts/security-and-access-control.md): The RBAC role ladder and secrets store behind the control plane. - [Observability](/products/affinidi-trust-fabric/agent-stream/concepts/observability.md): The telemetry the appliance produces and how to export it.