Overview

Agent Stream runtime architecture

What is Agent Stream?

Agent Stream is part of the Affinidi Trust Fabric suite of products, services, and tools for identifying, managing, governing, auditing, and controlling AI-agent and AI-adjacent traffic across organisations and clouds.

Technically, Agent Stream is an intercepting proxy for governing LLM service traffic. It sits between your agents and the models, APIs, and other services they depend on, applying policy evaluation, guardrails, cost and usage governance, and observability to every call. Your applications keep their existing model-call patterns, and you point them at Agent Stream instead of a provider endpoint.

As organisations move from pilot agents to production workloads, execution spreads across providers with different limits, cost profiles, and failure modes. Agent Stream gives you one control point for deciding which provider handles a request, what a call may send and receive, what it may cost, and how every execution is attributed.

Where Agent Stream sits

An application or the agent inside it calls an Agent Stream endpoint rather than a provider directly. Agent Stream can inspect, validate, transform, meter, cache, and route that request to one provider or across a provider pool. It applies the configured budgets, allow-lists, guardrails, and policy, records usage and outcomes for operational review and, on surfaces where governance recording is on, signs a record of each interaction for audit.

This works for a single team routing one application to one LLM provider. It also works for multi-provider, multi-tenant deployments where many teams, agents, and end users share a governed model estate under central safety controls and budgets.

Govern model execution without rewrites

Agent Stream provides a single interface for multi-provider LLM access that accepts OpenAI Chat Completions, Anthropic Messages, and OpenAI Responses requests. It combines AI-specific controls with the resilience features needed to operate critical model traffic:

• Safety and policy: layered guardrails and content safety, LLM-as-judge and LLM-as-jury review, prompt templates, and policies that act on caller identity from identity providers such as Microsoft Entra ID and Okta.
• Routing and performance: content-based and rule-based routing, parallel model runs for comparison and aggregation, exact and semantic response caching, load balancing by weight, latency, or cost, and cross-provider failover.
• Cost and operations: per-team and per-member cost attribution, budgets and quotas, secrets and API key management, rate limiting, monitoring, drift detection against a pinned baseline, and enterprise integrations.

Two surface types

A surface is the configuration and runtime unit for a governed connection through Agent Stream. The LLM Surface is the primary surface type: it fronts one LLM provider and carries the full pipeline of guardrails, routing, resilience, cost governance, and observability. The complementary IDE Surface aggregates one or more LLM Surfaces into a governed model-discovery menu for IDE clients and OpenAI-compatible agents.

See Surfaces → for how the two types relate, and IDE Surface → for the editor-facing model menu.

Use the browser-based management interface to configure and operate surfaces. You can compose a surface interactively on its canvas, for example by enabling the Prompt Guard to redact or block sensitive patterns, configuring an Expert Witness to consult a safety classifier, and enabling the Judge or Jury for reasoning-based review. Surface configuration is also available as JSON for developer-oriented workflows.

Key capabilities

Every capability below attaches to a surface and composes with the others. Each links to its full concept page.

Govern every model call

Attribute usage and secure access

Extend governed access to your organisation

Built on the Trust Fabric

Agent Stream shares its appliance foundation with Agent Gateway, including the underlying identity, policy, and governance model. An operator familiar with one appliance can use the same concepts in the other. Connecting Agent Stream with Agent Gateway installations, Trust Registries, and other Fabric-native services creates secure, auditable, and verifiable interactions between agents, people, and services across organisational trust boundaries.

Deploy and observe

Agent Stream integrates with enterprise observability and delivery systems through OpenTelemetry traces, metrics, and audit logs, and a Prometheus scrape endpoint for metrics. It can also deliver events to email, Slack, generic webhooks, and streaming systems including Kafka, Kinesis, Pulsar, and Redis Streams.

For identity, access, and federation in depth, see Security and access control →, OPA policies →, and Trust-fabric integration →.

Adopt it progressively