Overview

Agent Stream runtime architecture

What is Agent Stream?

Agent Stream is part of the Affinidi Trust Fabric suite of products, services, and tools for identifying, managing, governing, auditing, and controlling AI-agent and AI-adjacent traffic across organisations and clouds.

Technically, Agent Stream is an intercepting proxy for governing LLM service traffic. It sits between your agents and the models, APIs, and other services they depend on, applying policy evaluation, guardrails, cost and usage governance, and observability to every call. Your applications keep their existing model-call patterns; point them at Agent Stream instead of a provider endpoint.

As organisations move from pilot agents to production workloads, execution spreads across providers with different limits, cost profiles, and failure modes. Agent Stream gives you one control point for deciding which provider handles a request, what a call may send and receive, what it may cost, and how every execution is attributed.

Where Agent Stream sits

An application or the agent inside it calls an Agent Stream endpoint rather than a provider directly. Agent Stream can inspect, validate, transform, meter, cache, and route that request to one provider or across a provider pool. At every stage, it applies the configured budgets, allow-lists, guardrails, and policy, then records the result for operational review and audit.

This works for a single team routing one application to one LLM provider. It also works for multi-provider, multi-tenant deployments where many teams, agents, and end users share a governed model estate under central safety controls and budgets.

Govern model execution without rewrites

Agent Stream provides a single OpenAI-compatible interface for multi-provider LLM access. It combines AI-specific controls with the resilience features needed to operate critical model traffic:

• Safety and policy: layered guardrails and content safety, LLM-as-judge and LLM-as-jury review, prompt templates, and policies derived from identity-provider integrations such as Entra and Okta.
• Routing and performance: intelligent content-based routing, parallel model runs for comparison and aggregation, exact and semantic response caching, weighted and latency-based load balancing, retries, circuit breakers, and cross-provider failover.
• Cost and operations: per-team and per-member cost attribution, budgets and quotas, secrets and API key management, rate limiting, real-time monitoring, and enterprise integrations.

Two surface types

A surface is the configuration and runtime unit for a governed connection through Agent Stream. The LLM Surface is the primary surface type: it fronts one LLM provider and carries the full pipeline of guardrails, routing, resilience, cost governance, and observability. The complementary IDE Surface aggregates one or more LLM Surfaces into a governed model-discovery menu for IDE clients and OpenAI-compatible agents.

See Surfaces → for how the two types relate, and IDE Surface → for the editor-facing model menu.

Use the browser-based management interface to configure and operate surfaces. You can compose a surface interactively on its canvas, for example by adding a Prompt Guard to redact or block sensitive patterns, an Expert Witness to consult a safety classifier, and a Judge or Jury for reasoning-based review. Surface configuration is also available as JSON for developer-oriented workflows.

Key capabilities

Every capability below attaches to a surface and composes with the others. Each links to its full concept page.

Govern every model call

Attribute usage and secure access

Extend governed access to your organisation

Built on the Trust Fabric

Agent Stream shares its appliance foundation with Agent Gateway, including the underlying identity, policy, and governance model. An operator familiar with one appliance can use the same concepts in the other. Connecting Agent Stream with Agent Gateway installations, Trust Registries, and other Fabric-native services creates secure, auditable, and verifiable interactions between agents, people, and services across organisational trust boundaries.

Deploy and observe

Agent Stream integrates with enterprise observability and delivery systems through OpenTelemetry metrics, logs, and traces; native Langfuse exports for LLM traces and generations; and a Prometheus scrape endpoint. It can also deliver events to email, Slack, generic webhooks, and streaming systems including Kafka, Kinesis, Pulsar, and Redis Streams.

For identity, access, and federation in depth, see Security and access control →, OPA policies →, and Trust-fabric integration →.

Adopt it progressively