Providers and Models

How Agent Stream uses provider adapters and a hot-reloadable catalogue to select models, pricing, capabilities, and request profiles for LLM Surfaces.

Hard-coding a provider’s SDK, pricing, and capability quirks into every application makes it slow and risky to add a new model or switch vendors. Agent Stream reaches a broad set of LLM providers through a catalogue of provider adapters. An LLM Surface selects a provider and model when it is configured, or defers that choice to a Decider or load balancer at request time. The catalogue supplies pricing, capabilities, and request profiles, so a new model can be onboarded through configuration. Surface Variants →

Provider

A provider is the vendor adapter behind an LLM Surface. Native providers include OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Microsoft 365 Copilot, and Ollama.

First-class OpenAI-compatible presets include Groq, xAI, OpenRouter, Fireworks, Cerebras, DeepSeek, Moonshot (Kimi), SambaNova, Hyperbolic, NVIDIA NIM, and Nebius. A Custom provider accepts any OpenAI-compatible endpoint URL.

Model

A model is a specific identifier within a provider’s catalogue, such as gpt-5.5 on OpenAI, claude-sonnet-4-6 on Anthropic, or gemini-3.5-flash on Google Gemini. The catalogue supplies each model’s input-token, output-token, and optional prompt-cache pricing, as well as capabilities such as streaming, tools, and vision, and any request-profile transforms the provider needs.

Catalogue

The catalogue is the hot-reloadable provider and model configuration file, llm.json. It supplies provider request profiles, model pricing, and capabilities. Editing and reloading the catalogue makes a new model available across every LLM Surface without a code change.

Provider presets

A provider preset is a first-class shortcut for an OpenAI-compatible provider, such as Groq or Fireworks. It supplies the base URL and default headers, so an operator only needs to configure the key. Presets turn an OpenAI-compatible URL into a ready-to-select provider choice.

An LLM Surface's provider configuration with Groq selected as a first-class preset, showing its endpoint, model, and API key secret

Custom provider

A custom provider is an OpenAI-compatible endpoint not covered by a preset. An administrator adds it once, as JSON, in the LLM Providers catalogue: a unique identifier, display name, base URL, and the label to show for its API key field. Once saved, it appears in every provider picker across the appliance exactly like a built-in provider, and any surface can select it. There is no separate base-URL or authentication-header field to fill in per surface: a custom provider authenticates through the same secret-reference pattern as every other provider.

The Custom LLM Providers catalogue editor, with a draft JSON definition for a new provider's identifier, name, base URL, and API key label

How Microsoft 365 Copilot grounds answers without exposing raw documents

Microsoft 365 Copilot is a specialized provider adapter, m365_copilot. Agent Stream translates an incoming OpenAI chat call into a Microsoft 365 Copilot Chat API call made as the signed-in user, forwarding a delegated Entra token through identity pass-through.

Copilot grounds its response on the user’s OneDrive, SharePoint, mail, and calendar content while respecting their Entra access control lists. It returns only the synthesized answer: the agent, editor, and Agent Stream do not receive the raw documents.