Providers and Models

How Agent Stream uses provider adapters and a hot-reloadable catalogue to select models, pricing, capabilities, and request profiles for LLM Surfaces.

Hard-coding a provider’s SDK, pricing, and capability quirks into every application makes it slow and risky to add a new model or switch vendors. Agent Stream reaches a broad set of LLM providers through a catalogue of provider adapters. An LLM Surface selects a provider and model when it is configured, or defers that choice to a Decider, a load balancer, or a Router rule that picks a variant at request time. The catalogue supplies pricing, capabilities, and request profiles, so a new model can be onboarded through configuration. Surface Variants →

Provider

A provider is the vendor adapter behind an LLM Surface. The dashboard’s provider list comes from the model catalogue and includes Microsoft 365 Copilot, OpenAI, Anthropic, Google Gemini, AWS Bedrock, and Ollama. Azure OpenAI is listed in the provider catalogue but isn’t available on this appliance.

First-class OpenAI-compatible presets include Groq, xAI (Grok), OpenRouter, Fireworks AI, Cerebras, DeepSeek, Moonshot AI (Kimi), SambaNova Cloud, Hyperbolic, NVIDIA NIM, and Nebius AI Studio. A Custom provider accepts any OpenAI-compatible endpoint URL.

Model

A model is a specific identifier within a provider’s catalogue, such as gpt-5.5 on OpenAI, claude-sonnet-4-6 on Anthropic, or gemini-3.5-flash on Google Gemini. The catalogue supplies each model’s input-token and output-token pricing, capabilities such as JSON mode, audio input, and realtime voice, and any request-profile transforms the provider needs. A model with no catalogue price, such as a model of an OpenAI-compatible preset or Microsoft 365 Copilot, is metered at $0 unless the surface declares its own price. Ollama models are listed in the catalogue at $0.

Catalogue

The catalogue is the hot-reloadable provider and model configuration. It supplies provider request profiles, model pricing, and capabilities. The Models page shows it on the Built In, Custom, and JSON tabs. When an administrator saves a custom provider, its models become available across every LLM Surface.

Provider presets

A provider preset is a first-class shortcut for an OpenAI-compatible provider, such as Groq or Fireworks AI. It supplies the base URL and the label for its API key field, so a surface only needs the key. Preset models carry no catalogue price. Presets turn an OpenAI-compatible URL into a ready-to-select provider choice.

An LLM Surface's provider configuration with Groq selected as a first-class preset, showing its endpoint, model, and API key secret

Custom provider

A custom provider is an OpenAI-compatible endpoint not covered by a preset. An administrator defines it once as JSON in the Custom Model (JSON) card on the Models page’s JSON tab: a unique identifier and display name, plus optionally its base URL, the label for its API key field, and its models with their prices. Once saved, it is listed on the Custom tab and appears in every provider picker across the appliance exactly like a built-in provider, and any surface can select it. A surface that selects the provider takes its endpoint from the base URL, so set one for calls to reach the provider’s endpoint. A custom provider authenticates through the same secret-reference pattern as every other provider. When an endpoint sits behind a proxy or ingress that needs its own key, a surface can send extra upstream headers, each value read from a secret.

The Custom Model (JSON) editor on the Models page JSON tab, with a draft JSON definition for a new provider's identifier, name, base URL, and API key label

How Microsoft 365 Copilot grounds answers without exposing raw documents

The Microsoft 365 Copilot provider is a specialised adapter. Agent Stream translates an incoming OpenAI chat call into a Microsoft 365 Copilot Chat API call made as the signed-in user, forwarding a delegated Entra token through identity pass-through.

Copilot grounds its response on the user’s OneDrive, SharePoint, mail, and calendar content while respecting their Entra access control lists. It returns only the synthesised answer: the agent, editor, and Agent Stream do not receive the raw documents.