Surfaces

ReferenceWhat it covers
Core configurationThe fields every surface shares, no matter what else you add to it: name and enabled state, the listener and route it answers on, which client API formats it accepts, and identity pass-through.
ProvidersConnecting a surface to an LLM provider: the provider, endpoint, model, and API key fields, model parameters and the override policy, upstream headers, and how a call’s cost gets priced.
GuardrailsScreening requests and responses for sensitive content: Prompt Guard’s regex and PII rules, Expert Witness safety classifiers, and the Judge/Jury reasoning-based checks.
Routing and variantsSending different requests to different upstreams: content-based routing with the Decider, routing rules on the Router element, canary traffic splits, and the surface’s variant catalogue.
Resilience and cachingKeeping a surface up and fast: failover to backup providers, load balancing across a pool, streaming passthrough, and caching repeated responses.
Cost and usage limitsControlling spend: request-rate limiting, cost tracking, hard per-stage usage limits, and alerts when spend or error rates cross a threshold.
Server tools and modalitiesTurning on optional capabilities beyond plain chat: governed web search and web fetch, context-window compression, and the Inputs elements for documents, audio files, images, and realtime voice.
IDE SurfacesBuilding a governed model menu for IDE AI clients: which surfaces appear in the catalogue, central and per-member sign-in, and per-model access policy.