Server tools and modalities

Each capability on this page extends the standard chat path, and each stays off until you add it. Web Search and Web Fetch are elements in the Tools palette group, and Audio File, Audio Stream, Image File, and Document File are elements in the Inputs palette group. Drag an element onto the canvas and it attaches to the LLM node, switched on. Turn off its switch, or delete the element, to turn the capability off. Context management is set in the surface’s Config JSON view instead (see Context management). See Inputs and modalities → for how the Inputs elements decide what a surface accepts.

Server tools

The Web Search and Web Fetch elements let the gateway itself execute web search and web fetch tool calls inside the request, so the model can pull in live information without your application having to implement or govern that tool call itself. The model emits a tool call, the gateway runs it under OPA, per-surface domain policy, address checks (a fetch can’t reach localhost or a cloud provider’s instance-metadata address), metering, and governance, injects the result, and re-invokes the model. A model-initiated fetch is no less controlled than any other request.

• Add Web Search when the model needs to look things up, and Web Fetch when it needs to retrieve a specific known URL.
• Pick a domain policy based on which list is shorter for your use case: Allow all URLs (a block-list) when you mostly trust the open web and just want to exclude a few domains; Block all URLs (an allow-list) when the model should only ever reach a small, known set of domains.
• Choose a search engine based on what you already have an account with; Tavily is the default if you don’t have a preference.

Web Search fields:

Web Fetch fields:

Domain policy

Configure domains… on either tool’s panel opens a full-page editor with a Baseline card and an exceptions list.

Context management

Context management guards against an over-long conversation failing once it exceeds the model’s context window: instead of an error, older messages are trimmed to make room before the upstream call, while the most recent turns and every system message are preserved. Set it in the surface’s Config JSON view, in the context_management object:

"context_management": {
  "enabled": true,
  "context_window_tokens": 128000,
  "keep_recent_messages": 4,
  "reserve_output_tokens": 1024
}

• Set context_window_tokens to your upstream model’s real window size. Compression needs this value, since there’s no budget to compress against without it.
• Set enabled to true if you want compression to happen automatically for any request that would exceed the window. With it false, callers opt in per request with transforms: ["middle-out"].
• Raise keep_recent_messages if your use case depends on more than the last few turns staying verbatim. The rest of the budget is still filled from the middle of the conversation outward.

Document ingestion

The Document File element checks documents a caller attaches to a chat message and, depending on its mode, extracts their text and injects it as ordinary prompt content. Extracted text runs through the same PII masking, guardrails, and context-window guard as any other prompt text, so a model without document support can still read the document. A document embedded in the request is checked against the allowed types and file size, and a request that fails either check is rejected with HTTP 400. A document given by reference, such as a provider file ID, is forwarded as it is. Without the element, documents are passed to the provider as sent, without checks or extraction.

• Auto, the default, extracts Text, Markdown, CSV, JSON, and DOCX files and forwards any other allowed type, such as PDF, to the model as a file.
• Extract extracts the same types and rejects the request for any allowed document it can’t extract, such as a PDF.
• Use Passthrough when you’re forwarding to a model that reads documents natively; the allowed types and size limit still apply.

The detected document types are passed to policy as input.resource.document_types.

Audio input

The Audio File element accepts audio files attached to a chat message for a model that accepts audio input. Audio input is strictly opt-in: without an enabled Audio File element, a request carrying audio is rejected with HTTP 400. With the element in place, audio is still rejected when the surface’s model doesn’t accept audio input, or when a file’s format or the total audio size falls outside the settings below. For live, two-way voice, use the Audio Stream element instead (see Realtime voice).

The detected formats are passed to policy as input.resource.audio_formats. Governance records keep a digest of the request and the text prompt, not the audio itself.

Image input

The Image File element accepts images attached to a chat message for a vision-capable model. Image input is strictly opt-in: without an enabled Image File element, a request carrying an image is rejected with HTTP 400.

Format and size checks apply to images embedded in the request. An image URL is forwarded as it is, because the appliance can’t inspect a remote image without fetching it. The detected formats are passed to policy as input.resource.image_formats.

Realtime voice

The Audio Stream element turns on realtime voice: a caller opens a WebSocket connection to the surface’s /v1/realtime path, and the appliance relays live audio in both directions between the caller and the model. Without an enabled Audio Stream element, a realtime session is refused. The surface’s model must be realtime-capable, an OpenAI realtime model (for example gpt-realtime) or a Gemini Live model; with any other model, the session is refused with realtime_not_supported. A surface or variant whose model is a realtime model accepts only realtime sessions and rejects chat requests, which is why voice usually belongs on its own variant (see Routing and variants → Routing rules).

Before a session opens, the caller is authenticated with the surface’s source authentication and checked against the appliance-wide policies and the surface’s own policy, as a chat request would be. While governance recording is on, each turn, one user input and the model’s reply, is recorded as its own governance record.

Select the element to see its Enable realtime voice switch, then use Configure session, safety & audio streams… to open the full-page editor.

• With Review realtime model replies on, voice safety reviews each model reply with the surface’s Jury. The Judge doesn’t run on realtime voice, because caller audio reaches the model before any transcript exists, and the dashboard won’t save a canvas with both an enabled Judge and an enabled Audio Stream element.
• Gated guarantees no unsafe reply audio reaches the caller, at the cost of a short delay on each turn. Barge-in is low-latency, but a fragment can play before a cut-off.
• Set Max session length (seconds) based on how long a legitimate session should realistically run; it’s a hard ceiling so a stuck connection can’t hold resources indefinitely.
• Configure Audio streams if you want callers to hear something during a Gated pause or on a blocked or failed turn, rather than silence.

Enforcement options:

A blocked turn, or a Jury that fails to return a verdict, ends that reply with a refusal. While governance recording is on, a blocked turn is recorded as a Block record.

Audio streams

The Audio streams card picks pre-recorded clips the appliance plays to the caller. Clips are chosen by id from the appliance’s audio-stream catalogue, and the card header shows how many clips the catalogue holds.

Related

• Inputs and modalities: How client API formats, the Inputs elements, and realtime voice sessions work.
• Core configuration: Where these blocks sit on a surface, and the reasoning budget governor, the sibling limit for reasoning-token spend.
• Routing and variants: The Router element’s attachment-type rules, which send each kind of input to a variant whose model accepts it.
• Guardrails: The Jury configuration that reviews realtime voice replies.