AI Gateway

Admins only. The models Garrison and your own apps use, served by the appliance’s Open Agent Fabric gateway. Changes here are planned by OAF and applied only after approval: each shows as a diff of the cluster configuration under Approvals.

Models

Each served model with its catalog model, kind (chat or embedding), the box it runs on, its share of that box’s memory, and its status: queued, downloading, starting, loading, ready, failed or removing. Status updates live while a model changes.

Action
Add a model	Pick a catalog model, a served name and where it runs (a box, or a pair). The console shows whether it fits. Its files start downloading at once; it starts after approval
Replace	Swap a model for another in the same place. The old model’s memory counts as free, so a large model can replace a smaller one
Remove	Stop serving it

A model cannot be selected for routing or chat until it is ready.

External providers

Models outside the appliance (another OAF cluster, or a hosted provider you choose). Requests to them pass through the gateway, its guardrails and its audit like local ones.

Guardrails

Policies that check requests and responses. The recommended basic policy, for external models, rejects requests that contain credentials or private keys (AWS keys, GitHub and Slack tokens, passwords, private key blocks) and masks emails, phone numbers and card numbers in responses.

Routing

How model: auto picks a model. Rules are checked top to bottom; the first match wins; anything else goes to the fast model. Fallback is used when the target is unhealthy. Clients that name a model explicitly bypass the rules.

• Observe mode: every auto request still goes to the fast model, and each rule’s decision is only recorded in AI Audit. Check the decisions there first.
• Active mode: rules route requests.

Add “Complex → large model” adds a rule that sends analytical or long requests to the large model on a pair.

Keys

API keys for apps that call the gateway. Each app has its own key, optionally limited to some models. Issue a key, Rotate (the old key stops working) or Remove. Garrison’s own key is vclocal; rotate it under Settings → Appliance.