Smart Routing

Smart routing lets the server pick the best harness and model for a session automatically, based on the first message. Instead of committing to one model up front, the server asks a router, "given this task, which harness and model fit?", and applies the answer for the rest of the session.

The two routers

Omnigent ships two routers. They are not alternatives you choose between — a deployment can have either, both, or neither, and the server picks between them per decision.

• The built-in judge — an LLM call using the server's own llm: block. It names models from the candidate menu it is handed, so it can serve any harness, on any credential. No extra infrastructure, and no dependency on any vendor: this is the pure open-source router.
• The Databricks AI Gateway task_v1 router — a hosted routing service the server calls over the routing API with routing.provider: external. Its picks are AI Gateway catalog ids, so it can only serve a harness whose inference actually runs through that gateway.

Which one answers

One selector runs per decision:

The task_v1 router answers when it is configured and every model family the decision involves is AI-Gateway-backed on the target host. Otherwise the built-in judge answers. A server with neither configured has no routing.

Gateway backing selects the source, not whether the feature exists. A pane that is off the gateway still routes — the judge serves it — and the decision records which router answered.

A host that reports nothing about gateway backing counts as backed — "unknown" is not "off the gateway", so a deployment whose hosts cannot yet answer is not silently downgraded.

Configure

Smart routing is enabled by configuration alone — there is no env var and no feature flag. The examples below are complete ~/.omnigent/config.yaml files.

Point the server at the file explicitly:

omni server -c ~/.omnigent/config.yaml

The -c matters. providers: is picked up from ~/.omnigent/config.yaml whether or not you pass it, but llm: and routing: are read only from the file given to -c. An llm: block in a config the server was never pointed at is the most common reason a judge silently fails to appear.

Out of the box: routing off

Providers only, with no llm: and no routing: block. Nothing routes, and the Smart Routing options stay hidden.

providers:
  anthropic:
    kind: key
    api_key: ${ANTHROPIC_API_KEY}

The built-in judge

Add an llm: block to any provider setup. model is the only required field.

providers:
  anthropic:
    kind: key
    api_key: ${ANTHROPIC_API_KEY}

llm:
  model: openai/gpt-5.4
  connection:
    base_url: https://api.openai.com/v1
    api_key: ${OPENAI_API_KEY}

The model id takes either form: provider-prefixed (openai/gpt-5.4), or a bare databricks-… catalog id, which the server routes through its Databricks adapter. Credentials come from either a connection: mapping or a Databricks CLI profile — connection: wins when both are present:

llm:
  model: databricks-claude-haiku-4-5
  profile: my-workspace

This is the pure open-source path: no gateway, no external service, and the judge serves every harness on the host.

The Databricks AI Gateway task_v1 router

A kind: databricks provider makes both model families gateway-backed; the routing: block names the router to call.

providers:
  my-workspace:
    default: true
    kind: databricks
    profile: my-workspace

routing:
  provider: external
  base_url: https://<workspace-host>/ai-gateway/routing/v1
  router_name: task_v1
  profile: my-workspace
  model_prefix:
    - databricks-
    - system.ai.

Use profile: rather than a static api_key: where you can — the server mints a fresh bearer per call, so a long-lived server never sends a token expired by the ~1 hour OAuth window.

Two things worth knowing about this block:

• It is optional on a Databricks provider. Leave the routing: key out entirely and the server derives the same client from your default Databricks provider, using that workspace's /ai-gateway/routing/v1. Declaring it makes the target explicit. Note the derivation only fires when the key is absent — a routing: block naming any provider other than external or none leaves you with the judge alone.

• Deleting it does not turn routing off. Because of that derivation, the only true off-switch is:

  routing:
    provider: none
  

  This one is absolute: it disables the task_v1 router and the built-in judge, even if an llm: block is present.

Both

Configure llm: and routing: together and you get the full posture: the task_v1 router on gateway-backed decisions, the built-in judge everywhere else. This is what keeps Smart Routing available on a host where only one family is wired to the gateway — the unbacked family degrades to the judge instead of losing the feature.

model_prefix strips a prefix from catalog model ids before they are sent to the router and restores it on the answer, so a catalog full of databricks-claude-… ids can talk to a router that knows them by their bare names.

Verify your setup

Ask the server which routers it has:

curl -s localhost:50151/v1/info | jq '{smart_routing_enabled, smart_routing_sources}'

smart_routing_sources reports the two sides independently — external is the AI Gateway task_v1 router, oss is the built-in judge.

Then ask whether the host can actually run the gateway router's picks:

curl -s localhost:50151/v1/hosts | jq '.hosts[0] | {status, gateway_inference}'

gateway_inference is a per-family map, reported by each host at registration. A family reported false cannot run task_v1's picks, so its decisions fall to the judge. A family missing from the map could not be evaluated, and counts as backed — the map distinguishes "not backed" from "could not tell". Two families are evaluated, Claude and Codex; anything else is always unknown.

What makes a family gateway-backed:

Subscriptions and Bedrock are never gateway-backed. That is the most common reason a decision comes back from the judge on an otherwise Databricks-backed server.

What you see

Decision chips. Every routing decision renders a chip under the message showing the harness and model it picked and the router's own rationale — the rule trace behind the choice ("trivial task → cheapest arm"). Chips appear for the session's own decision and for each routed subagent spawn. A session's gear carries one in-session routing control, Subagent routing, which decides whether spawns route or run on the harness default; the session's own routing is decided once, when it is created.

Which router answered. A chip carries a small Databricks mark when the task_v1 router answered. No mark means the built-in judge answered. Pickers are never branded — only decisions. The exact answer is in the raw verdict JSON on the routing card, under router_source: "databricks-aigw" or "oss-llm".

Partial credentials degrade, they don't hide. If only one family is gateway-backed, every surface stays. The unbacked family's decisions — and any decision spanning both families — come from the judge, and their chips carry no mark. The CLI says so once per launch, in one line: "<harness> is not AI-Gateway-backed on this host — routing with the built-in router instead." Surfaces disappear only when the server has no judge either, and then the error is specific: "not AI-Gateway-backed" is a credential problem on your machine, "the server has no routing model configured" is a server problem.

From the CLI. --smart-routing routes a launch:

omni run --smart-routing -p "review the last commit"
omni run --harness claude-native --smart-routing -p "fix the flaky test"

Routing happens when the session is created, so --smart-routing starts a new session and cannot be combined with --resume or --continue.

First-message routing in a bare TUI. Native harnesses that hook their own first typed message can be launched with no prompt at all:

omni claude --smart-routing
omni codex --smart-routing

The TUI opens unrouted. Type your first message and the harness routes it in place: the prompt is held, a notice reads "Smart Routing selected <model>; rerunning your message on it.", the model switch is applied while nothing is running, and your message is replayed on it. The switch is scoped to the session, so your global CLI defaults are never touched, and the replay does not duplicate the message in the session. Later messages do not re-route — the decision is per-session.

External routing API

routing.provider: external calls a single endpoint:

POST <base_url>/routes:select
Content-Type: application/json

Request and response bodies follow the omnigent.api.routing.v1 schema (proto3, serialized as JSON with snake_case field names). The schema is versioned independently of any gateway, so the contract can evolve without coupling to a gateway's release cycle — task_v1 is one router served over it, not the protocol itself.

Configuration fields

If base_url or router_name is missing, the external side is skipped with a warning rather than failing the server — the built-in judge, if configured, carries on alone.

Request: SelectRouteRequest

RouteOption — model (string, e.g. gpt-5-5) and harness (string; may be omitted for a native harness, required for a meta-harness).

RouteSelector — router_name (string, from the config field) and an optional config (Struct) interpreted by the selected router.

{
  "route_options": [
    { "model": "claude-opus-4-8", "harness": "claude-sdk" },
    { "model": "gpt-5-5", "harness": "codex" },
    { "model": "gpt-5-4-mini", "harness": "pi" }
  ],
  "task": { "prompt": "Refactor the auth module and add tests" },
  "route_selector": { "router_name": "task_v1" }
}

Response: SelectRouteResponse

RouteSelection — route_option (the chosen model + harness) and an optional params (Struct) emitted alongside the decision.

{
  "route_selection": [
    {
      "route_option": { "model": "claude-opus-4-8", "harness": "claude-sdk" }
    }
  ],
  "rationale": "Multi-file refactor with tests: favor the most capable model."
}

The server maps the chosen model (and harness, when present) back to the matching catalog entry, restoring any stripped model_prefix. A model the server did not offer in route_options is rejected, and the session falls back to its default harness. On any error (HTTP 4xx/5xx, unparseable body, empty selection) the server surfaces the reason and degrades gracefully rather than blocking the session.

Related

• Models & Credentials — how each provider and credential type is wired, including Databricks workspace profiles.
• Harnesses — the runtimes a router can pick between.