Skip to content

Model IDs with reserved provider prefixes (e.g. bedrock/...) are silently reinterpreted as canonical refs on non-gateway provider types #27361

Description

@ericpaulsen

Summary

When an Agents model is configured under a non-gateway provider type (e.g. Anthropic) with a model ID whose first /- or :-segment matches a built-in provider name (anthropic, azure, bedrock, google, openai, openai-compat, openrouter, vercel), chatprovider.ResolveModelWithProviderHint parses it as a canonical provider/model reference. The prefix is stripped and the model is reassigned to the embedded provider type — overriding the provider the admin explicitly assigned the model to. There is no config-time validation or UI warning; the failure surfaces later as an opaque upstream error.

Reproduction

  1. Configure an Anthropic-type provider with a custom base URL pointing at an LLM gateway (e.g. LiteLLM) that namespaces its model catalog with bedrock/... IDs (a common LiteLLM convention).
  2. Add an Agents model with model ID bedrock/claude-opus-4-8 under that provider.
  3. Start a chat with that model.

Expected: the model ID is sent verbatim to the configured provider's endpoint, since the admin explicitly bound this model to a specific provider.

Actual: parseCanonicalModelRef strips bedrock/ and resolves provider=bedrock. The request goes out with model claude-opus-4-8, which the gateway rejects:

bad request: POST "http://coder-aibridge/v1/messages": 400 Bad Request {"error":{"message":"POST \"https://<gateway>/api/v1/messages\": 400 Bad Request {\"detail\":\"Model not found\"}","type":"api_error"},"request_id":"","type":"error"}

Chat debug logging confirms the stripped model name (a non-colliding ID like CDR/GLM-latest on the same provider passes through intact).

Escaping is impossible: bedrock/bedrock/claude-opus-4-8 doesn't help — the first segment still wins, and the model is now routed to the Bedrock provider type entirely (Bedrock invoke protocol, Bedrock credential resolution), not the configured Anthropic entry.

Why the existing workarounds are unsatisfying

  • Gateway provider types (OpenAI Compatible, OpenRouter, Vercel) pass slash-namespaced IDs through verbatim, but switching a Claude model from the Anthropic type to OpenAI Compatible loses Anthropic-specific capabilities (thinking budget, thinking display/sendReasoning, disableParallelToolUse, the context-1m beta header).
  • Renaming/aliasing the model on the gateway works but isn't always possible when the naming convention is established across an organization.

Proposed fix (either)

  1. Skip canonical-ref parsing when the model config is explicitly assigned to a provider — the assignment already answers the question the parsing tries to infer; or
  2. Add a per-model "verbatim model ID" toggle for non-gateway provider types.

Additionally (or minimally): validate at model-save time and warn when a model ID on a non-gateway provider type parses as a canonical ref naming a different provider.

Relevant code

  • coderd/x/chatd/chatprovider/chatprovider.go (ResolveModelWithProviderHint, parseCanonicalModelRef, isGatewayProvider)
  • coderd/x/chatd/model_routing_aibridge.go

This issue was drafted with Coder Agents assistance while debugging a customer deployment.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions