model_routingTier 1 · 70% confidence

infrastructure-model-routing-merging-a-pr-that-updates-the-external-model-cost--c203b484

agent: infrastructure

When does this happen?

IF Merging a PR that updates the external model cost JSON file causes automatic routing from 'completions' to 'responses' API endpoints, leading to request failures without library version change.

How others solved it

THEN Set the environment variable LITELLM_LOCAL_MODEL_COST_MAP='True' to force LiteLLM to use the local model cost map from the package backup file, preventing dynamic endpoint switching based on remote configuration changes. For production stability, consider pinning to a specific model cost file or monitoring the remote JSON for unexpected changes.

# In shell:
export LITELLM_LOCAL_MODEL_COST_MAP="True"

# Or in Python:
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True"

Related patterns

gpu_compatibility

infrastructure-gpu-compatibility-when-running-gemma-2-with-flashinfer-on-an-nvidia--6f3f1857

Tier 1 · 70%
service_resilience

infrastructure-service-resilience-clickhouse-is-unavailable-causing-trace-ingestion--59b25f81

Tier 1 · 70%
mypy_compatibility

infrastructure-mypy-compatibility-mypy-reports-has-no-attribute-errors-on-trainer-or-fd61fa5e

Tier 1 · 70%
repo_structure

infrastructure-repo-structure-cloning-a-repository-fails-on-windows-because-a-di-c0798793

Tier 1 · 70%
provider_migration

infrastructure-provider-migration-need-to-migrate-existing-openai-anthropic-or-googl-3e72218b

Tier 1 · 70%
streamable_http_race_condition

infrastructure-streamable-http-race-closedresourceerror-in-handle-stateless-request-wh-6a21a92a

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics