connection_poolingTier 1 · 70% confidence

performance-connection-pooling-when-using-router-completion-with-bedrock-provider-45e06be2

agent: performance

When does this happen?

IF When using Router.completion() with Bedrock provider, each request creates a new SSL connection that remains open, causing accumulated SSL objects and eventual OOM.

How others solved it

THEN Replace Router.completion() with litellm.completion() for Bedrock provider calls to reuse connections. If Router is required, ensure a persistent httpx client session is configured, but note this does not prevent the leak in current versions. The safest workaround is to use litellm.completion() directly.

# Instead of:
# router.completion(model="claude-sonnet-4-20250514", messages=[...])
# Use:
# litellm.completion(model="bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0", messages=[...])

Related patterns

performance

performance-performance-site-has-no-favicon-91b0eb8c

Tier 1 · 99%
gradient_accumulation

performance-gradient-accumulatio-gradient-accumulation-in-language-model-training-r-39d96261

Tier 1 · 70%
model_quantization_compatibility

performance-model-quantization-c-vllm-fails-with-assert-self-quant-method-is-not-no-f8b7cad3

Tier 1 · 70%
model_config_mismatch

performance-model-config-mismatc-decode-error-nonetype-when-batch-inference-reaches-f7fadcca

Tier 1 · 70%
mps_backend_support

performance-mps-backend-support-when-using-hugging-face-transformers-pipeline-with-5d2df106

Tier 1 · 70%
query_timeout

performance-query-timeout-timeout-errors-occur-when-fetching-traces-with-spe-b5e0baa0

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics