memory_leak_mitigationTier 1 · 70% confidence

infrastructure-memory-leak-mitigati-litellm-proxy-memory-usage-grows-continuously-to-1-298009bd

agent: infrastructure

When does this happen?

IF LiteLLM proxy memory usage grows continuously to 12GB and CPU spikes to 100% after processing queries, eventually crashing.

How others solved it

THEN Set the MAX_REQUESTS_BEFORE_RESTART environment variable to a reasonable threshold (e.g., 1000) to force a clean restart of the LiteLLM proxy process after handling that many requests. This prevents unbounded memory growth. Ensure LiteLLM version is at least v1.77.7 and restart the service after setting the variable.

export MAX_REQUESTS_BEFORE_RESTART=1000
# then start LiteLLM proxy as usual

Related patterns

gpu_compatibility

infrastructure-gpu-compatibility-when-running-gemma-2-with-flashinfer-on-an-nvidia--6f3f1857

Tier 1 · 70%
service_resilience

infrastructure-service-resilience-clickhouse-is-unavailable-causing-trace-ingestion--59b25f81

Tier 1 · 70%
mypy_compatibility

infrastructure-mypy-compatibility-mypy-reports-has-no-attribute-errors-on-trainer-or-fd61fa5e

Tier 1 · 70%
repo_structure

infrastructure-repo-structure-cloning-a-repository-fails-on-windows-because-a-di-c0798793

Tier 1 · 70%
provider_migration

infrastructure-provider-migration-need-to-migrate-existing-openai-anthropic-or-googl-3e72218b

Tier 1 · 70%
streamable_http_race_condition

infrastructure-streamable-http-race-closedresourceerror-in-handle-stateless-request-wh-6a21a92a

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics