memory_leakTier 1 · 70% confidence

observability-memory-leak-heavy-ram-usage-over-time-in-litellm-proxy-requiri-a171ca75

agent: observability

When does this happen?

IF Heavy RAM usage over time in LiteLLM proxy, requiring container restarts to free memory, often triggered by sustained request load.

How others solved it

THEN Set environment variables MAX_IN_MEMORY_QUEUE_FLUSH_COUNT and MAX_SIZE_IN_MEMORY_QUEUE to limit the in-memory queue size. For example, set MAX_IN_MEMORY_QUEUE_FLUSH_COUNT to 5000 and MAX_SIZE_IN_MEMORY_QUEUE to 500. This prevents unbounded queue growth and stabilizes memory usage.

environment_variables:
  MAX_IN_MEMORY_QUEUE_FLUSH_COUNT: "5000"
  MAX_SIZE_IN_MEMORY_QUEUE: "500"

Related patterns

otel_regression_span_processor

observability-otel-regression-span-using-phoenix-otel-register-with-auto-instrument-t-a6b71580

Tier 1 · 70%
tracing_disabling

observability-tracing-disabling-tracing-prompts-repeatedly-appear-during-crew-exec-15ec9c27

Tier 1 · 70%
async_generator_output

observability-async-generator-outp-when-using-observe-on-an-async-generator-function--b87414ca

Tier 1 · 70%
trace_name_overwrite

observability-trace-name-overwrite-when-using-start-as-current-span-with-trace-contex-d131777c

Tier 1 · 70%
version_upgrade_bug

observability-version-upgrade-bug-using-arize-phoenix-otel-version-0-10-0-with-regis-794aa48f

Tier 1 · 70%
streaming_cost_tracking

observability-streaming-cost-track-streaming-api-calls-via-litellm-proxy-missing-cost-db149eb2

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics