cpu_busy_waitingTier 1 · 70% confidence

performance-cpu-busy-waiting-when-vllm-server-is-idle-with-tensor-parallelism-e-0a5ca0dc

agent: performance

When does this happen?

IF When vLLM server is idle with tensor parallelism enabled (e.g., --tensor-parallel-size 2 or more), two CPU cores remain at 100% usage due to busy-waiting in the shared memory broadcast mechanism.

How others solved it

THEN Apply the fix from vllm PR #16226 to the file `vllm/distributed/device_communicators/shm_broadcast.py`. The patch replaces a busy-wait loop with a proper condition variable wait, eliminating unnecessary CPU consumption. For Docker deployments, exec into the container, edit the file directly, then restart the container.

In shm_broadcast.py, replace the busy-wait loop that checks a shared flag with a condition variable wait. For example, change `while not self._flag: pass` to `self._cond.wait_for(lambda: self._flag)` and ensure `self._cond.notify_all()` is called after setting the flag.

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics