infrastructure-connection-leak-unclosed-aiohttp-client-session-warnings-appear-af-ad0d9756

THEN Ensure that the aiohttp client session used internally by acompletion is properly closed after all requests complete. This can be done by using a context manager for the session or explicitly closing the HTTP client session. For batch processing, consider reusing a single session with proper cleanup, or update to a LiteLLM version that manages the session lifecycle correctly. Example: create a persistent session and close it after gathering results.

import asyncio
from litellm import acompletion

async def process_batch(prompts):
    session = None  # Track session if exposed
    sem = asyncio.Semaphore(20)
    async def single(prompt):
        async with sem:
            return await acompletion(model="vertex_ai/gemini-2.5-flash", messages=[{"role": "user", "content": prompt}], temperature=0.0)
    tasks = [single(p) for p in prompts]
    results = await asyncio.gather(*tasks)
    # Explicitly close any open aiohttp sessions (implementation varies; may require internal access or a client reference)
    return results