Properly Closing aiohttp Clients and Sessions

Mar 3, 2024 ยท 2 min read

When working with the popular Python asynchronous HTTP client/server framework aiohttp, it's important to properly close your ClientSession and connections to avoid issues like resource leaks or failing to send pending data.

Use async with for ClientSessions

The recommended way to create an aiohttp.ClientSession is using an async with block:

async with aiohttp.ClientSession() as session:
    async with session.get(url) as response:
        # use session

This automatically handles closing the session for you even if exceptions occur.

Without async with, you'd need to manually await session.close():

session = aiohttp.ClientSession()
try:
    # make requests
finally:
    await session.close() 

Forgetting to close sessions can lead to TCP connection leaks over time.

Close ClientResponses

Similarly, use async with when making requests to automatically close the ClientResponse:

async with session.get(url) as response:
   # response handling

This sends any pending data and frees the connection.

Lingering Tasks May Prevent Closing

One catch when closing aiohttp connections is that if you still have coroutine tasks executing that rely on the session/connection, the close will wait for those to finish.

So make sure you await any tasks before closing:

await fetch_data(session) # wait for tasks using session
await session.close()

This prevents errors from trying to access a closed connection in background tasks.

Key Takeaways

  • Use async with on ClientSession and ClientResponse instances
  • Manually await session.close() if not using async with
  • await any pending tasks first before closing
  • Following these tips will help prevent issues when wrapping up aiohttp clients!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: