Does asyncio use multiple cores?

Mar 24, 2024 ยท 2 min read

Asyncio is a powerful feature in Python that allows you to write asynchronous, non-blocking code. This means your program can perform multiple tasks concurrently within a single thread.

A common question that arises is whether asyncio itself utilizes multiple CPU cores to speed up execution. The short answer is no, asyncio does not automatically use multiple cores. However, you can leverage multiple cores in an asyncio program by running the event loop on a thread pool.

Here is a simple asyncio example:

import asyncio

async def task(n):
    print(f'Processing {n}')
    await asyncio.sleep(1)
    return f'Result {n}'

async def main():
    tasks = [asyncio.create_task(task(i)) for i in range(3)]
    for t in tasks:
        result = await t
        print(result)

asyncio.run(main())

This will process the 3 tasks sequentially on a single thread.

To utilize multiple cores, we can use asyncio.run in ProcessPoolExecutor:

with ProcessPoolExecutor() as executor:
    asyncio.run(main(), executor=executor)

Now the tasks will run concurrently on separate process threads, allowing true parallel execution on multiple CPU cores.

The key takeaways are:

  • Asyncio enables concurrency, but not parallelism by default
  • You can achieve parallelism by integrating thread pools and process pools
  • Asyncio shines for I/O-bound workloads, like network calls and file operations
  • For CPU-bound tasks, multiprocessing may provide better utilization
  • In summary, while asyncio itself won't magically utilize all your cores, with some extra work you can build highly performant applications that leverage parallelism and get the most out of your modern multi-core hardware.

    The event loop model makes asyncio extremely fast and efficient, so integrate it with processors and threads where you need heavy number crunching or calculations that can paralyze Python performance.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: