Why is asyncio faster than threading python ?

Mar 17, 2024 ยท 2 min read

Python's asyncio module provides a way to write non-blocking, asynchronous code that can execute faster than equivalent threaded code. Here's why asyncio shines and how it achieves better performance.

Blocking Calls Slow Things Down

The Python threads model suffers from something called the Global Interpreter Lock (GIL). This lock prevents multiple threads from running Python bytecodes at once, meaning only one CPU core can execute Python code at a time.

When a Python thread makes a blocking call, like requesting data from a web API, the entire Python process stalls until that call completes. Other threads cannot run during that blocking call no matter how many cores are available.

# Thread 1 makes a blocking call, stalling all other threads
response = requests.get(url) 

Asyncio Uses Cooperative Multitasking

The asyncio module provides an event loop that can juggle multiple tasks cooperatively. Each task explicitly gives control back to the event loop periodically so other tasks get a turn.

import asyncio

async def fetch_data():
  # Pause here cooperatively to allow other tasks to run
  await asyncio.sleep(0)  
  return requests.get(url)

loop = asyncio.get_event_loop()
loop.run_until_complete(fetch_data())

Since blocking calls like requests.get() are wrapped in await, they don't actually block the whole Python process. The event loop can interleave progress on multiple tasks during the wait.

Asyncio Maximizes CPU Utilization

While a threaded app is stuck at one Python thread during I/O, an asyncio app can continue executing other tasks. The event loop constantly switches tasks to maximize CPU usage.

Asyncio also minimizes expensive Python GIL contention. Threads in Python must acquire the GIL before running Python bytecodes, which is a bottleneck. But asyncio tasks like I/O waits don't need the GIL, freeing it for other work.

So asyncio allows efficient cooperative multitasking in Python. By minimizing blocking calls and GIL contention, it can achieve much higher throughput than threads.

Browse by tags:

Browse by language:

The easiest way to do Web Scraping

Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


Try ProxiesAPI for free

curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

<!doctype html>
<html>
<head>
    <title>Example Domain</title>
    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
...

X

Don't leave just yet!

Enter your email below to claim your free API key: