Async/await seems useful for handling lots of waiting, but it can also spread through a codebase. When would threads or another concurrency model be a better fit, especially in Python?
Is async/await really the best way to handle concurrency?
Asked by
re-sort_the-pile
Answers
The spreading is a real tradeoff. Once a function needs to await something, its callers usually need to become async too, all the way up to a place that runs the event loop. That can make it awkward to mix async libraries with an otherwise synchronous application.
It does buy you explicit suspension points, though. In a single event loop thread, other tasks normally get to run when the current task yields control. That is a different model from threads, where execution can switch between threads at less obvious points.
Reply to this answer
For a modest number of blocking operations, a thread pool is worth considering. You can often keep using synchronous libraries and reuse a fixed number of workers instead of creating a thread for every request.
Don't assume either model wins on speed alone. Library implementation, workload, concurrency level, and resource limits all matter. Benchmark the actual workload if performance is the reason for choosing.
grain_of_salt_please:
Thread pools have limits too. If all the workers are blocked, extra jobs have to wait, and shared mutable state needs careful handling.
Reply to this answer
I'd separate async syntax from the library around it. A messy project can make threads or event loops equally painful to debug. AnyIO and Trio are worth looking at if you want structured concurrency, where task lifetimes and cancellation are organized around scopes rather than left floating around.
re-sort_the-pile:
So cancellation and cleanup can be as much a reason to pick a library as raw throughput?
_lower_case_MAYHEM:
Exactly. The syntax alone doesn't tell you how easy it is to stop a group of tasks or make sure they finish before their parent exits.
Reply to this answer
Watch out for CPU-bound work. Async tasks on a single event loop don't make a calculation run on multiple cores. A long calculation that doesn't yield can also hold up the other tasks.
For parallel CPU work in Python, processes are one option. Threads can help in some cases, such as code that releases the GIL, but the details depend on the Python runtime and the libraries involved.
Reply to this answer
There are alternatives that keep a more synchronous-looking programming style while using lightweight managed tasks. Goroutines and Java virtual threads are examples from other language ecosystems. They may avoid some of the explicit async boundary, but they still need a runtime to schedule work and a way to deal with shared data.
Generators can also be used to build cooperative scheduling systems, but swapping await for yield doesn't automatically remove the underlying need to mark and manage suspension.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Fizzlepop:
Explicit suspension points don't eliminate shared state problems. Another task can still change state across an await, so you have to think about what can happen there.