I'm comparing two common ways to control concurrency in asyncio. With a semaphore, I can allow up to a fixed number of operations at once: `sem = asyncio.Semaphore(10)` followed by `async with sem: await do_work()`. Alternatively, I could put jobs into an `asyncio.Queue` and run a fixed number of worker tasks that pull from it. How do experienced Python developers choose between these designs? I'm especially interested in production use cases, including differences in backpressure, cancellation, fairness, task lifetime, failure handling, retries, and overall code complexity. Are there situations where a semaphore is clearly the better abstraction?
5 Answers
Task lifetime and memory usage can be the deciding factors. A common semaphore pattern creates one task per input item and has most of them waiting for the semaphore. That’s fine for a small batch, but creating a million waiting tasks can exhaust memory even though only ten operations run at once. A queue with ten workers keeps the number of active tasks bounded regardless of input size.
A semaphore is mainly a synchronization tool: use it when the operations already exist and you simply need to cap how many may run at once. For example, if you have a batch of URLs or records, wrapping each operation in `async with sem:` is usually the simplest approach. There’s no worker startup or shutdown protocol to manage.
A queue is more appropriate for producer-consumer designs. It works well when jobs arrive over time, the input may be very large or unbounded, or you’re building a pipeline with several stages. A bounded queue also provides real backpressure: producers have to wait when the queue is full, which a semaphore doesn’t provide by itself.
That also makes queues useful when you need explicit handling for retries, worker failures, graceful shutdown, or tracking which jobs are still pending. Those features are possible with semaphores, but you generally end up building a queue-like architecture around them.
Structured-concurrency libraries such as AnyIO can offer another option with a task group and a capacity limiter. That lets you limit active work without manually creating a large collection of tasks waiting on a semaphore. It’s worth considering when task startup, cancellation, and failure propagation need to be managed together.
As a rule of thumb, I use a semaphore for a straightforward concurrency limit around independent calls, such as limiting simultaneous API requests. I use a queue when work arrives asynchronously, needs buffering, or has its own worker lifecycle. Both can limit concurrency, so the choice is mostly about architecture: semaphores are lighter for a simple cap, while queues make flow control and producer-worker coordination explicit.
Cancellation follows the same distinction. Cancelling a gather may involve cancelling a huge collection of tasks, including many that never started doing useful work. With workers, you typically cancel only the fixed number of workers and leave the remaining jobs in the queue for inspection, retrying, or cleanup.