I'm a bit confused about the relationship between concurrent.futures and the threading and multiprocessing modules in Python. If concurrent.futures is available, does that mean I can't or shouldn't use threading or multiprocessing? Are there any capabilities in threading or multiprocessing that aren't supported by concurrent.futures?
5 Answers
If you're using VSCode, you can actually right-click on concurrent.futures and peek at what it's doing under the hood. It should clarify how it uses threading and multiprocessing.
Hey! If you're looking for detailed comparisons, check out this resource by Jason Brownlee. He provides clear scenarios on when to use each method along with code examples: https://superfastpython.com.
You're right to ask! Concurrent.futures acts like a convenience layer on top of the other two. It's perfect for simple parallel tasks, but once you venture into more complex scenarios where different workers must synchronize or wait on each other, you may need to rely on the lower-level threading or multiprocessing for better control.
Concurrent.futures actually builds on top of threading and multiprocessing. It's just a higher-level abstraction that makes it easier to handle tasks that can be easily parallelized. If you're just applying the same operation to many items, it's a great choice! But if you need to coordinate different tasks with distinct requirements, you might find concurrent.futures a bit limiting and want to dive into threading or multiprocessing directly.
Definitely keep an eye out for the new developments in the standard library since the GIL is expected to change in the future. Just remember, the GIL isn't gone yet, and some features are still experimental. It'll take time for everything to catch up!
That makes sense, thanks for explaining!