Futureproof - Bulletproof concurrent.futures#
concurrent.futures
is amazing, but it’s got some sharp edges that have bit me many times in the past.
Futureproof is a thin wrapper around it addressing some of these problems and adding some usability features.
Features:#
Monitoring: a summary of recent completed tasks is logged by default.
Fail fast: errors cause the main thread to raise an exception and stop by default.
Error policy: the user can decide whether to raise, log or completely ignore errors on tasks.
Backpressure control: large collections of tasks are consumed lazily as the executor completes tasks, drastically reducing memory consumption and improving responsiveness in these situations.
Current status: Alpha#
The API is subject to change, any changes will be documented in the changelog.
Futureproof was designed to wrap ThreadPoolExecutor, however version 0.2+ includes limited support ProcessPoolExecutor but only for Python3.7+.