Thread Pool — Parallel Map

Overview

Implement a fixed-size pool of worker threads that runs a function over many inputs at the same time and returns the results in the original input order. The pool is created once, reused across all items, and shut down cleanly when the work is done.

The core problem is feeding work to a fixed set of long-lived threads instead of spawning one thread per item. A shared work queue hands tasks to whichever worker is free, which keeps every thread busy and bounds the number of threads regardless of how many items there are. The two things to get right are preserving output order even though tasks finish out of order, and shutting the workers down so the program does not hang on exit.

Interfaces

A ThreadPool that workers pull from, plus a convenience parallel_map:

import threading
import queue


class ThreadPool:
    def __init__(self, num_workers):
        # start num_workers threads, each looping on a shared queue
        ...

    def submit(self, fn, *args):
        # enqueue one task; return an index/handle to fetch its result later
        ...

    def shutdown(self):
        # signal workers to stop (poison pill) and join them
        ...


def parallel_map(fn, items, num_workers):
    """
    Run fn(item) for every item across num_workers threads and return the
    results in the same order as items.
    """
    ...

Each worker loops: pull a task off the queue, run it, store the result at the task's index, repeat. A None (or sentinel) task is the poison pill that tells a worker to exit. Writing results into a pre-sized list by index keeps the output ordered without extra locking on the results.

Inputs and outputs

• Input: a function fn, a list of items, and the worker count (1–256).
• Output: a list of fn(item) results in the same order as items, computed concurrently. Every item is processed exactly once.

Requirements

• Exactly num_workers threads do all the work, no matter how many items.
• Results are returned in input order, even though tasks finish out of order.
• Use a thread-safe queue to distribute tasks; no manual locking on the queue.
• Shut workers down with a sentinel and join them — no leaked threads, no hang.
• If fn raises on some item, surface the error rather than silently dropping it.

Examples

parallel_map(lambda x: x * x, [1, 2, 3, 4], num_workers=2)
# -> [1, 4, 9, 16]   (order preserved)

parallel_map(str.upper, [], num_workers=4)
# -> []              (no items, no work)

# slow tasks still come back in order
parallel_map(slow_square, [5, 1, 3], num_workers=3)
# -> [25, 1, 9]

Follow-ups

• A caller wants just one result back, not the whole batch. How would you return a future from submit that they can await?

• What happens when work is submitted faster than the workers can drain it — and how would a bounded task queue change that?

• When would you pick this over concurrent.futures.ThreadPoolExecutor, and when would you not?

• And more...