Modification of a thread Pool algorithm with multiple task queues

In this paper, the author suggests a modification of the thread pool algorithm that was presented by Sean Parent at NDC London 2017. The suggested algorithm is as simple as the original implementation and demonstrates similar performance, while eliminating a potential drawback of the original implementation consisting in the fact that under certain circumstances, multiple tasks can be executed on the same thread, while other threads may be waiting for a task.

The suggested idea consists in tracking the total number of tasks that are in the queues of the thread pool. When the main thread pushes a new task to one of the queues, the tasks counter is incremented. When a task is removed from the queue, the task counter is decremented. When a thread wants to get a task, it keeps checking the queues until it succeeds in getting a task from one of the queues, or until the tasks counter becomes equal to zero. When the tasks counter becomes equal to zero, the thread becomes idle until the counter becomes non-zero again. Then, one of the threads wakes up and starts checking the queues. An important point is to maintain even distribution of tasks in the queues since it has a significant impact on the performance of the algorithm.