Multiple arrays in Python ThreadPool map

I’m writing a python program that gives the workers their work. I’m facing a problem with multiprocess.pool.ThreadPool that it only allows 1 iterable. I want the two workers to have the same (or almost) amount of work. Code as follow:

from multiprocessing.pool import ThreadPool

def f(worker,work):
    print(f"{worker} | {work}")

workers = ["worker1","worker2"]
work = [1,2,3,4,5,6,7,8,9,10]

if __name__ == "__main__":
    with ThreadPool(len(workers)) as p:
        p.map(f,zip(workers,work))

Expected output (work order is not necessary):

worker1 | 1
worker2 | 2
worker1 | 4
worker2 | 3
...

Any help would be appreciated. Thanks.

Answer

I think you want ThreadPool.starmap:

from multiprocessing.pool import ThreadPool
from itertools import cycle

def f(worker,work):
    print(f"{worker} | {work}")

workers = ["worker1","worker2"]
work = [1,2,3,4,5,6,7,8,9,10]

if __name__ == "__main__":
    with ThreadPool(len(workers)) as p:
        p.starmap(f,zip(cycle(workers),work))

Output:

worker1 | 1
worker2 | 2
worker1 | 3
worker1 | 5
worker2 | 4
worker2 | 6
worker1 | 7
worker1 | 9
worker2 | 10
worker2 | 8

Note that you also need to use cycle() when zipping the workers, otherwise only the first two elements from the work list will be used.