process - Python What is the difference between a Pool of worker processes and just running multiple Processes? -
i not sure when use pool of workers vs multiple processes.
processes = [] m in range(1,5): p = process(target=some_function) p.start() processes.append(p) p in processes: p.join()
vs
if __name__ == '__main__': # start 4 worker processes pool(processes=4) pool: pool_outputs = pool.map(another_function, inputs)
as says on pymotw:
the pool class can used manage fixed number of workers simple cases work done can broken , distributed between workers independently.
the return values jobs collected , returned list.
the pool arguments include number of processes , function run when starting task process (invoked once per child).
please have @ examples given there better understand application, functionalities , parameters.
basically pool helper, easing management of processes (workers) in cases need consume common input data, process in parallel , produce joint output.
the pool quite few things otherwise should code (not hard, still, it's convenient find pre-cooked solution)
i.e.
- the splitting of input data
- the target process function simplified: can designed expect 1 input element only. pool going call providing each element subset allocated worker
- waiting workers finish job (i.e. joining processes)
- ...
- merging output of each worker produce final output
Comments
Post a Comment