Parallel run and write in python
Do a parallel map(func, tasks) with numProc processes
- func should always return something
- If bar is true, show a progress bar
- Tasks should be an iterable which items are acceptable args for func
- The output of func is loggedgradually in outFileName
- If errFileName is not None, all exceptions produced by func are intercepted and gradually logged in errFileName
How to use
def testFunc(arg):
ret=str(arg)+str(os.getpid())
sleep(randint(2,5))
return ret
def test():
parallelRunAndWrite(testFunc, range(5), "testplop", numProc=2, errFileName="errors")