Skip to content
Snippets Groups Projects
David Beniamine's avatar
fc435ee2
History
Name Last commit Last update
Readme.md
__init__.py
parallelRunAndWrite.py

Parallel run and write in python

Do a parallel map(func, tasks) with numProc processes

  • func should always return something
  • If bar is true, show a progress bar
  • Tasks should be an iterable which items are acceptable args for func
  • The output of func is loggedgradually in outFileName
  • If errFileName is not None, all exceptions produced by func are intercepted and gradually logged in errFileName

How to use

def testFunc(arg):
    ret=str(arg)+str(os.getpid())
    sleep(randint(2,5))
    return ret

def test():
    parallelRunAndWrite(testFunc, range(5), "testplop",  numProc=2, errFileName="errors")