Parallel run and write in python

Do a parallel imap_unordered with numProc processes

• func should always return something
• Tasks should be an iterable which items are acceptable args for func
• The output of func is loggedgradually in outFileName
• If bar is true, show a progress bar
• If errFileName is not None, all exceptions produced by func are intercepted and gradually logged in errFileName
• numProc sets the number of process to use
• if ordered is true, results are outputed in the same order as tasks
• chunksize : chunkszise for imap
• if debug is set to true, errors are also printed to stderr

How to use

from pyParallelRunAndWrite import parallelRunAndWrite as prw
# The following dependencies are only useful for testFunc
import os
from time import sleep
from random import randint

def testFunc(arg):
    ret=str(arg)+str(os.getpid())
    sleep(randint(2,5))
    if(randint(0,100)%2 ==0):
        raise Exception("COUCOU"+ret)
    return ret

prw.parallelRunAndWrite(testFunc, range(5), "testplop",  numProc=2, errFileName="errors")