Skip to content
Snippets Groups Projects
Select Git revision
  • 7205826ab3f36c27007433600e2504f1b351997b
  • master default protected
  • dev
3 results

pyParallelRunAndWrite

Name Last commit Last update
Readme.md
__init__.py
parallelRunAndWrite.py

Parallel run and write in python

Do a parallel map(func, tasks) with numProc processes

  • func should always return something
  • If bar is true, show a progress bar
  • Tasks should be an iterable which items are acceptable args for func
  • The output of func is loggedgradually in outFileName
  • If errFileName is not None, all exceptions produced by func are intercepted and gradually logged in errFileName

How to use

from pyParallelRunAndWrite import parallelRunAndWrite as prw
# The following dependencies are only useful for testFunc
import os
from time import sleep
from random import randint

def testFunc(arg):
    ret=str(arg)+str(os.getpid())
    sleep(randint(2,5))
    return ret

prw.parallelRunAndWrite(testFunc, range(5), "testplop",  numProc=2, errFileName="errors")