Safe runner in python
In some process such as data extraction, a lot of unpredictable failure can happen. Often we still want to run the whole process but log all the errors so we can fix them later
This repo provide a simple yet flexible runner to launch these kind of process without having to think about exceptions.
Install
pip
pip3 install git+https://gitlab.tetras-libre.fr/librairies-du-tetras/pySafeRunner@master#egg=pySafeRunnerFor developpement
git clone https://gitlab.tetras-libre.fr/librairies-du-tetras/pySafeRunner
pip3 install -e .As a dependency in your setup.py
from setuptools.command.install import install
from setuptools.command.develop import develop
from setuptools import setup, find_packages
import pip
...
gitDeps = [
'git+https://gitlab.tetras-libre.fr/librairies-du-tetras/pySafeRunner@master#egg=pySafeRunner',
]
def customRun(command_subclass):
orig_run = command_subclass.run
def modified_run(self):
for dep in gitDeps:
pip.main(['install', dep])
orig_run(self)
command_subclass.run = modified_run
return command_subclass
@customRun
class actionsOnInstall(install):
pass
@customRun
class actionsOnDevelop(develop):
pass
...
setup(
...
cmdclass={'install': actionsOnInstall,
'develop': actionsOnDevelop,
...
)
Usage
Single argument
Suppose you want to run a doStuf function that takes exactly one argument :
from pySafeRunner import safeRunner
runner = safeRunner.safeRunner()
for i in range(0,5):
runner.run(doStuff, i)Multiple arguments
Using anonymous args
Suppose you want to run a doStuf function that takes exactly two arguments :
for i in range(0,5):
for i in range(5,10):
runner.run(doStuff, (i,j))**kwargs
Suppose you want to run a doStuf function that takes 3 named arguments :
for i in range(0,5):
for i in range(5,10):
runner.run(doStuff, {'i':i,'k':5,'j':j})Using the map function
You can also use the map function :
# Single argument
for res in runner.map(compute, range(0,5)):
print(res)or
# Two arguments (i.e two loops)
for res in runner.map(compute, range(0,5),range(5,10)):
print(res)