Skip to content
Snippets Groups Projects
Verified Commit 6b7e351d authored by David Beniamine's avatar David Beniamine
Browse files

Initial commit

parents
No related branches found
No related tags found
No related merge requests found
Readme.md 0 → 100644
# Safe runner in python
In some process such as data extraction, a lot of unpredictable failure can happen. Often we still want to run the
whole process but log all the errors so we can fix them later
This repo provide a simple yet flexible runner to launch these kind of process without having to think about exceptions.
## Install
### pip
```bash
pip3 install git+https://gitlab.tetras-libre.fr/tetras-libre/pySafeRunner@master#egg=pySafeRunner
```
### For developpement
```bash
git clone https://gitlab.tetras-libre.fr/tetras-libre/pySafeRunner
pip3 install -e .
```
### As a dependency in your setup.py
```python
from setuptools.command.install import install
from setuptools.command.develop import develop
from setuptools import setup, find_packages
import pip
...
gitDeps = [
'git+https://gitlab.tetras-libre.fr/tetras-libre/pySafeRunner@master#egg=pySafeRunner',
]
def customRun(command_subclass):
orig_run = command_subclass.run
def modified_run(self):
for dep in gitDeps:
pip.main(['install', dep])
orig_run(self)
command_subclass.run = modified_run
return command_subclass
@customRun
class actionsOnInstall(install):
pass
@customRun
class actionsOnDevelop(develop):
pass
...
setup(
...
cmdclass={'install': actionsOnInstall,
'develop': actionsOnDevelop,
...
)
```
## Usage
### Single argument
Suppose you want to run a `doStuf` function that takes exactly one argument :
```python
runner = safeRunner()
for i in range(0,5):
runner.run(doStuff, i)
```
### Multiple arguments
#### Using anonymous args
Suppose you want to run a `doStuf` function that takes exactly two arguments :
```python
runner = safeRunner()
for i in range(0,5):
for i in range(5,10):
runner.run(doStuff, (i,j))
```
#### \*\*kwargs
Suppose you want to run a `doStuf` function that takes 3 named arguments :
```python
runner = safeRunner()
for i in range(0,5):
for i in range(5,10):
runner.run(doStuff, {'i':i,'k':5,'j':j})
```
### Using the map function
You can also use the map function :
```python
# Single argument
for res in runner.map(compute, range(0,5)):
print(res)
```
or
```python
# Two arguments (i.e two loops)
for res in runner.map(compute, range(0,5),range(5,10)):
print(res)
```
import logging
from itertools import repeat
class safeRunner():
def __init__(self, name=__name__):
self.logger = logging.getLogger(__name__)
# Run a function intercepting and logging errors
# Supports **kwargs, *args and single arg
def run(self, func, args):
try:
if isinstance(args, dict):
return func(**args)
elif isinstance(args, tuple) or isinstance(args, list):
return func(*args)
else:
return func(args)
except Exception as e:
self.logger.exception("Intercepted {}:".format(e))
# Calls run but in a map
def map(self, func, *args):
return map(self.run, repeat(func, len(args[0])), list(zip(*args)))
[aliases]
test=pytest
setup.py 0 → 100644
from setuptools import setup, find_packages
from os import path
here = path.abspath(path.dirname(__file__))
with open(path.join(here, 'Readme.md'), encoding='utf-8') as f:
long_description = f.read()
setup(
name="pySafeRunner",
url="https://gitlab.tetras-libre.fr/tetras-libre/pysaferunner",
version="1.0",
license="Gpl v3+",
author="Tetras Libre",
author_email="Contact@Tetras-Libre.fr",
description="A python wrapper to run stuff that are highly probable to fail without killing the whole process",
long_description=long_description,
packages=find_packages(),
install_requires=[""],
setup_requires=["pytest-runner"],
tests_require=["pytest"],
)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment