- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Need to rewind Generators that are passed as argument #66
Comments
Not sure whether the solution here should belong in the library. Seems to be good for retry to cater for side-effects. |
Seems to be affecting simple iterators too. |
Its unreasonable to even try with the infrastructure available in python |
The only solution that I see is to iterate before ever trying the first call. That sounds like something dangerous. Best thing is probably to document that the caller is responsible for not passing generators. |
Cloning an iterator seems possible in python. import itertools
# Side effect of Retrying is generator is not rewound on every retry attempt.
# Workaround when Retrying does not work with iterators as arguments.
class Wrapper(object):
def __init__(self, f):
self.f = f
self.backup = None
def __call__(self, iterator, *args, **kwargs):
(iterator, self.backup) = itertools.tee(self.backup) if self.backup else itertools.tee(iterator)
return self.f(iterator, *args, **kwargs)
retrying.call(Wrapper(grpc_stub.StreamFileContent), chunk_generator_object) |
We could have a need_cloning = [arg for arg in args if inspect.isgenerator(arg)] Or we cloud have a |
It'd probably better to have a flag because iterators can have different and side effect behaviour, so it might do weird stuff otherwise. |
I realise I'm probably just making a can of worms worse, but if you have a feature to rewind a generator before retrying, should you also have a feature to record the position a seekable file-like object was in before the first attempt, and |
Found an issue while using tenacity.
For a wrapped function that receive a generator as arguments, the generator is not rewound before each retry attempt, causing subsequent attempts to skip.
Not sure whether this can be solved within the core of this library.
The text was updated successfully, but these errors were encountered: