mrjob.retry - retry on transient errors

class mrjob.retry.RetryWrapper(wrapped, retry_if, backoff=15, multiplier=1.5, max_tries=10, max_backoff=1200, unwrap_methods=())

Handle transient errors, with configurable backoff.

This class can wrap any object. The wrapped object will behave like the original one, except that if you call a function and it raises a retriable exception, we’ll back off for a certain number of seconds and call the function again, until it succeeds or we get a non-retriable exception.

RetryWrapper.__init__(wrapped, retry_if, backoff=15, multiplier=1.5, max_tries=10, max_backoff=1200, unwrap_methods=())

Wrap the given object

Parameters:
  • wrapped – the object to wrap
  • retry_if – a method that takes an exception, and returns whether we should retry
  • backoff (float) – the number of seconds to wait the first time we get a retriable error
  • multiplier (float) – if we retry multiple times, the amount to multiply the backoff time by every time we get an error
  • max_tries (int) – how many tries we get. 0 means to keep trying forever
  • max_backoff (float) – cap the backoff at this number of seconds
  • unwrap_methods (sequence) – names of methods to call with this object as self rather than retrying on transient errors (e.g. methods that return a paginator)