
Opnieuw: A simple and intuitive retrying library for Python - arianvanp
https://tech.channable.com/posts/2020-02-05-opnieuw.html
======
miguendes
What's the difference between this one and tenacity?

[1] [https://github.com/jd/tenacity](https://github.com/jd/tenacity)

~~~
BiteCode_dev
At least a good readme, which is always a good sign.

------
oefrha
Personally I would avoid this kind of npm-esque micro-dependency in my Python
projects. I’ve written essentially the same decorator for many of my projects;
it takes ten minutes and a few dozen lines of code.

~~~
whateveracct
Would you really want to roll your own exponential backoff w/jitter for every
project though? It's not the easiest to get right, and since you're looping in
a somewhat complicated way, it's definitely prone to accidental infinite loops
in my experience.

Retrying seems perfect for a library to me. For instance, Haskell has a
popular retry library
[https://hackage.haskell.org/package/retry-0.8.1.0/docs/Contr...](https://hackage.haskell.org/package/retry-0.8.1.0/docs/Control-
Retry.html) and I wouldn't consider it bad to just bring it in.

Is this more symptom of Python deps being "riskier" than Haskell deps due to
types or versioning or something?

~~~
oefrha
This library posted here is only ~150 loc without docstrings, and it’s more
complicated than my typical equivalent decorator factory with basically the
same features, sans logging.

I just don’t find this tricky. If only every problem I need to solve is as
simple as this one.

> Is this more symptom of Python deps being "riskier" than Haskell deps due to
> types or versioning or something?

No, it’s a distaste for tons of small third-party packages that could easily
be done in place. If anything I found Haskell deps more prone to breakage with
upgrades compared to Python deps (given my limited experience with Haskell),
at least with the set of popular packages I tend to use. Part of that is
Python packages tend to _not_ depend on a ton of other packages.

~~~
whateveracct
By "breakage with upgrades" do you mean bugs introduced due to
compiler/library bumps or just PVP constraint violations?

~~~
oefrha
The latter — not necessarily PVP constraint violations though, but rather
dependency hell situations. One example that pops to mind: packaging git-annex
for Homebrew was always “great fun” until we settled on resolving with LTS
Haskell (IIRC).

------
ekimekim
The thing that always gets me with retries is error handling.

What errors should be considered retryable, vs raise immediately?

Do you log a retryable error? I've had systems mysteriously "hung" before
which on closer inspection were infinitely retrying with a retry library that
didn't log anything...

And if you do exceed your max retries, which error do you raise? The first
one? The last one? There's no guarentee they'll be the same. A completely
different "TooManyRetries" error?

These are the concerns I'd like to see addressed in a high level overview of a
library like this one.

~~~
inetknght
Also: what if you get different retriable errors every time you re-try: should
you have different retry count for each distinct retriable error? When you
finally give up, should you report _all_ of the distinct errors or just the
earliest or latest retriable error, or ...

------
karlicoss
Would be interesting to read more specific comparisons with existing mature
libraries, e.g. backoff [0]. Feeling I get from reading the motivation behind
a new library is that most of it can be achieved by using backoff and perhaps
a custom delegating decorator that remaps parameter names and does extra
checks if necessary.

[0] [https://github.com/litl/backoff](https://github.com/litl/backoff)

------
tiku
The title is a dutch word, in English translated to "again" or "repeat".

~~~
jillesvangurp
The vowel combinations always trip up foreigners. Nieuw is Dutch for new and
it is pronounced essentially the same. The ew sound basically transliterates
to ieuw. Op (on) combined with Nieuw, basically means "on new", or 'again' in
proper English.

~~~
mercer
I'll be the pedant and argue that if you want to figure out how to sound
proper Dutch, there's a difference.

"new" sounds like "neeuuuuu" in US English, and a diphtong "nuuu" in British
English.

"nieuw" sounds like "niii" followed by a very short "oooo" as in "do" followed
by a slightly more pronounced "w" at the end.

I'm aware that there are proper ways to codify pronunciation, but it's more
fun this way and I know nothing of it.

------
timwaagh
Still seems a decent company. Nice that you take the time to properly deal
with technical problems instead of just doing the whole 'plough forward even
though you're just ploughing air' thing that happens just about everywhere.

------
leetrout
I don’t have a better solution but I was taken back by the length of the very
verbose / descriptive param

retry_window_after_first_call_in_seconds

Am I the only one that thinks that is a bit excessive?

~~~
m12k
If you're using something like this, then the details of when, what and how
you retry is probably something you care about, so having the library be
explicit about it and making it possible to know what something means without
having to look at the docs seems sensible to me.

~~~
anton_gogolev
This is some Java-level explicitness. Maybe, drop the `_seconds` suffix and
replace it with something like `retry_window_after_first_call = '60s'`?

~~~
bluebasket
60s. was wondering how we could add physical units to python? highjack the
number literals and variables like this: 1.s, pi.radians? or maybe the way
golang does it: 1 * time.seconds?

would be nice to have

~~~
bobbiechen
I also found this package:
[https://github.com/sbyrnes321/numericalunits](https://github.com/sbyrnes321/numericalunits)

Which uses the interesting approach of setting all "units" to random floats:
_A complete set of independent base units (meters, kilograms, seconds,
coulombs, kelvins) are defined as randomly-chosen positive floating-point
numbers. All other units and constants are defined in terms of those. In a
dimensionally-correct calculation, the units all cancel out, so the final
answer is deterministic, not random. In a dimensionally-incorrect
calculations, there will be random factors causing a randomly-varying final
answer._

Which presumably is done to avoid overhead from carrying around units with
each number, but forces you to run calculations multiple times (with different
random seeds) to verify the result is correct.

------
tignaj
If you are not familiar with how retrying is used and why exponential back
off, jitter, etc are useful here is a shameless plug to a blog post I wrote
recently [https://blog.najaryan.net/posts/autonomous-
robustness/](https://blog.najaryan.net/posts/autonomous-robustness/) (the
first half pretty much is about all these things).

------
sgt
I like the name. Maybe this should be merged into requests at some point?

~~~
sammorrowdrums
Do you speak Dutch? It's like the Python Zeep (SOAP - API protocol) client is
Dutch for soap, Opnieuw is Dutch for again.

I always wandered if that was a nod to Guido Van Rossum that people like
naming python packages in Dutch?

~~~
tijmendj
Channable is a Dutch company, and a lot of people who work here speak Dutch. I
don't remember where we got it from but it is possibly a nod to Zeep.

I joked that we should name our library such that you would have to do

    
    
      from driemaal import scheepsrecht

~~~
Kim_Bruning
"There should be one-- and preferably only one --obvious way to do it.

Although that way may not be obvious at first unless you're Dutch."

\-- Tim Peters, The Zen of Python

O:-)

------
sdan
I think I’ll try this, but for devs:

a simple while True + try/catch + time.sleep(n*2) will do exponential backoff
pretty easily.

~~~
x3ro
The post clearly states the problems they were trying to solve, none of which
are taken into account here (consistency, ease of parameterization,..) It
seems like people are reading too many "X in Y lines of code" posts :D

------
bluebasket
[https://github.com/invl/retry](https://github.com/invl/retry) works well for
me. only problem is that logging issue that's still open i believe.

seems to me that the authors wrote their own library instead of trying to make
the ones that didn't meet their need better. well, they never mentioned they
tried...

anyways, some of their choices like that long variable name are odd to me. i
can understand the unit problem though. golang solves that so nicely!

