
Boltons: A set of BSD-licensed, pure-Python utilities - pmoriarty
https://github.com/mahmoud/boltons
======
mhashemi
Library author and lazy README updater here to report that technically as of
October 09, 2018, boltons is 79 types and 146 functions, for a grand total of
225 utilities.

While I'm here, here are the ones I use most:

\- OrderedMultiDict:
[https://boltons.readthedocs.io/en/latest/dictutils.html#bolt...](https://boltons.readthedocs.io/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict)
\- esp the .inverted() and .sorted() methods

\- Exponential backoff and jitter generator:
[https://boltons.readthedocs.io/en/latest/iterutils.html#bolt...](https://boltons.readthedocs.io/en/latest/iterutils.html#boltons.iterutils.backoff)

\- remap (recursive map):
[https://boltons.readthedocs.io/en/latest/iterutils.html#nest...](https://boltons.readthedocs.io/en/latest/iterutils.html#nested)
(recipes here:
[https://sedimental.org/remap.html](https://sedimental.org/remap.html))

\- Atomic File saving:
[https://boltons.readthedocs.io/en/latest/fileutils.html#bolt...](https://boltons.readthedocs.io/en/latest/fileutils.html#boltons.fileutils.atomic_save)

\- Traceback utilties, structured tracebacks:
[https://boltons.readthedocs.io/en/latest/tbutils.html](https://boltons.readthedocs.io/en/latest/tbutils.html)

Also, my favorite bolton that's not a bolton, glom:
[https://glom.readthedocs.io/en/latest/](https://glom.readthedocs.io/en/latest/)

If you have a bolton you'd like to submit, I'm pretty merge-happy. Here are
the criteria:
[https://boltons.readthedocs.io/en/latest/architecture.html](https://boltons.readthedocs.io/en/latest/architecture.html)

~~~
jmadler
Why not contribute these upstream into the standard library?

~~~
mhashemi
Because most of these utilities were created to build things, and contributing
to the stdlib has never been on one of those critical paths?

But really, @marmaduke's got it right. The standard library has a MUCH higher
barrier to entry than PyPI. Have you seen python-ideas [1]? I truly do not
have time to have it out with some of the more vocal elements of that group.

But, down the list:

\- OMD might be a good fit for the collections module, I've chatted about it a
bit with Raymond Hettinger on and off. He wasn't totally against it, so that's
something!

\- Exponential backoff is too opinionated for itertools, and generally a lot
more high-level and conceptually modern than the rest of Python's built-in
networking facilities.

\- remap: I really like it, even though I'm typically not a big fan of
functional programming in Python. But the Python devs are even less
appreciative than I, with GvR disliking lambdas, and Py3 dropping reduce()
from the builtins. Plus after years of watching Python core dev, it doesn't
seem like the people with the time to work on core Python have much time left
over to work with IRL complex APIs and other sources of dynamic nested data?

\- Atomic file saving probably should go in the stdlib, but probably not
relying on ctypes (on Windows) and I'm not going to pull out my old Windows
laptop again just to start writing C.

\- Traceback utilities: Probably the one I wish I had time to push for most.
traceback's string-only approach feels really dated in the structured-logging
age. I'd expect a lot of FUD around messing with error handling.

Maybe now that boltons is pretty popular it's worth giving it a go just to see
what lies beyond the typical listserv response: "put it on PyPI and if it's
popular, we'll see" :)

[1]: [https://mail.python.org/pipermail/python-
ideas/](https://mail.python.org/pipermail/python-ideas/)

~~~
jmadler
Ah,ok!

------
j88439h84
Toolz is one of my favorite python libraries. I use it all the time. It
provides even more of this useful functionality. It also has a sped-up version
called Cytoolz written with Cython.

[https://toolz.readthedocs.io](https://toolz.readthedocs.io)

[https://github.com/pytoolz/cytoolz](https://github.com/pytoolz/cytoolz)

~~~
ktpsns
Toolz is really a game changer since it allows you to write elegant functional
code (lazy evaluations, currying, pipes, parallelism) in Python which feels
native.

As python has the "batteries included" philosophy, such useful gems should be
included into the standard library...

~~~
TylerE
I will kind of disagree.

While having stuff in stdlib can be convenient, it all tends to be where
active development goes to die. Compare httplib with something like Requests.

~~~
maxerickson
The comparison would be with urllib.request:

[https://docs.python.org/3/library/urllib.request.html](https://docs.python.org/3/library/urllib.request.html)

Which right there recommends Requests, but for simple tasks it's kind of
unfortunate that people bring in a library dependency.

------
fermigier
I'm a big fan of @mhashemi's work in Python in general, even if I don't use
Boltons personally.

The discussion regarding inclusion in the stdlib is sensible, and I fully
agree with @mhashemi's comments. See similar discussions, for instance,
regarding dataclasses vs. attrs
([https://github.com/ericvsmith/dataclasses/issues/19](https://github.com/ericvsmith/dataclasses/issues/19)).

Sometimes, however, it's a bit annoying when something that you think should
be widely available isn't included in the stdlib. The cons of course are:

\- Fragmentation (several project pursuing similar objectives)

\- Not being able to use a powerful construct to simplify code in the stdlib
itself (the stdlib can only depend on the stdlib).

Regarding, for instance, functional programming, I maintain a list of
interesting projects in this domain in Python:
[https://github.com/sfermigier/awesome-functional-
python/blob...](https://github.com/sfermigier/awesome-functional-
python/blob/master/README.md#libraries)

There are at least half a dozen libraries that strive to achieve similar
goals, and would probably benefit from being included in the stdlib (at least
partially). It's hard to pick a winner among the list (like other in this
thread, I went with toolz).

I believe that a way to solve (at least partially) this conundrum could be to
work on some common specifications, like the JS community did with Fantasyland
([https://github.com/fantasyland/fantasy-
land](https://github.com/fantasyland/fantasy-land)), and then let projects
implements the specification the way they want.

WDYT ?

------
jszymborski
Some of these are really fantastic. Particularly the atomic file saving and
cache utils, but I can see myself using a lot of these handy util modules.

A lot of this stuff is great to implement and test yourself in large
codebases, but when you're prototyping or writing a quick notebook, this stuff
ranges to very handy to very powerful.

------
VectorLock
[https://github.com/mahmoud/boltons/blob/master/boltons/easte...](https://github.com/mahmoud/boltons/blob/master/boltons/easterutils.py)

~~~
maxpert
Gob's program!

------
candiodari
> Chunked and windowed iteration, in iterutils

I _need_ this. Makes so many functional expressions so simple.

~~~
pmoriarty
Could you give some examples?

~~~
btgeekboy
I wrote some similar code one time that generated SQL which was manually
reviewed. It generated a massive WHERE IN clause. Oracle only accepts 1000
values. So I fed it a list of thousands of values, and it chunked them into
rows of N objects, and into larger chunks of 1000. Having some code to do that
automatically would be quite nice.

~~~
Bootvis
Why not insert the values in WHERE IN to a temporary helper table and join on
that? It seems somewhat easier and might perform better.

~~~
orf
Oracle has a limit on the number of _parameters_ in a query, not just in the
`WHERE IN` clause. So you'd need to insert them into the helper table in
batches of 1000.

~~~
Bootvis
Good point, but isn’t that done automatically by whatever tool your using?

And if the table your selecting from is big enough I would expect that this
setup would be faster but it’s hard to say. It would also more easily
guarantee that your results will be consistent.

------
kristaps
While convenient, some of the utils can be slowish due to input validation
which might not be strictly necessary, etc. - check the implementation if
using in a performance critical task.

Apart from that, very useful lib, highly recommended.

------
_pmf_
Be careful when googling ...

------
WCityMike
First thought was a mental picture of multiple Michael Boltons.

