Rb: A Redis parallelization toolkit for Python

[zeeg]

http://blog.getsentry.com/2015/08/20/rb-redis-parallelization-toolkit.html

[the_mitsuhiko]

Armin from Sentry here.

If you have any questions about the library or the whole problem in general feel free to shoot questions my way.

[btown → the_mitsuhiko]

So why not just use:

    gevent.joinall([gevent.spawn(call_get, get_client_for_key(key), key) for key in keys])

gevent uses epoll/kqueue in the background as well; as far as I know, no threads are spawned or locked. Is there some kind of fancy scheduling going on in rb?

[the_mitsuhiko → btown]

We are not using gevent anywhere and we can't really because of it not working well for the majority of our work. The redis pyrhon library only works with gevent if you monkey patch everything and that would be very problematic for us.

Lastly that does not support pipelining still, something that rb does for you n

[zeeg → btown]

Forcing a dependency on people isn't always the friendliest thing, especially when it comes to version, especially in Python.

We don't use gevent anywhere else, and while I'm certain it has a better implementation, it's about the right tradeoffs.

[itamarhaber → the_mitsuhiko]

no questions, only kudos and one of my favorite items in the latest edition of Redis Watch (COOLEST. LOGO. EVER.)

[kevinastone → the_mitsuhiko]

Could you expand on why you didn't implement this using Futures?

[the_mitsuhiko → kevinastone]

Futures as a concept compared to Promises as a concept? I think the difference is largely naming though i assume one implies being read only from the outside? The promise API just mirrors ES6 so that it's familiar.