
Python Generator Tricks for Systems Programmers - dood
http://www.dabeaz.com/generators/
======
iseff
The author of this, David Beazley, was a professor of mine for two courses as
an undergrad at UChicago. He taught Networking and Operating Systems and was
one of the top 3 professors in the CS department. He left teaching for a
while, but it looks like he's back now, which is extremely encouraging for all
CS students!

If anyone has the chance to listen to him speak or teach, I highly recommend
it. (Also, if anyone has the chance to see him perform jazz music, I highly
recommend that, too!)

~~~
kirubakaran
This tutorial is awesome. It is not just the content. The sequence and the
presentation is just plain awesome. You are a lucky basketball.

------
jcl
I was a little surprised that the genlog.py program is said in the slides to
be slightly faster than the nongenlog.py... That is, several chained generator
expressions actually ran faster than an explicitly written "for" loop. I
suppose it may have something to do with using the C-implemented sum()
function rather than doing a running total in Python; nevertheless, I expected
more overhead from the generators. Very cool.

------
jmtulloss
Stackless Python makes a lot of these possible without fancy generator tricks.
Not that I don't love fancy generator tricks, but I think the stackless model
is more intuitive.

~~~
bayareaguy
Can you provide a more explicit example, perhaps a Stackless alternative to
one of the ones from Beazley's page?

~~~
jmtulloss
The most obvious is coroutines:

    
    
      def ping():
          while ping_channel.receive(): #blocks here
              print "PING"
              pong_channel.send("from ping")
    
      def pong():
          while pong_channel.receive():
              print "PONG"
              ping_channel.send("from pong")
    

Most of the things that he has in there don't need to be done with generators,
though generators make it more efficient and arguably cleaner. However,
sometimes generators are just a twisted workaround for controlling program
flow. Channels and tasklets in Stackless express program flow very well, in my
opinion.

You can see more of what stackless has to offer here:
[http://members.verizon.net/olsongt/stackless/why_stackless.h...](http://members.verizon.net/olsongt/stackless/why_stackless.html)

------
mrtron
Very cool, just going through it now. It has the feel of my course slides in
university, but much more to the point and with better examples.

------
omakase
nice examples. just a note, pickle shouldn't be used for serialization if the
source is untrusted.

~~~
ntoshev
So add another step: sign the pickled data and check that upon arrival.

~~~
omakase
sorry i didn't elaborate, it isn't a question of signing requests, kirubakaran
posted a link below that explains the basics (unpickling can create objects
and run potentially harmful methods)

depending on what you are sending there are several alternatives for
serializing data. there are a handful of json libraries to take a look at and
some good benchmarks if you google for them.

~~~
icky
> sorry i didn't elaborate, it isn't a question of signing requests,
> kirubakaran posted a link below that explains the basics (unpickling can
> create objects and run potentially harmful methods)

Parent poster meant signing the pickled data as a text stream, not as a
collection of unserialized python objects. If the signatures don't match, you
just discard the data, and don't unpickle it.

This guarantees that (as long as your keys aren't compromised, and your
signing algorithm is unbroken) you will never unpickle tampered-with data.

