

Sexy Lexing with Python - rogercosseboom
http://www.evanfosmark.com/2009/02/sexy-lexing-with-python/

======
tdavis
Correct me if I'm wrong, but isn't this a bit like reinventing the wheel when
PLY is around? <http://www.dabeaz.com/ply/>

There's another popular library that escapes me, too.

------
blinks
I wrote one of these a while back: <http://knot.appspot.com/e/simple-python-
scanner>

For his example, it needs one more line:

    
    
      simple = Scan(
        IDENTIFIER = r'[\w_]\w*',
        OPERATOR = r'[-+\*=]',
        DIGIT = r'\d+(\.\d+)?',
        END_STATEMENT = r';',
        SPACE = r'\s+',
      )
      simple.ignore('SPACE')
      simple('foo = 5 * 30; bar = bar - 60;')
      # generates tokens.

