
Show HN: Inline C in Python - joejev
http://quasiquotes.readthedocs.org/en/latest/c.html
======
phaefele
This is great. Wasn't too clear on exactly how this is used from the documents
(e.g. do I just run 'python') - didn't seem that there where any 'imports'
either.

Found this example which seems a little clearer:
[https://pypi.python.org/pypi/quasiquotes/0.2.0](https://pypi.python.org/pypi/quasiquotes/0.2.0)

~~~
joejev
I was considering linking to the README (the index of the readthedocs page)
but I wasn't sure if that would highlight the C inlining as much as I wanted.
To summarize from the readme: After `pip install`ing the package, you can run
quasiquoted code by including `# coding: quasiquotes` and then running under
the normal CPython interpreter.

~~~
p4wnc6
This still doesn't make it easy to understand how it is executing. Is there a
hook on import that detects the file in which it's imported, searches for the
# coding language pragma line, and then actually alters the text of the source
code before the Python interpreter resumes parsing the rest of that file?

Initially I thought this was something like James Powell's "astexpr" literals
example (towards the end of < [http://www.youtube.com/watch?v=2O7yj-
Nh6AY](http://www.youtube.com/watch?v=2O7yj-Nh6AY) >), which would require
actually implementing a new C-API type, adding parser logic for its literals,
and recompiling Python.

If you can add a comment or two to clear up my confusion, I would really
appreciate it.

Great work on this!

~~~
joejev
I think some of this might be addressed in the implementation section of the
docs.
[http://quasiquotes.readthedocs.org/en/latest/impl.html](http://quasiquotes.readthedocs.org/en/latest/impl.html)
To summarize: there is no import hook or ast parsing happening. Instead, this
works by manipulating the way python reads the bytes and converts it to
strings. When the raw bytes are read, it looks for token patterns that match
the syntax of a quasiquote. It then replaces those patterns with normal
function call syntax that is valid. A cute thing that can be done (that I need
to document) is that you can "translate" any python from quasiquotes by doing
something like:

'[$qq|this is the body|]'.encode('utf-8').decode('quasiquotes')
"qq._quote_expr(0,' this is the body')"

~~~
p4wnc6
Ahh, that makes sense now. So it's leveraging the built-in hook for custom
decoding. I actually did not know that the `# coding: utf-8` happened at parse
time, nor that it allowed for custom decoding function.

I'd be very happy to help better document this stuff in the code. Searching
around, there are not very many good documents describing how to create and
register a custom decoder.

I have read and re-read your setup.py file a few times now, and followed the
trail to the .pth file and to the locations where codecs.register is used. But
it's still not clear to me how this functionality gets "installed" such that
the decoder hook that parses your decoder's name ("quasiquotes") is able to
map that back to the search_function from the installed module's source code.

Basically, how does the codec registry become permanently aware that
"quasiquotes" maps to the search function in your installed package?

~~~
joejev
The .pth imports a file which at module scope registers a search function with
the codec system. This function checks if the name is 'quasiquotes', if not,
return None otherwise return a custom object that defines all the methods
needed to turn bytes into string objects or go the other way. Because it is a
pth in the site-packages, it gets executed at interpreter start. If you wanted
to chat more about this you can email me with the email on my gh page. Not
sure how long hn threads normally stay up.

------
hogu
wow that is pretty nuts! do you think it would be possible to add a quasi
quoter that could inline cython?

~~~
joejev
Cython would definitly be possible. It would probably work the same way the C
quasiquoter does where it creates a function that accepts the scope and then
executes the code. It would even be easier because it wouldn't need to do a
first pass compile to get the missing names. I will try to work on this soon.

~~~
hogu
thanks! looks really cool

btw - I suspect a different title might help get more traction on hn -
inlining C isn't that interesting (scipy.weave has done this for years) but
this approach to inlining is super cool, and can be extended to other langs.
none of this is obvious from the title

