From my understanding of the video, the queries are generated by the analysts via a frontend-generated or hand-written SQL query. The SQL query is parsed by PostgreSQL which forwards it via FDW[1] to Multicorn[2]. Their custom data storage and processing backend implements the API expected by Multicorn (e.g., you implement the multicorn.ForeignDataWrapper interface); this is where they transform from the parsed, serialized SQL into their custom DSL (the metaprogramming bit) which compiles to LLVM.
Yeah, that's pretty much how it works. The frontend is anything that supports PostgreSQL as a database. Right now we use Tableau but we also used to have a custom WebUI on top of this service, it was very functional-inspired.
Unfortunately Ville is on vacation right now, otherwise he'd be glad to dive more into the details of how that piece worked.
Any interest in also trying Parakeet (https://github.com/iskandr/parakeet) for the backend? I'm curious to see how the performance would compare with Numba. I also have a semi-usable Builder API which constructs typed functions at a higher-level than llvmpy.
Thanks! Our approach supports both discrete and continuous values. It is mainly optimized for the use case where you want to aggregate continuous variables over discrete filters.
--
[1] http://wiki.postgresql.org/wiki/Foreign_data_wrappers
[2] http://multicorn.org/