I'm a big fan of reactive notebooks such as Observable in ~javascript and Pluto in Julia.
I've been thinking about something somewhat similar using js_of_ocaml instead of Rust+wasm, but I guess it should be similar. The error messages given by chumsky seem very helpful.
A couple questions come to mind:
- I see that Datalog isn't turing-complete so you added the possibility to run js inside backquotes. Is it possible to add syntax highlighting to make it easier to read ?
- Why does it compile to js instead of wasm ? Wouldn't it be potentially faster ? Is it harder because more low-level ? (I don't know much about compilers)
- Since the execution happens inside web workers, I guess it could be more scalable than the other single-threaded alternatives ? Are intermediary results cached ?
- What do you think could be the killer use-case for Percival ? Exploratory data analysis of public datasets ?
- Could it be possible to embed some number-crunching stuff, like lightweight machine learning ?
And finally, what did you enjoy the most while building this ? Were there many pain points ?
Hey, thanks for the very insightful comments! Happy to chat about that :D
- It is possible to add syntax highlighting to the JS components using nested grammars in CodeMirror! I haven't gotten around to it yet though.
- It compiles to JavaScript because the data structures I'm using and passing around are all JavaScript data structures, and modern web browsers also have quite fast JIT compilers for JavaScript. There would likely be some speed bonuses from using WebAssembly in the future, but it would be significantly more complicated - most likely, I'd have to develop some kind of runtime library for the balanced binary search tree indices and everything (likely in Rust), then compile that to Wasm and link everything together. :D
- Execution is done in Web Workers primarily to be sandboxed. As mentioned in the tutorial notebook, we don't want long-running computations or infinite loops to block the render thread and freeze the user's browser; that would be bad! It also lets the user cancel computations if the cell becomes stale. Having multi-threaded execution is also a bonus. Cell outputs are not "cached" directly, but in the sense that they aren't rerun unless their dependencies change. :)
- Exactly! That's what the vision is - exploratory data analysis of public datasets that is reproducible, tangible, and easy to share. I'm not sure if the current form of Percival is the best for this, but that's precisely the problem that I've been thinking about.
- Hopefully it would be possible to embed number-crunching like you mention, though I would really like a declarative interface for such computation. Maybe this could be a cool idea for future research or extension!
I liked working with chumsky for parsing a lot. What really made me excited as this project was taking shape was how _fun_ it felt to interactively play with data queries and see the results appear in my browser in real time. It makes me happy as a web developer as well as a PL designer, both as someone who makes user interfaces. :)
I also just wanted to mention here, because I saw that many people only saw the technical details in the GitHub README but not the actual tutorial / demo notebook - please see the website below for an introduction to Percival! This was my bad for not being very clear in the description (you don't need to install anything).
I'm a big fan of reactive notebooks such as Observable in ~javascript and Pluto in Julia.
I've been thinking about something somewhat similar using js_of_ocaml instead of Rust+wasm, but I guess it should be similar. The error messages given by chumsky seem very helpful.
A couple questions come to mind:
- I see that Datalog isn't turing-complete so you added the possibility to run js inside backquotes. Is it possible to add syntax highlighting to make it easier to read ?
- Why does it compile to js instead of wasm ? Wouldn't it be potentially faster ? Is it harder because more low-level ? (I don't know much about compilers)
- Since the execution happens inside web workers, I guess it could be more scalable than the other single-threaded alternatives ? Are intermediary results cached ?
- What do you think could be the killer use-case for Percival ? Exploratory data analysis of public datasets ?
- Could it be possible to embed some number-crunching stuff, like lightweight machine learning ?
And finally, what did you enjoy the most while building this ? Were there many pain points ?