The main idea behind Clerk is letting folks use the editors they know & love and not force them to edit code inside a web browser. (This came to us only after working for a few years on nextjournal.com where folks can edit notebooks in a web browser but better late than never :)
In order to keep the feedback loop fast Clerk keeps a dependency graph of all forms and vars in a notebook to figure out what changed between runs on a notebook and only runs those.
> In order to keep the feedback loop fast Clerk keeps a dependency graph of all forms and vars in a notebook to figure out what changed between runs on a notebook and only runs those.
This sounds interesting. Is it essentially an abstraction where to the programmer it seems like the whole notebook runs again for every change? Almost similar to how React makes it seem to the programmer that they are regenerating the entire page?
Hey, I’m a founder of Nextjournal and one of the authors of clojure-mode.
Highly recommend checking out CodeMirror 6 [1], it’s really well done. On of the main benefits is its blazingly fast incremental parsing system called lezer [2], which is heavily influenced by tree-sitter. The Clojure grammar file is at [3].
I’ve done a small video about this at launch [4] and you can also play with this on Nextjournal [5].
For our notebook format at Nextjournal we use an edn appendix to store metadata like results and environment (docker image). The user-edited content is just plain markdown, while also being runnable reproducibly. See [1] and run it at [2].
Hey HN, I'm one of the founders of Nextjournal. The `cm6=1` query param is a feature flag to enable our new Clojure code editor built on CodeMirror 6 [1] with lezer for fast incremental parsing. You can find the keybindings on [2], be sure to try grow / shrink selection.
I used mmhmm today to record a quick demo [1] and think it works really well for that. I like the looks a lot more than the normal rectangular overlays in Zoom or ScreenFlow.
This is how runkit does it for nodejs and I think it’s working quite well for them.
We (at Nextjournal) tried doing the same for other languages (Python, Julia, R) and felt that it didn’t work nearly as well. You often want to change a cell at the top e.g. to add a new import and it can be quite annoying when long-running dependent cells re-execute automatically. I now think that automatic execution of dependent cells works great when your use case is fast executing cells (see observablehq) but we need to figure out something else for longer running cells. One idea that I haven’t tried yet is only run cells automatically that have executed within a given threshold.
I hear a lot of complaints about hidden state but I think it’s less of a problem in reality. It’s just a lot faster than always rerunning things from a clean slate. Clojure's live programming model [1] works incredibly well by giving the user full control over what should be evaluated. But Clojure's focus on immutability also makes this work really well. I rarely run into issues where I'm still depending on a var that's been removed and then there's still the reloaded workflow [2].
Overall I think notebooks are currently a great improvement for people that would otherwise create plain scripts – working on it is a lot quicker when you have an easy way to just execute parts of it. Plus there's the obvious benefit of interleaving prose and results. That doesn't mean we should not be thinking about addressing the hidden state problem but I think notebooks do add a lot of value nevertheless.
Author here, I did run it a few times but you don’t have to take my word for it, you can rerun the notebook yourself if you sign up for Nextjournal and remix my notebook. Full disclosure: I’m also a Cofounder of Nextjournal.
But I wouldn’t suggest to use Nextjournal for serious benchmarking (yet). We’re running on Google Cloud and it’s not suited for benchmarking unless you pay for a (big) sole tenant instance. In the future we plan to offer dedicated instances for benchmarking.
I don't think there is enough information to come to a conclusion here, but I doubt your proposed failure mode.
Based on my experience with largish data in mongo, my guess would be that the database size was >> than ram and, due at least in part to mongo's design, when the master failed over the memory state of the new master didn't have the working set present in RAM. This lead to a huge inrush of disk IO resulting in all sorts of bad until the RAM state sorted itself out.
This “spite” reveals serious core design flaws in Mongo that still have yet to be addressed.
If someone has data showing otherwise — that the abject reliability and scaling issues were resolved via something like a complete rewrite, I’d be glad to reconsider my views (they won’t have any though).
Honestly I'm a bit surprised they don't use a more mature, fully managed solution. They can't have so much traffic to not justify getting some traditional relational database (like Oracle).
It can take a namespace like https://github.com/nextjournal/clerk/blob/fd79f7504ebffaa279... and turn it into https://storage.googleapis.com/nextjournal-snapshots/clerk/b...
The main idea behind Clerk is letting folks use the editors they know & love and not force them to edit code inside a web browser. (This came to us only after working for a few years on nextjournal.com where folks can edit notebooks in a web browser but better late than never :)
In order to keep the feedback loop fast Clerk keeps a dependency graph of all forms and vars in a notebook to figure out what changed between runs on a notebook and only runs those.
There's no homepage for it yet but Clerk is running a handful of sample notebooks on every commit and building a static website out of them. The latest version is at https://storage.googleapis.com/nextjournal-snapshots/clerk/b...