Thanks for this post! I'm very inexperienced with Clojure, but I'm a big fan of React. I've always wanted to understand how Om encourages immutability and how it deals with "UIs are stateful by definition", but I never managed to wrap my head around it, probably due to my missing Clojure skills.
Your post explained it in general terms. I will probably apply the same in plain JS now. It sounds like an interesting challenge to implement the Cursors idea using React's Update addon [1] instead of depending on Mori. (I don't like Mori because I don't get the feeling that it's intended to be used without ClojureScript - its API is rather cumbersome from plain JS)
I'm curious about the "cumbersome" claim. Is it because Mori forces you to write in a functional style? I've tried as much as possible to supply an API that doesn't stray too far from conventions JavaScript programmers are used to from underscore, lo-dash, etc.
Oh, it might have very well been me being to much of a newbie to even understand what was going on, so please don't take the claim too seriously.
I believe that it had something to do with it being relatively much work to update things deep in a tree.
Like, if I have Model -> Users -> User -> WallUpdates -> WallUpdate -> Comment, and I want to make an update to a comment, I'd have to do a lot of code writing. Is that true, or did I just miss the magic function that helps me do this?
But I'm now realizing that maybe, if I want to "manually" do that, I'm just doing it wrong - this should partly be what Om's cursors are meant to avoid, right? (this, and locality to components)
In general, if it helps you as feedback, I found Mori a little difficult to approach, having no Clojure experience. I suspect that some more examples of typical usage patterns at the top of the docs would do a lot. Like, which methods are supposed to be commonplace, and which ones are more for edge cases? What kind of things am I supposed to do all the time, what kind of things should I wish to avoid except in these and these cases?
FWIW, I feel like the same holds for Underscore/Lo-Dash's docs, except it's been less of a problem for me because I'm more familiar with typical usage patterns of mutable data structures.
But I bet you lose a surprisingly large number of JS devs due to just having the "wrong" names. Maybe I should publish a camel-case-mori module on npm...
This is because mori is written in ClojureScript. For the most part, all mori is doing is exporting pieces of Clojure so that the native JavaScript environment can access them[1]. In Clojure, the naming convention is "foo-bar", and the cljs compiler translates that to "foo_bar".
I can't imagine changing the cljs compiler or adding a piece to mori that must be maintained is important enough for this. I'd imagine Swannodette's answer would be "you should use ClojureScript anyway" :)
I don't think it'd be a huge issue in any case, probably just a case of providing a handler for name conversions. Not sure if it's worth the extra code lying around, but anything that potentially makes cljs a better actor in the js world is nice.
> I've tried as much as possible to supply an API that doesn't stray too far from conventions JavaScript programmers are used to from underscore, lo-dash, etc.
This is a nitpick but JavaScript convention is to use camelCase, not underscores, and it feels weird whenever you encounter a library that uses them, usually it's a library that is written by Ruby or Python programmers. It's irritating if it's a library you need to use a lot.
fwiw it's possible to achieve all of these benefits in vanilla react using javacsript with regular javascript data structures
* single mutable ref to normalized app state
* cursors for encapsulation and modularity
* O(1) deep equality checks
* fastest possible react performance
source: I have done it in two large react apps, but i have not documented it yet. Blog post coming soon.
Interestingly, as your app state gets bigger, you are actually forced to use something like cursors for all state updates, for performance reasons. Naive immutability in javascript means clone-before-modify, which is far too slow when your state values are large. It quickly bubbled up in our profiler. Funneling all immutable updates through a cursor means the cursor can use structure sharing on the state value, skipping the defensive clone and preserving reference equality for equal subtrees. Also afaik, implementing a general shouldComponentUpdate (edit: in js) without something like cursors is not possible, because you need === equality for onChange props.
One question I have for you: In those large React apps, did you put all state changes in the big single central app state data structure?
I'm currently building an app where the core state is simply a tree of plain old mutable JS objects. We have all the expected disadvantages (mostly: no easy shouldComponentUpdate), but things are easy to reason about, otherwise. We found that stuff like "has the user expanded a dropdown" or "which auto-completeable search term has just been entered" works perfectly in a local component's setState. The idea is, basically, anything that the user doesn't care losing when navigating away from the current screen can go in a local component's state. This seems to go entirely against the functional immutability idea, but it significantly reduces the amount of updates we need to do to our big central data structure.
I'm going to build a new thing and I'm considering going the nice functional way. However, I can't see any downsides to using directly mutable state to simple, local, unimportant stuff.
Would you do it like that? Or would you avoid component state altogether and go with props only?
We also use component local state for things like that, and David Nolen (Om) likes component local state too. But the state goes up top by default. The biggest reason for this is because you might want to attach a diagnostic component to your state (e.g. a json editor [1]), and you don't want to have to change the shape of your state to do that.
There are like six implementations of cursor-y things for React on the internet in varying degrees of quality. I feel pretty strongly that a quality cursor implementation should make its way into react addons before 1.0.
You can find mine from my github but it is not yet ready for public consumption. As far as I am aware mine is the only one that allows === checks on value & onChange. (I will find time to release and document it in the next week)
refs are not the highest performance reference type due to transactional semantics. ClojureScript already has efficient mutability in the form of atoms and transients. You can actually build a transient vector faster than you can build a mutable JavaScript array in recent builds of V8. I actually demo'ed this at JSConf. Fun stuff.
An Om question: what do you store in the app state and what do you store in component local state?
In my own application, I started off storing everything in app state, using local state only for short-lived transient data, but over time I found that having a bunch of information like what tab is currently visible in the app state became difficult to manage and made the app state a nightmare. So now I like to keep a strict separation: app state is basically my model - the data being processed (if I were writing a text editor, its the document being edited) and local state is everything else: which dialog is open, which tab is selected, button toggle state, mouse state etc (in a text editor, whether bold is selected, whether I'm in print preview mode or edit mode, etc). Basically app state is what is being processed and local state is how to process or display it. This separation also means that the app state is what gets synced with the server. I like to think of it as "if its something I might want to persist in the database, then it lives in app state, if its something I would never want in the database, then it lives in local state"
I'm curious what other peoples experience with this is.
Another Om question: how do you sync state between client and server, or how do you handle communication?
I looked at how om-sync does this (using tx-listen to automatically sync changes), but ended up building a more traditional request/response model using sente[0] and having my app ask for sub-trees of my app state as needed. Again, just wondering about peoples experience with this and different approaches tried.
We're still figuring out exactly where the boundary lies, but our current approach is much like yours -- view state that's purely local to a component goes in local state, and everything else goes in app state. This feels more natural, but means you lose some of the nice benefits of things like replay/undo for this local state.
For client/server, we've started building up some nice abstractions around our API (like fetching more data in a generic paginated list stored in a cursor), but the write side is still mostly manual. We're trying to use the same API for iOS and web, which probably prevents us from doing some fancier things that continue the same architecture back into the server.
So far, I've only made some (large-ish, granted) prototypes with Cljs and Om, but I've run into this as well. I ended up opting out of local state. I put view state and model state in the same atom, but I tried my best to keep them separate within the atom.
Easy replay/undo was one reason. Another reason was that I wanted to be able to take a _complete_ snapshot of the app state, for debugging purposes. A third was that I wanted to keep all the state that a component needs to know in order to render itself in one place, not many places.
I put viewmodel state with the app state - my app state basically is viewmodel state, domain state source of truth is the database really, so you need to do ajax to traverse the domain object graph, its not stored on the client.
Just like you want to control a react.dom.input, you also may want to control a tabstrip. Maybe you have some biz logic that says "When you click save, select the next record, and if record.type===2, select the third tab". You never know in advance what state needs to be controlled from the top, I found myself refactoring my viewmodel state higher and higher.
The mouse state is never going to be controlled from the top, so that is component local state.
It sounds like you're differentiating between model state and view model state.
I haven't used Om yet, but a potential approach could be to maintain two stateful objects: one for the model (e.g. stuff that you'd persist permanently), one for the view model (e.g. which tab is selected).
If you wanted your app to come back to exactly the same state after a page reload, you could persist the model in databases, and the view model in browser local storage.
Maybe a stupid question, but since I didn't see anything about that : what kind of database do you use, and how is it with Clojure ? React + ClojureScript seem a given for me on the interface side, especially with core.async and the other possibility, but I'm a bit more weary for the server side, and that's one of the last showstopper I have.
And, how do you go for teaching new comer on the project ? I do suppose you don't only hire Clojure or functional dev, so the transition part seem very interesting, especially with Om getting more (deserved) attention !
This is also something I'm curious about, particularly how migrations are managed within a team. Most of the solutions I've seen require writing plain SQL in migration files which you name yourself. It's simple but I miss the productivity of DjangoORM or ActiveRecord for example.
Or do most people use a schema-less database?
Most functional languages don't really go for any ORM. It's usually considered a cumbersome abstraction for relational databases. Clojure makes it really easy to use SQL directly with the java.jdbc package [1], or through some helper library like SQL Korma [2]. There are also several migrations libraries you can use [3].
It takes a bit of getting used to coming from an ORM, but you'll love the lack of overhead soon enough :)
What do your "templates" look like, and how easy they are to edit? In Montage we've kept our templates as close to plain HTML documents as possible, and the CSS experts/designers we've worked with have really appreciated how easy it is to understand.
I'm wondering if this is an issue for you (or perhaps everyone is comfortable enough with the ClojureScript/Om syntax that this isn't a problem)?
Strategically, how did you approach the rewrite? It sounds like you had a significant amount of frontend code; replacing everything at once can be risky. Did you choose specific pages and work your way in, or scrap everything and work your way from the inside out?
I'm a Javascript/Python/Ruby professional, and have made a few really swell SPA's in JS.
But my dream is to be a really good Clojure(Script) lisp programmer. I'm not so good at any lisp, but have went thru a couple of books, try to do every tutorial and to play in the LightTable instarepl and implement things that I can easily do in JS.
How can I, who am far from a lisp pro, get a job in a lisp/clojure shop? I think by doing something 8h/day in a team is the best way to learn, but of course no one will hire you if you slow down the rest (at least in the beginning).
How do you deal with propagating events through the system? If I'm clicking Follow as people are following and unfollowing, how does that number change?
Clicking follow mutates the global app data. Changes to the app data causes the entire application to re-render, and React pushes those changes out to the DOM.
So, to answer your question, we don't propagate events :)
Just learned about that recently -- it's a retina display issue (fix going up tonight). Thanks for pointing it out!
I have heard of it and used it myself.
The primary difference is really the intended use case. Where ETM is focusing on highly-specific (usually on a nutrient/caloric basis) meal plans to hit a target nutritional intake, we are more centered around using general preferences about food (ingredients, cuisine, cooking methods, etc.) in a "browsing" capacity -- currently in the form of recipes aggregated from other sites and bloggers. The goal is eventually to also incorporate nearby restaurant dishes.
For people who have wanted to try this type of programming out but have not jumped the boat from JavaScript to ClojureScript, I created a little library that mixes Backbone and React in a similar model described in the article: https://github.com/andrejewski/reactbone
It takes away a lot of the data-binding cruft that you find in Backbone.View and works well with React without making you switch languages, design paradigms, or add new libraries. The Backbone component makes it easier to build data hierarchies than with plain JS, but I assume other MVC frameworks could be retrofitted to work with React.
It's going to be much nicer when ClojureScript runs in Node without needing a JVM. Now a ClojureScript project requires a lot of boilerplate and running cljsbuild auto. For big projects this is fine but I want to write smaller scripts in ClojureScript as well. Basically I want to do:
cljsbuild app.cljs > app.js
And that's all. No project.clj, no waiting 5 seconds for clojure/jvm to load and Closure to compile it all.
Pretty much this. If I can get a ClojureScript interpreter (similar to CoffeeScript) that would be amazing and would lower the barrier for a lot of people.
I think what GP is complaining about is the compilation step, which requires a JVM and isn't exactly the fastest.
Touche - You don't need project.clj and leiningen and can run a command basically like you're asking for. I'm not sure how much faster it is than using leiningen and cljsbuild, as I haven't tried it: https://github.com/clojure/clojurescript/wiki/Quick-Start#cl...
this is what I was reffering too, the compilation step. It seems very daft for me to loose the closure compiler. I assume lein keeps the jvm hot, if not writing a script to do it should be pretty simple.
`cljsbuild auto` keeps the jvm hot and rebuilds any time the source changes.
You could also use something like Nailgun too, if that is more desirable.
When I develop in ClojureScript, I generally avoid rebuilding altogether whenever possible (auto or otherwise) and instead have my editor send changes to the browser REPL. I don't know if you can do this with node too, but it would seem odd if not. This seems to be the preferred approach by most Clojure devs.
Yes, but I factor in having to create a folder structure, edit the project.clj file to add my compilation targets, and all of that, it's not "light weight" for something like a small script that just is a single widget.
Btw, I ran: time lein cljsbuild once on a project with a minor change (added a line break) and it took 22 seconds.... so this is also what I'm talking about.
You're misunderstanding me, I know closure is written in Java. But closure is not the slow part. I timed doing lein cljsbuild once and it takes like 22 seconds; that's not Closure. That's Clojure itself. ClojureScript would be much faster, it has virtually none of the slowness that JVM Clojure has.
it sounds to me that the jvm is not kept running and you are paying a startup cost each time.
Transpiling a bucket load of any language in a warm jvm will be less than 2-3 seconds, and could easily be done on page refresh.
to give you some perspective, we have massive closure apps, first compilation on my (very powerful) desktop takes a minute 30 seconds or so (advanced optimizations and several other extremely stringent flags along with quite a few compiler plugins we wrote) keeping the jvm warm, and compiling with these same flags + plugins, the time comes down to under 5 seconds.
JVM startup time is slower than a lot of VMs, but it's not that slow. Write a hello world in Java and see how long it takes, I'd guess 100ms or so. Do the same in Clojure and you're looking at 3-4 seconds.
This is why I think ClojureScript/Node should become the "default" environment for writing Clojure. It would open up Clojure to a new class of apps that are currently not practical.
A hello world in java however has next to zero depedencies.
The jvm startup is notoriously slow keeping your jvm warm yields excellent results. Just try it please.
Things like nailgun exist for this exact reason.
I can't see them porting the cross compilation of clojurescript to node.
I'm real curious about the potential interplay between core.async and om/react. Bruce Hauman has a great tutorial on using core.async channels (as opposed to event callbacks) to process user input and gestures.[1] David Nolen (creator of Om) also covered core.async.[2] But these tutorials pre-date Om; core.async came to clojurescript in summer 2013, Om wasn't released until 2014.
I guess the way React handles input is by attaching event handlers to child components, so that changes to the child properties bubble up to the root component state. Is this the optimal pattern for handling events in a React app? Or is there an advantage to be gained by using core.async channels to process event input for Om/react?
I haven't seen any examples which use both core.async and Om/react, but I'm very curious about the possibility.
Ah yup, there it is in the basic tutorial, "Intercomponent communication". It is strange that I don't remember noticing that when I skimmed it. I must've been too distracted counting parentheses and trying to wrap my head around clojure syntax.
I'm relieved to know it was there the whole time and I simply missed it, thank you.
I rewrote the AngularJS tutorial in Om [1] and utilize core.async there.
The basic [2] tutorial on the Om wiki uses core.async heavily as well. Also, there are a ton of Om examples [3] within the repo itself that are a good reference.
I've been a very heavy user of Om for the past three months, so here's my take on it.
In my application, there is a lot of data that doesn't always need to be loaded - for example, various metrics can be viewed in charts, but since there can be a lot of data points, you wouldn't want to load metrics which aren't being viewed (since there could be many dozen different metrics, yet you may only care about 3 or 4 of them). So I think this is a good example for your question.
In itself, Om does not deal with this and assumes that the entire application state is loaded at all times. As far as Om is concerned, everything is always there in a big tree structure.
But since Om lets you update the state easily through cursors, it is easy to update sub-trees when needed. So there are two very similar approaches that I've tried out:
1. If you have data where you basically select one of a collection datasets (eg in my application, if I can only chart one metric at a time), then you can have an end-point in the tree for this data and you attach the currently selected data when it is needed, swapping the old for the new. You can easily do this with an om/update! command. It is also easy to then send the old data to the server while the new data is loaded into the state, if you have changes you need to keep in sync (om/transact! gives you the old value).
2. I have experimented with creating Om components that read the value of their cursor for some sentinel value (maybe nil or a special keyword or something) that tells it the data is not loaded, which then requests the data from the server before passing it to its child component for consumption (perhaps throwing up a loading indicator in the meantime). You can then also implement logic to unload data again (maybe a timeout after the child stops using it). This works quite well for me.
A possible improvement on #2 that I want to try out when I get a few hours is to make use of Clojures lazy sequences, rather than using a sentinel value to implement lazy loading. It seems like a natural fit, but I have yet to try it.
There are, of course, many other approaches one could take (eg, keeping track of what's loaded in a collection instead of using sentinels), but I have not tried any others.
So I guess, in summary, no, Om doesn't deal with this for you, but it gives you the tools to build it yourself to suit your needs. Personally, I like it this way because I know my access patterns better than Om does.
I saw some stuff about cursors as a way of displaying/modifying only a part of the global state. But it wasn't clear to me whether this is a formalized way of only loading part of the global state, or if cursors still require that the entire underlying "global state" is loaded into memory.
--
To give more background, let me give more specifics over what I'm thinking about when I ask this.
I've been working on a JS accounting app. One thing about accounting is that the balance of an account is dependent on every prior transaction that the account has ever had. So modeled in functional terms, it seems like you'd have to make this a tree of depth "N" where N is the total number of transactions in the account. It seems like you have to model it this way to get the data dependencies right (a change in an early transaction affects every subsequent transaction).
A functional approach with such a deep tree seems to break down (at least from the relatively little I know about Om). For one, I don't want to have to load every transaction into memory. Even though the most recent transaction depends logically on the first one, I don't want to be forced to load the first one just to show the most recent one. One possible way to work around this is to collapse all transactions prior to the ones I want to display into a single "fake" transaction that stores the accumulated balance as of this point. But now my in-memory model has to differ from my real model, and as the two diverge things start to get more complicated.
A bigger problem: I can't think of a good way, with this design, to indicate to the data layer which transactions the UI actually cares about and wants to have loaded into memory. If the tree has only one logical root (the most recent transaction) and is logically N transactions deep, how does the UI indicate which of those N it actually cares about? Maybe Om cursors can do this somehow?
I've been using a more traditional mutation-based/observable design, which seems to be working well. The UI can "subscribe" to certain transactions/balances, and this serves as a signal to the data layer about what needs to be loaded and what derived values need to be kept up to date. The data layer can be smart about how many transactions it actually needs to load to compute what the UI cares about.
I feel like a lazily loaded datastructure would solve this problem - you only need the last k transactions, then it only loads the last k transactions...
With regard to Om or Clojure I'm not sure of the specifics of such an implementation, but doing something like that with other languages that have functional programming support is one of the Major Selling Points.
No, you are still free to organize your application state however you wish. For example, our app dynamically pulls in content as you browse from feed to feed and use infinite scroll. It's up to you to decide what to fetch when, and what to cache/discard as the user navigates around.
where the app creates the page and the page creates the widgets and communication is flowing through channels?
and how do you deal with url transitions?
do you capture and flow up the click from a widget back to the app so it knows it's time to render a different page? and have some (case ...) at the app level?
or using some kind of mix with secretary and changing the location.href/a hrefs?
Yep, that sounds about right. For us, API communication and some global events flow through channels into the app state, where it's distributed to components. For routing we've built up a library around secretary, where we write down a map of uris, handlers, and metadata (can be a modal, etc). Info about the current page and such goes into the app state and that drives the transitions between page components.
Your post explained it in general terms. I will probably apply the same in plain JS now. It sounds like an interesting challenge to implement the Cursors idea using React's Update addon [1] instead of depending on Mori. (I don't like Mori because I don't get the feeling that it's intended to be used without ClojureScript - its API is rather cumbersome from plain JS)
[1] http://facebook.github.io/react/docs/update.html