Hacker News new | past | comments | ask | show | jobs | submit login
Why We Use Om, and Why We’re Excited for Om Next (circleci.com)
215 points by nwjsmith on Jan 21, 2016 | hide | past | web | favorite | 65 comments

I was the main author of Circle's frontend (the code is open-source: https://github.com/circleci/frontend).

I went on to build Precursor (https://precursorapp.com), which uses Datascript (https://github.com/tonsky/datascript) to accomplish a lot of the things that Om Next promises. If you haven't tried Datascript, you should really take a look! It does require a bit of work to make datascript transactions trigger re-renders, but the benefits are huge. It's like the difference between using a database and manually managing files on disk for backend code.

My understanding is that Om Next will integrate nicely with Datascript, so you can keep using it once you upgrade.

If you're interested in learning more about building UIs with Datascript, I'm giving a talk on Monday at the datomic/datascript meetup: http://www.meetup.com/sf-datomic-datascript/. I'll be going over Dato (https://github.com/datodev/dato), a framework for building apps like Circle and Precursor.

There's a typo in your datascript url (s/k are switched). Here's the corrected one: https://github.com/tonsky/datascript

Thanks, fixed!

Exciting. Are you plan on recording the meetup? Cuz that would be awesome :)

I've spent the last few weeks building a side project in Om Next, and this article is spot on. Really excited to see CircleCI's plans to migrate, as it'll be fun to read their code and learn how they use it.

Relay and Falcor are great, but when I look at their docs it's unclear how to integrate with whatever backend I want (especially Relay). Looking at Om Next, it was totally clear how to write my own backend. The tradeoff is that everything is a little more manual, but that control gives you a ton of flexibility.

In a small amount of code, I have a client that can query financial data in a bunch of different ways, and if the data isn't available it sends the query to the backend, which executes it against a SQLite database and returns it to the client. The components are all unaware of this: they are just running queries against data and everything just works.

Combine this is with first-class REPL and hot reloading support via Figwheel (both frontend and backend) and I'm blown away at how fast I'm going to develop this app.

It seems like the setup I use is pretty similar to Om Next, but via independent libraries. I am looking forward to seeing the finished Om Next to compare.

ClojureScript is shaping up to be a fantastic way to program browser-based applications. This is what I use:

* Reagent -- another ClojureScript React wrapper

* Datascript -- An in-memory database with datalog query lang, this is used as the central store for all application data.

* Posh -- Datascript transaction watcher that updates Reagent components when their queries' return data changes

* core.async -- used for handling any kind of event dispatch and subscription. I do a unidirectional data flow type thing and it only took like 15 lines of ClojureScript.

This is one of the nicest front end development experiences I've had. Just the composition of these four libraries gives you a ton of flexibility and a good way to structure your application. You can use this setup to write a real-time syncing/fetching system with a backend database pretty easily.

Our team used Om for our app (balboa.io) for the first 3 months of development. We switched to Reagent and have been using that for the last 8 months.

We ran into the same problems with Om as the CircleCI guys, specifically: 1) our front-end data model wasn't complex enough to merit a heavy-weight data access system that required a huge amount of extra digging to get right. We spent far too much time arguing about how to structure app-data, and it only got worse as the app got more complex. The cursor system in its first iteration was just too cumbersome (for exactly the reasons this author states). We kept trying to restructure the data model in order to get it to do what we needed. To be fair though, this is well known, and David Nolen has done a lot to alleviate this in recently releases (ironically by making it more Reagent-like). 2) our app is end-to-end encrypted and requires pulling down potentially hundreds of blobs, decrypting them, and inserting them into the dom. Under these conditions, Om would kick it and the UI would grind to a halt.

We switched to Reagent, and found that it was far faster and "got out of the way" of development. Add-watch is amazing too. Our app is quite large (front end SLOC is around ~50k lines), and Reagent has scaled beautifully and is a beast at large-scale insertions (on the order of 1000).

Om has some delightful features (undo ability is very powerful, routes coupled with Secretary is also great for Om), and David Nolen is a genius, but I think even the author has to acknowledge that the app-data/cursor construct is more of a pain than it's worth...

I really like David Nolen as a conceptual visionary. His work with Om and core.logic is great and has inspired a lot of derivative work. But I would never rely on his libraries in production. It seems like he always gets to 90% before moving on to the next new thing. 90% documentation, 90% cljs->js coverage, 90% tested, 90% issues addressed. I wouldn't touch Om unless I was willing to employ at least one person to work on Om full time.

Seems like that's as much (or more! esp. with docs) as you can expect from an open source maintainer.

For something to take off, the community has to step up. I get the concern, though. There's a phase where (if you're an early adopter) it seems like "everyone" is interested, but like, 5 people are experts.

I'd say 90% is pretty good for a single person. Open-source requires a community to work.

It is pretty good, but the community never seems to form. Maybe it is just the lisp NIH culture, maybe it is the way the projects are managed, but either way it doesn't make for production-ready software. You have to be able to support it yourself if you want to use it.

Did you know that David wrote all of ClojureScript? https://github.com/clojure/clojurescript/graphs/contributors

Umm, taking nothing away from his incredible stewardship and everything, but that's not true. I believe Rich Hickey et al did the first implementation. Not to mention that contributors have added hundreds of commits.

I don't have experience with Om, but another comparable React wrapper in the Clojurescript world is Re-Frame, and it has a top-notch community. The original developers are still active, but there are many new contributors who are adding substantial new features and polish.

It's David Nolen and the way he runs his projects, making people sign Contributor Agreements and such. He wants to remain the BDFL which is fine, his project, but the downside is that people are less enthusiastic to contribute.

As a contributor to some of David's projects, I gotta say, he does a great job of helping you contribute. He walked me through every issue I had until I could make a meaningful contribution. The one thing which makes it hard to contribute, is that he expects very high quality code, which requires time. I think it is a good tradeoff.

Theres a 160 people in the Om slack channel. There's more ways to contribute than sending patches, and theres quite a lot of community effort behind Om Next these days.

> Most data is not a tree

This is huge. I think it might even be the single largest problem to most projects' progress. I've seen a lot of projects that have tried to force non-tree data into tree-structures, and it never works out well. Projects grind to a halt after 6 months to a year because nobody can keep track of the dance steps they have to do with the tree-oriented code to manage their graph-oriented data.

Real, actual tree structures are just incredibly rare. Even some things that "obviously" seem like they should be modeled as a tree are far better off as a directed graph. Like databases of family trees - it's possible someone is literally married to their sister! Less cringe-worthy examples involving large families living near other large families with generational overlaps causing the children of one group marring the grand-children of the other, and vice versa.

You don't really need React. If you can do the ostensibly hard work of figuring out the DOM edits yourself, your app will actually be faster than if you're using React, i.e. React has its own overhead. As long as the data relationship was right, I've never found it difficult to manage state thereafter. It's when the shoe doesn't fit that things become a problem.

The problem is, we have a systemic problem of treating front-end devs as not "real" developers, not capable of forging their own paths. It's not just from the outside-in, I see a lot of front-end devs lacking a lot of confidence in their own skills. As a culture, we yell at any JavaScript programmer going his or her own way, building their own thing. "Don't reinvent the wheel!" they are told. Screw that. I can think of at least 3 times off the top of my head that the wheel itself was significantly and usefully re-invented in the 20th century alone. The problem is not "reinventing wheels". The problem is this institutional fear of making ones own decisions, leading people to think they need to learn everything.

> Application state as a single, immutable data structure.

react-cursor gives this pattern in javascript, immutability and all, but with regular old javascript objects. It also comes with all the same caveats as in this article. (I don't speak for the creator of Om, I speak for myself as the author of this library which was inspired by Om and Clojure)


The beautiy of the state-at-root pattern with cursors, is that each little component, each subtree of the view, can stand alone as its own little stateful app, and they naturally nest/compose recursively into larger apps. This fiddle is meant to demonstrate this: https://jsfiddle.net/dustingetz/n9kfc17x/

> The tree is really a graph.

Solving this impedance mismatch is the main UI research problem of 2015/2016. Om Next, GraphQL, Falcor etc. It's still a research problem, IMO. The solution will also solve the object/relational impedance mismatch, i think, which is a highly related problem, maybe the same problem.

> Solving this impedance mismatch

The solution seems now to get rid of the REST paradigm(GraphQL ...).

While I have no opinion on what should be the right solution, I welcome the idea of questioning the usefulness and the significance of REST, especially in the era of fat web clients.

My opinion is that REST is exposing the O/R impedance mismatch to the client, and we are only feeling this now as our client apps get really complicated, and i think the O/R imnpedance mismatch is caused by pervasive mutability, and i think the problem can be solved by making the database immutable (e.g. CQRS/ES, Datomic), and i think that will make the problems with REST go away too. (This is not quite what GraphQL is trying to do, GraphQL is progress but doesn't get to the crux of it) This is all research of course, and completely my opinion. Om Next is meant to be paired with Datomic so while his solution is to throw away everything that is even a little bit RESTy, i think that might be throwing the baby out with the bathwater. time will tell.

"Om Next is meant to be paired with Datomic" - although datomic makes some things easier, Om Next has no opinion on what backend you use.

This is a more coherent summary of Om than exists elsewhere, including the official Om site.

I wasn't surprised when I saw who wrote this. Peter's a great writer and educator. He has an excellent video series for explaining basic computer concepts.

What's not stated in the article, re: "Why we use Om", is that much of Om's adoption was because of its high-profile creator and all the status/momentum that brings along.

But I think it's approaching a consensus already within the CLJS community that, on API alone, reagent is the React interface you want.

It's extremely elegant and performant; probably the best frontend library I've ever used in close to a decade of web development.

So I've been looking at Elm recently, what would the advantages/disadvantages for something like Elm over Om/Om Next?

Elm's HTML engine conceptually the same as React+Redux and Om Now. It's just a big single atom app state with ways to describe how to update it. It has all the same problems as Om Now (maybe not quite as bad because it doesn't have cursors, but it doesn't solve the data-as-a-graph problem).

> Elm's HTML engine conceptually the same as React+Redux and Om Now. It's just a big single atom app state with ways to describe how to update it. It has all the same problems as Om Now (maybe not quite as bad because it doesn't have cursors, but it doesn't solve the data-as-a-graph problem).

That's really not true. The nature of the Elm architecture means it already does this:

> Instead, Om Next asks you to define a set of mutation operations which can change your application’s state.

except it calls that "the `update` function".

The graph problem also seems slightly different, although it is way less convenient than Om Next as it is less declarative and must be bubbled through intermediate "components", Elm "render state" (and render hierarchy) doesn't have to match the application state, it's perfectly possible (and very useful) to duplicate, denormalise and process application state during rendering to match your rendering hierarchy (although it is a very bad idea to denormalise the application state itself).

Elm's trouble is mostly the "remote mismatch".

Ok I see now. I think both methods have a place going into the future.

What I've understood from Relay/Falcore/etc. is those technologies are great for something like Facebook or Netflix, hence why they made the frameworks, but for many other cloud/web apps, it may not make sense. A single atom app state works really well. Is that along the lines of the right thinking? I'm using React+redux along with websockets and haven't had any problems with large state trees, but the whole state can be sent to and from the server very fast as it's not too big.

I'm sure Elm will have something comparable to Om Next/Relay/falcore as well then, to solve those use cases.

I don't think it's that's simple. It's not that it works better for hugely complex apps. It just works better for certain kinds of apps, even if they are small. The project I'm working on is small, but Om Next is far better suited to it (the complexity of using Redux+Redux/etc on it would be far greater).

The question is, are the majority of apps better suited for a tree-based state or a queryable graph? I'm not sure, but it's at least even.

I've heard Elm is working on something similar, so I wouldn't be surprised if that happens.

Is the same data displayed into multiple branches of the tree? :)

You mean the component tree? Sure, it can mixed in whatever way the component tree wants it.

Om Next has dropped the use of cursors.

"It depends".

Are you writing a whole new app? or are you integrating with pre-existing js? Elm is designed around statically-typed functional reactive programming, and as part of maintaining that integrity at runtime, it puts some ceremony around js interop. Clojurescript <-> javascript interop is lower-friction, with all of the potential runtime failures that entails. So it seems that the less your component needs to interact with js, the more pleasant Elm would be, and vice-versa.

What do you and your team already know? In circle ci's case, they already used clojure, so it was an easy choice to stick to that language on the front end. If you or anyone you know don't already have some familiarity with either an ML (in elm's case) or a lisp (in clojurescript's), consider looking into a language like that on the server at the same time.

And of the two, clojurescript is more popular. This entails some confusion, sure (om? om next? reagent? wha? vs the 'just use the stdlib' of elm) but it is generally a Good Thing. In my experience, you're more likely to find answers to cljs questions, to have better tooling ('top speed'), and an easier time setting up that tooling ('acceleration').

You'll notice I compared Elm and Clojurescript, not Elm and Om/Om next. I think the attributes of the two languages probably outweigh any particular differences in app structure between Elm's "StartApp" recommended structure and om.

Some interesting points made on this thread comparing ClojureScript vs. Elm: https://www.reddit.com/r/elm/comments/3zkkrw/clojurescript_o...

Clojure and ClojureScript for president in 2016! No, but seriously, one of the most promising pair of practically useful languages in existence.

I would love to see some code examples of this part "If we try to show a component that needs to know the current user’s initiated builds, that triggers an API call that asks the server for the data. If we stop using that component, we stop making that request. Automatically."

I'm currently knee deep in a react/redux implementation, which I guess is quite similar.

There's a basic start to a remote syncing tutorial here: https://github.com/omcljs/om/wiki/Remote-Synchronization-Tut...

Basically, you specify a query as a remote query and it calls a `send` function that you specify, and you manually pass the query to the server to execute, which will return results and automatically be merged into the app state (and of course, re-render components that depend on that query).

For the part where they stop making that request, they must do custom lifecycle code so that when the component unmounts, they are able to access current open requests (probably indexed by component manually) and cancel it. That part is not builtin to Om Next.

I am surprised their backend is written in closure. I would think it make hiring developers much harder (smaller group of people know it) and training people a lot harder. You can jump on to a project and learn enough go to fix bugs in a day or so (less than a week for sure). I am not sure the same could be said about closure.

It definitely did not make hiring harder. In fact, it was probably easier, because at the time, Clojure was more unique. The people interested in working in rare languages tend to be above average.

Also, I would avoid optimizing TTFBugFix the wrong way. You want the system to be easy to fix because it's well designed & documented, not because it's written in a language you already know.

What characteristics about these people makes them above average? I need some goals to set my sights on :)

Intellectual curiosity, and a desire to experiment with new languages to be more productive/write cleaner code.

I speak from personal experience: it's a lot easier to find a good Clojure guy than to find a good Rails guy for example. Because there's not many Clojure shops, you tend to get the good and motivated people :) As per my Rails example, it's full out there of Rails shop.. you're just another one.

I think you overestimate how hard it is to pick up a new language. Even the designer at Circle (https://twitter.com/dannykingme) learned enough ClojureScript within the first month to build components completely independently. It only required a little extra engineering time to hook things up properly.

From the last Clojure/conj (conference), it definitely seems that Clojure is at, or even past, an inflection point.

I doubt any companies (that are themselves, aside from tech stack, appealing to work for) are having trouble at least getting a lot of interest for Clojure positions.

>2 We could throw out the entire list and rebuild it in one go, but re-rendering large amounts of DOM is also slow.

Out of curiosity I tried swapping "<ul><li>Artichokes</li><li>Broccoli</li><li>Cabbage</li><li>Dill</li><li>Eggplant</li></ul>"

and the same without the broccoli and dill, back and forth a few thousand times using jquery.

The average time per change was 28 nanoseconds or 35000 changes per second (Chrome, MacBook Air). Trying swapping a list of 300 fruits for a list of 500 fruits was 1.4 milliseconds per change.

I wonder if using some convoluted framework to "solve—or at least mitigate" this might be premature optimisation? (As well as actually slower).

Author here. I actually glossed over some of the problems here, as the article was already getting quite long. In a trivial example like this, you're right. If you do this to the entire page, though, it gets stickier. An additional problem is that you're throwing away and rebuilding state that's associated with the elements, such as event handlers. (Also, letting React do the actual DOM manipulation makes it easier to implement transitions, where a Virtual DOM element goes away now, but the real DOM element sticks around for a bit while it animates off.)

To be fair, I haven't done actual benchmarks, and I'm basing this on the stated rationale for React. I'd be surprised if swapping out the DOM of most of the page wasn't considerably slower more difficult to work with than what React does, though.

Can anybody comment on how security is handled in Om? How do you ensure that certain parts of the database (which may depend on complicated rules) are not inadvertently exposed to the client?

In short that's up to the developer. It would be like asking how does Redux handle security? Well it doesn't, it just dispatches an action object to functions, the functions you wrote.

Om.next takes code you writes and makes it go.

> We could insert new list items into the existing DOM, but finding the right place to insert them is error-prone, and each insert will cause the browser to repaint the page, which is slow.

I don’t think the last part is true. Browsers don’t repaint (nor they reflow) the page until it’s really needed. So if you have a loop that modifies the DOM multiple times, but does not read from the DOM, there performance hit described by the author should not occur.

Essentially this is exactly what we have with Redux and pure JS. Gaearon has led the way here:

- Redux as your single state tree/graph

- Normalizr to denormalize data and store it in a graph, including pulling nested resources out as records

- Reselect to query on your denormalized data

And the best thing is this is production ready, in JS, today.

Nice article! I'm the author of https://github.com/zubairq/AppShare which using Om, and can say that Om Next is definitely the future of clojurescript apps

> Om’s creator, David Nolen, likes to show off how easy this makes it to implement undo: just remember a of list old states, and you can reset! the state atom to any of them at any time.

How does that work if multiple users are collaborating on the same state simultaneously?

> How does that work if multiple users are collaborating on the same state simultaneously?

The state they're talking about here is local to the client--it's stored in memory on the browser. One one user can use it at a time.

The post mentions borrowing ideas from GraphQL and Falcor: if I used GraphQL or Falcor, is there some pain I'd hit that Om Next would avoid?

Yes, a few. Look for David's talk on Om Next he usually lists were he departed from GraphQL/Relay and Falcor. A big one is that both GraphQL and Falcor are string based, while Om Next uses data structures which are easier to programmatically manipulate and compose.

The new plan sounds a lot like Redux.

No, Redux is far more like Om Now. In fact, Om Now was one of the early influencers for the single atom app state idea. The only difference is that, instead of using cursors, Redux uses flux-style actions for mutations. You select data from the app state by digging into the app state manually, exactly like you would in Om Now.

It's a common misconception (I fell for this too) that simply having a single atom app state and colocating state accessors with the component is similar to this architecture. It's really not. The query syntax is an extremely important part here, with transparent server syncing, and support for arbitrary graph representations of the state.

Om's "parser" and query syntax seems to decouple state read/writes from components in the same way that action creators do, with similar benefits. Reading through Om.next didn't change that impression, is there any other resource I should look at?

Action creators do not read anything from the state. They are only for mutations. You read from the state by literally digging into the JS object.

Not sure what else to say, actions and queries are completely different things.

It's pretty different. It's not flux or flux-like. And almost every Redux example or tutorial does use ES6 or more. As far as the platform, I had an easier time learning ClojureScript and Leiningen than Gulp/Webpack/Babel.

Not saying that Redux isn't easier, though. It's very nice.

That's a whole 'nother topic. I'm currently using redux with plain ES5, no compiler or build tools other than a minifier.

does anyone know how challenging it is to add Datomic subscriptions to Om Next?

This is pretty easy, because Om Next is low level and by design it can integrate into any back end. However, there is very little "hand holding" right now and it can be daunting to understand exactly how to do it.

For a lot of companies however a good first step would be to ensure that the pages work w/o Javascript.

When a basically crud website tells me my perfectly fine browser isn't supported I say: FAIL.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact