

Rebuilding the Shopify Admin: Deleting 28k lines of JavaScript - jashkenas
http://www.shopify.com/technology/15646068-rebuilding-the-shopify-admin-improving-developer-productivity-by-deleting-28-000-lines-of-javascript/

======
drderidder
Complexity can hurt on the server just as much as the client. The same thing
that Shopify experienced with Batman.JS had affected our server-side codebase:
uncontrolled complexity stemming from ad hoc design and frameworkophilia. We
achieved incredible performance improvements by moving away from server side
rendering to a lightweight, in house micro framework [1] and making
improvements in our server side API code, going from many tens of thousands
SLOC down to one-tenth of that, and from 0% test coverage to ~85%.

AFAICT its _very_ rare to find developers with that unique ability to write
truly minimalist code. The natural tendency of almost all of the developers
I've worked with seems to be to add complexity and to revel in their ability
to navigate that complexity. That's why, given the choice, I try to hire or
recommend developers who also have a background in music composition and/or
visual arts - people who are motivated more by producing an end product that
evokes beauty, not by the intricacies of the medium used to produce the work.

Batman was led by someone who previously worked on the Cappuccinno web
framework, which is based on its own programming language (Objective-J) and an
attempt to port most of the Cocoa framework over to the web environment.
Batman was different, but it might say something about its tolerance for
complexity. Cross-compilation from 3rd generation language X to 3rd generation
language Y has always been a bad idea imho, and seems to be based on a desire
to mitigate the effort of learning, much like the tendency to reach for a
framework with the most bells and whistles instead of understanding the
simplest way to solve a problem. Trying to counter that tendency towards
complexity has motivated a few talks I've done at our local JavaScript meetup
[2][3][4] (apologies for being overly self-referential).

[1]
[https://github.com/darrenderidder/talks/tree/master/jswidget...](https://github.com/darrenderidder/talks/tree/master/jswidgets)

[2] [http://ottawajs.org](http://ottawajs.org)

[3] [http://51elliot.blogspot.com/2012/08/a-simple-intro-to-
mvc-p...](http://51elliot.blogspot.com/2012/08/a-simple-intro-to-mvc-pubsub-
and.html)

[4]
[http://darrenderidder.github.io/talks/ModulePatterns/#/](http://darrenderidder.github.io/talks/ModulePatterns/#/)

~~~
cageface
_AFAICT its very rare to find developers with that unique ability to write
truly minimalist code._

Too true. I find that the gating factor for development for me is the number
of things I have to keep in my head at one time while working with a codebase.
So simpler and cleaner abstractions mean that development goes faster with a
lower defect rate. Maybe it's because I do have a background in music that I
prefer to write minimalist code but I think there are a lot of objective
advantages in it.

------
jawngee
From the perspective of someone deeply rooted in UI development for the
desktop, seeing stuff like this makes me glad I am deeply rooted in UI
development for the desktop. If I had to build a Cocoa app with this messy,
complicated shit ... I'd blow my brains out. All over my desk. Maybe I'd leave
a note, but I think people who knew what I did for a living would _get it_ and
not need me to leave one.

It also makes me wonder, how we've all decided that HTML is the way to go for
this class of application, how hacking a document display format to ape a
native UI is a good idea.

I have to think that at some point people think that there has to be a better
way to deliver a network transparent, portable UI without worrying about
strange things like a Shadow DOM, the hack that is CSS, and all of the other
weird shit web developers have to put up with to create something that mimics
(usually poorly) a desktop app.

Like is there a breaking point where people are like "fuck it" and come up
with an alternative browser that is specific to this and similar kinds of
applications? I guess not because the gray line between what HTML is good at
and what people need to HTML to do despite itself is a pretty big fat line.

~~~
mox1
HTML + CSS + Javascript is obviously far from perfect, but is there any other
way _TODAY_ to write a single piece of code that will run on just about every
computing device in circulation?

Honestly, I challenge anyone to write a simple UI (lets say draw a circle and
put some text on the screen) that just works across ALL major operating
systems, screen sizes, versions, etc.

If do actually end up doing it in anything besides "web" languages, its going
to be more complicated and take longer as well....

To make a car analogy, The Chevy small block V8 engine was/is never the best,
fastest, most efficient engine. It's just that for the past 40+ years they've
put the damn thing in every vehicle imaginable. It's pretty ubiquitous in the
auto world....It just works and everyone still uses it. Sort of like how you
will find a HTML, CSS, Javscript rendering engine in everything today...

------
carsongross
Money quote from the article:

 _The new admin would make a return to more classic architecture with some
modernizations. Our approach would be ERB views and server-side rendering,
with the use of Turbolinks, and a lightweight custom JavaScript binding
system. This allowed us to tackle problems of code duplication and developer
productivity in a single blow._

As always, the devil is in the details and I'm looking forward to seeing what
their JS binding system looks like, but the pendulum appears to be swinging
back towards server-side rendering and models, even amongst the cool kids.

I have been working on a small library for doing HTML partial AJAX
programming, using fairly straight-forward HTML attributes and traditional
server side rendering (plus some goodies like custom HTTP header support,
timers, etc.):

[http://intercoolerjs.org](http://intercoolerjs.org)

I'm using it successfully in a few projects and very much enjoy the simplicity
of the whole approach when contrasted with full MVC systems.

~~~
tericho
Can we please drop the "cool kids" cliche? It's a tired generalization.

Clearly at this point both architectures work, it's childish to see this as an
argument for server-side rendering. Obviously it works, so do client MVC
systems.

The value in this article is the humble detailing of their mistakes that many
of us also experience in our careers.

~~~
matthewmacleod
I think part of that is a reaction to how many people jumped on to the client-
side JS framework bandwagon very early, because it was "the next big thing",
but before the tools were ready for production.

I've had to deal with more than my fair share of crappy, hacked-together Ember
apps that were developed like this. There have been too many breaking changes
to upgrade, performance is bad, an they're full of hacks to work around
deficiencies that existed back then.

Even though we actually have production-ready frameworks now, that sort of
thing can leave a bad taste in your mouth. Especially when you see the same
mistakes made by people jumping onto the Meteor bandwagon, for example. But
it's defintiely something to try and get over :)

~~~
matthewrudy
Ember now feels like a complete framework, but fundamentally any Ember app
will be tightly coupled to Ember the framework.

In many ways this is the same as Rails Apps have traditionally been very much
tied to Rails.

Having worked on many Rails Apps that have grown beyond the scope of the
framework, the apps I build today are as decoupled as possible from the
framework.

I hope that the Ember community can grow in that direction too, and at a
distant look, it feels like Web Components will be that method of decoupling.

Right now I tell everyone that I'd never render HTML (or JS) on the server,
and I hope that JS frameworks will keep maturing with my clients' needs, so I
don't have to renege on that approach.

------
coldcode
While it's refreshing to see people talk about the things they did wrong and
how they intelligently approached fixing them, building your own javascript
framework from scratch (batman) is something I would have said no to from the
start. Not for the faint of heart or those who have not suffered such an
endeavor before.

~~~
Bahamut
If I had to guess, none of the solutions back then sufficed for them. Batman
has been around for a while.

~~~
funion
You're dead on. We started work on Batman some time ago now - options then
were quite limited.

~~~
tericho
2 years ago I was exploring frameworks for an enterprise project. I tried
Batman along with most others, I can easily see why building your own
framework would have been an option any earlier than that.

Most frameworks weren't even at 1.0 stable yet. I initially wrote a fairly
extensive prototype with Ember, but there were breaking changes so frequently
I couldn't keep up with the learning curve. In the end I chose Angular because
it had the best documentation, Google dogfooding, and testing was an obvious
priority. Worked out so well we're on our 4th major Angular app now.

------
fidz
A little bit OOT, i need to connect to my VPN in US to be able to see the
post. It is always redirect to my country's TLD (www.shopify.co.id), in which
there is no related link
([http://www.shopify.co.id/technology/15646068-rebuilding-
the-...](http://www.shopify.co.id/technology/15646068-rebuilding-the-shopify-
admin-improving-developer-productivity-by-deleting-28-000-lines-of-
javascript/))

~~~
wvanbergen
Ah, this is a bug in our redirect system. I will look into it.

~~~
wvanbergen
fidz: should be fixed now. Thanks for reporting this.

------
cageface
With all the buzz around SPA frameworks like Angular or Ember it's a
worthwhile reminder that there's a still a lot of room in the world for more
conventional server-side rendering approaches. Not every web app needs to be
an SPA and old-fashioned Rails apps can be a lot simpler to implement.

~~~
jbergens
On the other hand I had to develop a small admin framework and decided to do
it with server rendering since I thought it would be easier. I soon found all
the small details I needed to think about and that there were a number of
server side components/files that I had to create to do what I wanted to do.
It would probably have been easier to just do it in the SPA framework I am
used to (Angular).

~~~
bigpeopleareold
Sometimes I wish I can do more server-side rendering etc.

I have been working on single page applications for awhile now; I miss the
ease that is afforded by just capturing state through server-side page
transitions.

Specifically, with SPAs (existing codebases now mostly), I run in bugs a lot
that involve Backbone events: when are they firing, what part of the codebase
is firing them, who is ignoring the global state and firing anyway. Sometimes,
looking through a call stack is not even helpful. Every project that I worked
on had these problems. The last big bug I dealt with involved, for me, an
unfamiliar codebase and about a week-and-a-half of sitting with the debugger
tracing every line.

While for new projects, you can work very hard to ensure that firing events is
hygienic, existing projects might not be so hygienic.

With a server-centric approach, each page load limits the mental load with
state. On page-transition, the page state is clean.

While I think frameworks like Angular is very interesting, I tend to question
my own ambition personally. Something like the approach that Turbolinks takes
might actually be more appealing.

------
josho
I started down a similar path with ember.js and was starting to see the same
issues. Primarily the business models duplicated on the client. So, it strikes
me that we are still figuring out the right path forward on the client/server
balance.

------
mromanuk
_Duplication and poor documentation made it difficult for developers to make
changes to the admin. Once we started questioning Batman, we saw that the
evidence strongly indicated it was time to move on to the next chapter._

I don't see a SPA vs Server side rendering debate in the article. Building
your own framework was the problem, is not a core competence of Shopify to
build JS frameworks. Maybe they could choose other route, like going with
React or Angular there.

~~~
wvanbergen
We actually evaluated other options as well, including Angular, but found them
not the best solution for our problem. We documented our experiments and
evaluations at the time here:
[https://gist.github.com/kristianpd/f4c2e0aeb53d09f6def1](https://gist.github.com/kristianpd/f4c2e0aeb53d09f6def1)

------
Gigablah
I ported one of Shopify's open source projects (their dashboard tool) to
Golang a while back, and in the process I also managed to wrangle the
Batman.js frontend portion into a Grunt pipeline. It was a challenge, to say
the least :p

[https://github.com/gigablah/generator-dashing-
go/blob/master...](https://github.com/gigablah/generator-dashing-
go/blob/master/app/templates/Gruntfile.js)

I've been toying with the idea of replacing the whole frontend with something
much more lightweight. Perhaps a good time for me to play with Mithril or
React...

------
mabbo
My teammate and I compete to see not how much code we can write, but who can
have the highest sum of lines deleted in all commits. Any old hack can write a
lot of code, but to remove code while still adding functionality (or at least,
not removing it) is the sign of a good developer, imho.

~~~
iagooar
We do that too, in our team. Reminds me of: [http://delete-your-
code.herokuapp.com](http://delete-your-code.herokuapp.com)

------
peeyek
Wondering why Shopify redirected my request on that post URL to
[http://www.shopify.co.id](http://www.shopify.co.id) . Ngghhh, i can't read
those sweat post :(

~~~
hackerboos
Try:

[http://webcache.googleusercontent.com/search?sourceid=chrome...](http://webcache.googleusercontent.com/search?sourceid=chrome-
psyapi2&ion=1&ie=UTF-8&q=cache%3Awww.shopify.com%2Ftechnology%2F15646068-rebuilding-
the-shopify-admin-improving-developer-productivity-by-deleting-28-000-lines-
of-
javascript&bav=on.2,or.r_cp.r_qf.&ech=1&psi=t9JQVPqgIYn1ONu8gYgJ.1414582968361.3&ei=t9JQVPqgIYn1ONu8gYgJ&emsg=NCSR&noj=1)

~~~
peeyek
Thanks!

~~~
richardkmichael
Commented above:
[https://news.ycombinator.com/item?id=8526930](https://news.ycombinator.com/item?id=8526930)

------
cnp
I know its difficult to admit mistakes in your own libraries, but I'm curious
if anyone at Shopify had considered switching to say Ember or Angular or React
before the rebuild?

~~~
AYBABTME
Yes, it's also mentioned in the article.

------
aikah
So AFAIK they send html back to the client right? that's my understanding of
turbolinks

The problem with this is that it makes client and server tightly coupled.One
cannot update the client app without touching the server code.

I understand that's a tradeoff but I think a restful architecture serving only
json/xml data is better. And you dont have to duplicate logic that much.If any
validation needs to happen clientside,create some validation resource for each
model and do it entirely serverside for instance. Even SPAs dont need fat
clientside models or a complex service layer on the client.

~~~
matthewmacleod
I don't know that what you've identified there is actually a problem in
practice – I'd wager that in a system like Shopify's admin, the client and
server are inherently coupled regardless of what architecture you choose.
Trying to abstract to a client-side app and JSON API in these situations often
results in more complexity for few benefits.

~~~
wvanbergen
This. We ended up creating models for the exact same stuff in both Ruby and
JS. Lots of code duplication, and losing client-side performance due to
deserializing JSON all the time.

------
jbergens
I think it is great that they hade the courage to redo something that wasn't
working for them anymore.

