
Farewell to Regular Web Development Approaches - hunvreus
http://devo.ps/blog/2013/01/31/farewell-to-regular-web-development-approaches.html
======
jacques_chester
Everything old is new again.

This used to be called Service-Oriented Architecture.

Before that, client-server computing.

Before that, online terminals.

No, it's not exactly the same. History in our industry, as Mark Twain said,
doesn't repeat. But it sure does _rhyme_.

My main concern is that people have a problem with Completely Legitimate
Architecture X, and then stumble on Completely Legitimate Architecture Y. What
was a nightmare in CLA-X is a cakewalk in CLA-Y.

Then come the blog posts (previously the CACM articles). The books. The
conferences. The whole grand crusade springs to life and promises to evict the
horrid evils of CLA-X, now that the scales have fallen from our eyes.

Then after a while, CLA-Y becomes the norm and CLA-X fades into infinite
obscurity. CLA-Y is unquestioned, unexamined. Just how things are done. Why?
_Because_.

Then one day a bright programmer notices that if they rearrange the pieces
like _so_ , they solve a number of well known problems with CLA-Y. Not
improve, not ameliorate. Solve!

And somewhere, a 32-year-old curmudgeon called Jacques Chester begins to write
a long post about how this is actually just CLA-X, with a different name ...

~~~
andrzejkrzywda
It may help, if we focus on some specific examples of CLA-X and CLA-Y.

Rendering html on the backend was the only option for years, as the clients
didn't have the computation power. Now, we're getting to the point, where
rendering client-side becomes possible.

We're not rendering html client-side for fun. It makes the UI more responsive,
while keeping the overall architecture more simple. There's a cleaner
separation between the client and the server.

We can (and we do) have separate teams working on client/server. It wasn't
that easy before.

I'm 33, if that makes my comment more legit ;)

~~~
drdaeman
> clients didn't have the computation power

I highly doubt so.

In early 2000s desktop browsers were already quite powerful, ran Flash applets
and had XMLHTTPRequest, so client-side webapps were perfectly technologically
possible. Yet almost everyone generated HTML on the server side.

I even remember one webforum using pure XML "pages" and XSLT on client-side to
build HTML. It's just that this overall approach (client-side apps, not XSLT)
wasn't popular for some reason, until recently.

~~~
blowski
A few reasons:

1\. Because about 90% of web users were on awful Microsoft browsers which was
extremely slow at processing JavaScript, and made a very bad job of rendering
CSS. Now, 80-90% of users are on a modern browser.

2\. Google's methods of indexing and scoring content changed - it now parses
JavaScript, gives value to speedy pages, is good at detecting keyword
stuffing, etc.

3\. Always-on internet is now pretty much a given, even on phones.

4\. There are more developers, trying more things, sharing more experience.

5\. Tools. In 2006, we had Drupal, Wordpress, Plone, and a few other odds and
ends. These days, we have Varnish, nginx, Jekyll, CloudFront, MongoDB, etc.
There are a lot more options, without having to spend a fortune or build
everything yourself.

~~~
ams6110
Microsoft browsers were the first ones that could do AJAX.

~~~
dgabriel
Indeed. The first AJAX development I did in the early/mid-2000s was IE-only
because the other browsers didn't have the correct support.

~~~
blowski
Yeah, this wasn't Microsoft bashing. Just pointing out that, for whatever
reason, the state of browsers during that period limited what could be done
with static HTML and AJAX.

------
DanielBMarkham
I had a blast building hn-books (<http://hn-books.com>)

All static. I could have deployed the app on a thumb-drive. And it had
advanced searching, a book listing and recommendation system, a fully-
configurable countdown clock for your startup, and a few other things. It got
to the point that whatever I could imagine, I could imagine building it as a
static site. And I would prefer doing so.

This is truly separating the view from the data. The same back-end services
(for hn-books I didn't even have a back-end!) could plug into zillions of
variously-looking front-ends. And as it turned out, the Query String was a
heckuva lot more useful than I imagined when I started.

Neat stuff.

ADD: One more thing. If, like most folks, you have a class of data that
doesn't change very often, say lookup tables or in hn-books' case the actual
database, there's no reason you can't deploy _that_ information as a static
file as well. You can pull JSON just as easily from a static file somewhere as
as server, and it uses a lot less resources. For my newspaper23 site, I used a
combination: 4 times an hour I rebuild the database and deploy as a JSON. Then
the clients all just pull from that file. When combined with a CDN this has
tremendous scaling potential. And it's very simple to do. Why thrash the
database every time somebody visits your site when it's just returning the
same stuff?

~~~
hunvreus
I second the "ADD": Development Seed did something very similar on
[http://developmentseed.org/blog/2012/nov/29/open-undp-
launch...](http://developmentseed.org/blog/2012/nov/29/open-undp-launches/)
and my friends at Wiredcraft are thinking about the same thing for a couple
visualization websites (lots of map, lots of data, low amount of edits).

Jekyll can be pretty damn slow though (especially on Github pages) so be
careful with the size of your data set.

------
davidw
It sounds like the big jump here might not have been so much moving to
Javascript/frontend-based apps, as moving away from a CMS towards a model
where they write and control the whole app.

~~~
hunvreus
Interesting, I hadn't looked at things that way.

I think though that the control over the code base wasn't much of the issue:
the mental bloat it was bringing in our thought process though definitely was.
We many times solved problems following the underlying CMS' patterns, and that
ended up eating more and more of our resources.

IMHO, our main gain here is the fact that we can focus back on the end user
and the story we're trying to tell through our apps.

------
jsiarto
Every time I see posts like this, I come back to warpspire's "Dumb Software:"
<http://warpspire.com/posts/dumb-software/>

"But wait until you see the light! Software today is intelligent! Asynchronous
front ends generated from esoteric scripting languages running on clusters of
distributed virtual machines around the globe! Death to repetition! It’s more
productive! It’s scalable! It’s fault tolerant! I followed the light and it
only ended in darkness."

I also don't understand comments like "we can make drastic changes in the UI
without any change on the backend"

In Wordpress or a Rails App, I can change the entire UI without having to
touch any "backend" code (controllers, models, functions, etc). Am I missing
something here?

~~~
ams6110
_we can make drastic changes in the UI without any change on the backend_

This has always been possible with well-architectected systems, but more
importantly it's mostly a platitude. It just doesn't happen in real life.
"Drastic changes" to the UI are almost always driven by changes in
requirements and hence they impact the entire system.

------
obituary_latte
If their mobile site is any indication, were in for a rough ride.

Of course there may be more pressing issues mentioned in the article, but I
couldn't read it as 20% of every single line is cut off.

I really think it's important tha

and approach. When it isn't consi

class. After all, there are only

size.

~~~
hunvreus
We put it out there without checking on mobile devices: it's fixed. Our
thoughts were not much about mobile in particular but the overall approach of
using static clients and APIs in the backend.

------
WA
I have a hard time following this discussion and would be glad if someone
could clarify a few aspects.

1\. What is a "static client" in this context? From my understanding, a static
client is a non-JS web browser that does one thing: render HTML code that is
composed and served by the server.

2\. People here talk about "(not) rendering HTML on the server". What's that
supposed to be? I thought "rendering HTML" means that you take HTML and
display it on a screen, which happens ALWAYS in the client and never on the
server.

Anyone care to explain?

~~~
felideon
(OT: wow I just learned the hard way that in Chrome C-k kills the rest of
input at point and cannot be undone. Also, C-a goes to the beginning of the
input, rather than beginning of the line, so my C-a C-k habit made me lose my
typed-up comment twice already.)

Answering your second question sheds light on the first.

A better term would be "generating" HTML on the server — the browser is
obviously in charge of rendering. In Rails for example, you can have ruby code
in an ERB template which the framework compiles to HTML. Hence, the HTML is
"dynamically" generated (on every request).

The opposite of dynamic is "static" where the HTML is either written by hand
or generated just once (like Jekyll) rather than on every request. In either
case, as a sibling comment mentions, an actual HTML file is served. However,
thanks to JavaScript development maturing and JS engines getting faster, a lot
of functionality that used to be done mainly server-side can be relegated to
the client.

I personally wouldn't use the phrase "static client" much, though.

(Edit: Problem with "static client". To me a 'static' site means that it has
no communication with a database, such as marketing sites or statically-
generated blogs. 'Client' is too ambiguous as it's not necessarily a browser.
curl is an HTTP client, for instance, and a JavaScript-driven "single-page
application" would be pretty limited via curl.)

~~~
jfoutz
(OT reply) on the mac, at least, you can yank back your kill with C-y, no full
kill ring however, just one item.

~~~
felideon
Hmm, C-y does yank something, but the last killed line. I'm on Snow Leopard,
though. :/

Someone also suggested C-z for undo, but no luck either.

------
drdaeman
I wonder what were the _reasons_ to actually change the whole
stack/architecture and start to re-invent the world.

There are tons "traditional" frameworks that are mature, proven to be reliable
and are full of features.

I've tried about a dozen of both various so-called "RESTful API" (server-side)
and webapp frameworks (client-side), and my overall impression is things are
far from being mature. Primitive todo-list apps are easy, but more complex
things quickly run into lack of features, documentation or just bugs.

~~~
PommeDeTerre
A lot of those problems you mention are due to JavaScript realistically being
the only language available for client-side web development. It's just not a
good language for anything but the shortest of scripts.

It lacks a proper type system. It lacks proper support for namespaces and
modularity. It is full of far too many inexcusable flaws ("quirks", as its
advocates like to call them), even in such basic functionality as its equality
comparisons. It's full of features that are outright harmful. Prototype-based
OO has proven, in practice, to be inferior to the more traditional class-based
OO approach. It's just one problem after another when using JavaScript.

CoffeeScript and other languages that "compile" down to JavaScript may help
somewhat, but their JavaScript legacy is still generally quite apparent.
Instead of being "good", they're just "not as outright horrible" as
JavaScript.

Things would be much better if there were a wider variety of languages
supported by browsers, including ones with the feature set necessary for
effective, large-scale software development.

~~~
rbrcurtis
With commonJS javascript is a perfectly good language to write real, large
code base apps in. With coffeescript, it's a pleasure.

~~~
drdaeman
I've recently used Require.js to load JS libraries (client-side) and had more
problems (working around libraries that know nothing about R.JS/AMD), than
benefits. Not sure how it is with CommonJS.

IMHO, when compared to asset compilers in many modern frameworks, the profits
are disputable.

------
boothead
I've noticed this as well (and there was a post from Tarek Ziadé's blog a
couple of days that mentioned it too) [1]

I've evolved to this approach - with the addition of WebSockets and a data
binding layer (knockout) - to building the infrastructure for an algorithmic
hedge fund over the last couple of years and it works extremely well.

Your backend components are free to do whatever they need to (in whatever
language/technology) and publish data periodically. The web server simply dips
into these streams of updates and transforms and routes them to the front end
as needed. It's an extremely flexible and agile way to develop applications, I
can't recommend it enough! In fact it's so nice to develop this way that I'm
writing a book about the ideas and techniques I've used which I hope to post
to HN in a couple of weeks when I've got the infrastructure and homepage up
and running.

[1] Post: [http://blog.ziade.org/2013/01/25/a-new-development-era-
essay...](http://blog.ziade.org/2013/01/25/a-new-development-era-essay/)
Discussion: <http://news.ycombinator.com/item?id=5120020>

------
andrzejkrzywda
It's both simple and revolutionary.

It starts with saying "well, let's just render html on the client-side instead
of on the backend". Suddenly you're in a new world of programming.

~~~
tluyben2
How about Google SEO? If content is pulled via AJAX and rendered in the
client; what does Google see?

~~~
steeve
One approach is to use PhantomJS as your app server (much like you would use
Django, Rails or something else) to render HTML for the crawlers.

See <http://github.com/steeve/angular-seo>

~~~
crucialfelix
that's just crazy enough that it might work. have you run this in production ?

it renders each fragment to a file and serves that to the bots, right ?

~~~
steeve
it does, I'm doing a version for html5modes angular apps (slightly more
complicated!)

careful to only serve these pages to bots tho

------
dougk16
I do think this is the immediate future, especially as we utilize typed
languages that compile to JS in order to create more complex applications than
are feasible with JS alone.

I also think that there will be a swing _back_ to an almost complete server-
centric model after that, where all logic _and_ rendering will take place on
the server and be streamed to extremely dumb clients...like TVs with basic
touch input. This will require a pretty intense infrastructure, but with some
nice perks for hardware/software developers. For one, absolute control over an
application...bump versions any time you like, easy access to debugging
information (crash reports, etc.), watch user usage in real time, on and on.
Hardware will also be insanely cheap. Users benefit from the practical
equivalent of having a top-of-the-line computer embedded in their smartphones.

------
danso
So...where's the examples. If the OP works with a lot of non-commercial
entities, how does a data API trump the client just being able to create posts
and static pages?

Most humanitarian groups want an easy to use CMS with some features, like
donation modules, added in. Is the OP building out the content editors too?
How is that not a CMS?

I've yet to work with a humanitarian group for which Wordpress was not more
than they could handle on their own. When something breaks in their new-wave
architecture, how day is it for them to fix on their own?

Edit: I realize the OP is talking about web apps, not just websites...WP is
likely not a great option then. But that's why I'm asking for some examples of
what they've done and why it justifies a "farewell" to old web dev

~~~
hunvreus
I believe I linked to our work in Sudan (wiredcraft.com/work/southern-sudan-
referendum/index.html). We've also done stuff for the UN
(<http://reliefweb.int>), the World Bank, WRI (<http://brtdata.org>) and a
whole range of other things my colleagues at Wiredcraft will be happy to
mention.

I still think there are relevant use cases for using a CMS. However, from
experience I think that:

1\. In many cases, CMS like Drupal or Wordpress are seen as the silver bullet
for everything and anything; "Why use a 3rd party? Just built it in the CMS!".
That's how you end up with expensive, tightly coupled platforms that are
effectively locking these organisations in.

2\. Many organizations are just engaging in some kind of arms race, copying
what their neighbors and competitors have.

~~~
danso
Yes sorry, you caught me in my "reflexively-check-HN-just-as-I-wake-up-
groggily" mood...Thank you for the links.

I completely agree with you that CMS is seen as an intractable silver bullet.
However, I think it is too extreme to say that web development, as we know it,
is out the window. While Wordpress is not the silver bullet, the ability to
graft on 3rd-party APIs and client-side apps (even something like
Disqus/Facebook comments) makes knowing just Wordpress _still_ incredibly
useful, for clients who do not need bespoke data solutions.

But even with a client who needs data-resources, it seems like Wordpress could
still play a key part by being the HTML page server with javascript includes
to an external data app...deciding to build a new framework just to accomodate
unique data-resources can still be overkill.

~~~
hunvreus
Right. From the post:

"It’s not that we fundamentally disliked the previous approach. We just
reached a point where we felt our goals weren’t best served by this model.
With lots of dynamic data, complex visualizations and a set of “static”
interfaces, the traditional way was hindering our development speed and our
ability to experiment."

It's very much targeted to our specific needs. Our approach is no more
universal than Wordpress or Drupal. We have however harvested interesting
conclusions from moving away from these CMS and that's mostly what I was
interested in sharing.

With regards to using Worpress as a middle man for data centric apps, this is
the kind of things we've done. My opinion: CMS are good at doing regular type
of content CRUD based apps, but it breaks quickly. Moreover, most popular CMS
are becoming bloated pretty quickly.

Have a look at Jekyll (<http://github.com/mojombo/jekyll>), it's a much better
fit for that.

~~~
danso
Yes, I love Jekyll and use it (with Octopress) myself...however, how easy is
it for non-developers to use? I mean, I _know_ it's easy...but even when non-
developers can figure out HTML (nevermind Markdown), they are still more
comfortable with a rich-text interface (until they try to embed code, of
course...)

~~~
hunvreus
We have strong opinions on WYSIWYG and rich-text too: Avoid WYSIWYG Editors
(<http://wiredcraft.com/blog/avoid-wysiwyg-editors/index.html>).

------
Osiris
The product I'm working on right now is taking this approach. We're upgrading
a legacy system. In doing so, we're writing a REST API and the components that
we replace are all built to use the API rather than the old legacy data layer.

The goal to to completely replace the application with an API and several UIs,
including a full web UI, a mobile UI, and also 'embeddable' UIs.

The API also serves as an integration point for other products in our suite.
That allows us to scrape a dozen different SOAP services that are targeted to
specific integrations and have those other products all utilize a single API
that can do anything that the full product can do.

I'm really happy with this architecture.

------
steeve
Next thing you know, we'll install JS apps just as desktop apps!

...wait a minute.

------
Kiro
What about security? How do you authenticate users?

~~~
zanny
You can still preserve session cookies over only a JSON exchange. I recently
had to deal with this in an android app syncing with a backend server.

------
why-el
One small issue dev friends I talk to still face is pricing and the need to
convince clients that a move to a ruby or js based tech stack vs Wordpress is
worth it. In the country where I live, wordpress is far, far cheaper to host
than anything else.

~~~
jonascopenhagen
Static HTML apps + REST APIs aren't dependant on Ruby/JS at all. You could
easily use a REST-centric PHP framework such as
<http://www.slimframework.com/> as the backend. Same hosting price as
WordPress. On top of that, Slim Framework is great to work with and is fast to
set up compared to e.g. Sinatra or Node.

~~~
why-el
I am sure they aren't and I did not mean to suggest they are. When it comes to
what backend to hook your static stuff to, it's just a matter of personal
preference. All I am saying is that its harder in some parts of the world
where Wordpress is still looked at as the cheapest, most viable option.

~~~
wyck
But you can stick a REST API onto WordPress. Several plugins have a JSON API,
there is Hookpress, it's also not very hard to roll your own restful code. It
also has XML-RPC Support which is an alternative to REST.

It's actually quite simple to provide "static" outs for WordPress and the
benefit is the client gets a nice easy to use admin. It's not the best thing
in the world or anything, but it works.

------
yycom
That's not static. That's client-side dynamic.

Words have broadly agreed meaning, despite what you or Humpty Dumpty may
think.

------
dos1
Client side rendering and single page applications are not the best solution
for most applications. I posit that a partial page refresh approach using PJAX
(or something like it) where the server always sends back HTML is a more
efficient and rapid approach to development.

There was an article yesterday about how Airbnb was going through some serious
gymnastics to get a fully rendered HTML page on initial request and then keep
it interactive and client side rendered after that. Why bother? Just send back
HTML all the time.

I worked on a very large (100+ developer) application that was fully bought in
on Backbone + JSON api consumption. In the few cases where there was a lot of
client interactivity (editable grids and drag and drop, most notably) the
client side rendering approach made sense. For 90 percent of the application
though, partial rendering of HTML from the server would have made things
easier and less complicated (and faster!) The initial time to usable page is
MUCH longer when you render static content, then make an ajax call to get the
data.

~~~
amalag
It was interesting a few years ago that Twitter went from server rendered HTML
to client rendered HTML back to server rendered HTML (current).

