
The sad state of web app deployment - cespare
http://eev.ee/blog/2015/09/17/the-sad-state-of-web-app-deployment/
======
wpietri
As another old-school type, I really enjoy this rant. And my understanding is
that the piece of software in question is a nest of snakes, so I can well
believe that there is no good way to install it, only bad ways and worse ways.

But having tried out Docker for some production deployments, I think that it
(or some less goofy successor) is the way forward. You get a sealed box with
all the necessary dependencies, and the box has just enough holes poked in it
that you can connect up the bits you care about. It turns apps into
appliances, not things you need expert craftspeople to lovingly install.

As much as I have enjoyed 25 years of doing everything on an OS whose
conceptual model is a 70s university department minicomputer, this era is
coming to a close. We already know it's a poor match for existing hardware and
use cases because we now mostly run it in virtual servers. But "virtual
server" is the new "horseless carriage". It's a phrase that tells us we are
living in a future we don't yet fully understand.

~~~
csirac2
I love Docker; it's solved a lot of problems for me. But this article
highlights the laziness that it can enable: there has to be a middle ground
between traditional package management and all of the curation/QA that goes
into getting stable releases out to multiple distributions, versus chucking
shit over the wall into a git repo with a Dockerfile and calling it done.

It's a better, more isolated mess - but for anyone trying to enforce
configuration policy into all of the running services in their environment,
untangling gortesquely basic shit like not granting superuser privs on a
database to your webapps - which would never fly in a traditional distro
package - becomes even more work than the old source tarballs with
INSTALL.txt.

~~~
jahnu
> the laziness that it can enable

This is the entire value proposition of all successful technology. Dubious
business practices notwithstanding, all successful yet imperfect technology is
successful because more people found it let them be lazier than the
competition. But in reality of course it's not laziness, it's efficiency. They
can get on with other tasks sooner than before.

~~~
csirac2
> But in reality of course it's not laziness, it's efficiency. They can get on
> with other tasks sooner than before.

Indeed, and I use Docker myself to improve my own efficiency, and I've seen
great stuff built with Docker that has been architected well.

However, it is painfully obvious that without the pressures that used to force
most developers to keep their shit sane, there's now more workload for serious
users that actually need to untangle this mess in order to support, secure and
get stuff deployed.

------
quaunaut
Wow. This is ridiculous.

"I've got a weird set up OS for a reason, but this software should work even
though no one else would ever run this."

"Whatever- this database should work, even though I'm using a 4 year old
version of the free software(that the docs specifically say needs to be a
newer version)."

"Also, I'm extremely inexperienced installing Rails and Rails apps, and
despite the fact that it's a language and server that we literally teach the
newest of the new, it's just impossible to do anything with."

This isn't a story about the state of web app deployment, this is about the
state of the server from hell. This sounds like the kind of thing you hear out
of megacorps with 20 year old Mainframes.

Should open source be supporting such convoluted, needlessly minor cases of
awful environments?

~~~
eevee
Yes, my OS is slightly "weird" for reasons I did not choose. Funny story:
while I was trying to figure out what the hell RVM was doing, I mostly ran
across people running OS X who had the same setup: 32-bit OS on a 64-bit chip.
(And for what it's worth, Python native extensions build for the architecture
of the Python executable — you know, the thing that has to load them — rather
than the architecture of the machine.)

I never complained that it didn't work with an old database. Ubuntu's
versioning was just a fun surprise. I don't see why it _wouldn 't_ work with
Postgres 9.1; I just figured I'd do the upgrade while I already had my foot in
the door.

Maybe if installing Rails apps is still a huge headache after the fourth or
fifth time I've done it, something is wrong with Rails. Maybe.

I have a fairly mundane server running the latest Ubuntu LTS with all stock
vendor packages. If your app is such brittle crap that _this_ is the "server
from hell", well, it's no wonder everyone is using Docker.

~~~
justincormack
32 bit x86 setups are very rare now under Linux, and you are really on your
own - people no longer test with them and some software no longer supports 32
bit (eg Docker; I gather it will go multi architecture soon but I doubt much
stuff will be built or tested on 32 bit x86 anyway). As your hardware can run
64 bit you have no reason to run 32 bit.

~~~
eevee
I hope you can imagine why I might not want to reinstall the OS and have to
redeploy half a dozen other things, just so this particular one gets easier.

~~~
justincormack
Because you are managing your servers as pets not cattle? Because you did the
installs manually and can't replicate them? Thats what you need to fix, with
automated deployments that is a non issue.

------
pliu
The tone of this article is very upsetting. It begins with "I like to think
I’m pretty alright at computers", but frankly I don't really agree that this
is the case. Good at programming, maybe, but clearly not very good at ops.

Rather than accept that there are skills and experience they don't have, the
author instead chooses to blame the entire world for their ignorance. The
frustration on display is understandable and real and I totally get it, but
the attitude is less forgivable.

Faced with a problem, instead stopping for a hot second to read some
documentation, the author instead concludes that Docker is garbage, does some
insane shit and then claims that the software industry has failed them. What
incredible arrogance.

This article says way more about the character of this person than it does
about the state of web app deployment.

~~~
jodrellblank
Your reaction to "server software deployment could be better" is to scoff at
the problems and ad-hom the author.

[http://i.imgur.com/SMPYGEU.jpg](http://i.imgur.com/SMPYGEU.jpg)

~~~
dikaiosune
The author's reaction to "we've decided to try and simplify deployment of our
hideously complex app" is to ignore the supported deployment, bury themselves
neck deep in the guts of the complex deployment the devs worked to abstract
away, and then rage about it. I'm not really seeing "server software
deployment could be better" as a fair read of that.

~~~
jodrellblank
His point is not "I don't know how to deploy a Rails app", it's "there is no
standard way for a server program to tell the OS how what services to
configure, or to query the OS for what services are available. Instead of
fixing this, we wrap a pre-configured-OS-image and run it inside our OS. This
is daft". As well as "the instructions are incorrect and incomplete, and
that's normal".

e.g. _Set aside the oddball tool breakage and consider that if you follow the
instructions to the letter, this web forum requires: cloning (not installing!)
the software’s source code and modifying it in-place. Copy-pasting hundreds of
lines of configuration into nginx, as root, and hoping it doesn’t change when
you upgrade. Installing non-default Postgres extensions, as root. Running
someone else’s arbitrary database commands as a superuser._

and

 _You’d think that a web app could just have some metadata saying “I need
Postgres and, optionally, Redis”, but this doesn’t exist. And the other side,
where the system can enumerate the services it has available for a user,
similarly doesn’t exist. So there’s no chance of discovery. If you’re missing
a service the app needs but failed to document, or you set it up wrong, you’ll
just find out on the first request._

and

"What we should have by now" section.

and

 _And I’m really not picking on Ruby, or Rails, or this particular app. I hate
deploying my own web software, because there are so many parts all over the
system that only barely know about each other, but if any of them fail then
the whole shebang stops working._

~~~
krisdol
Service discovery, orchestration, and on-the-fly config/environment changes
are things that are solved by tools like consul and mesos (to scratch the
surface). The ops world is so much more than just dockerfiles. There is
absolutely a "chance of discovery". Using consul has helped us move to zero-
configuration, discoverable microservices. Then for the more hobbyist stuff,
there's docker-compose. Not looking into the options and ranting is not the
same as there not being any options.

------
cwyers
> See, I actually have a 64-bit kernel, but a 32-bit userspace. (There’s a
> perfectly good reason for this.)

Well, maybe there is, but a lot of this article seems like "the sad state of
nobody has this web app set up to be installed on my frankenlinux."

------
PlzSnow
I think Discourse is an over-engineered catastrophe, a prime example of
highly-skilled technical people unable to make business decisions. As the
author says, this a web-forum software, all it is doing is displaying some
text on a page and maybe sending an email. This was a solved problem 15 years
ago with PHP. But they wanted to re-implement everything in a technically-
purist manner, and end up with the absurd situation where you need to install
a virtual container to perform one of the most basic computing functions there
is: displaying text on a page. And that's just the server-side, the client
rendering is so over-engineered that it only works on the very latest
browsers. TO DISPLAY TEXT! Completely bonkers. They needed a "user-experience
evangelist" or whatever you call them on that team.

~~~
nstart
> As the author says, this a web-forum software, all it is doing is displaying
> some text on a page and maybe sending an email. This was a solved problem 15
> years ago with PHP

This reminds me of what Jeff Atwood had to say about people over simplifying
the nature of stack overflow (and by extension, claiming it to be over
valued).

He eventually responded about the fact that the little details mattered. From
auto searching for existing similar questions to little prompts on framing the
question.

Similarly, Discourse is being built with principles in mind that go beyond a
threaded discussion. It's attempting to answer the question of "can we use
software to encourage a civilized discourse?". It's easy to ignore this, and
look at it as bloated. But in that vein, Stack overflow is a bloated Yahoo
answers or any other. And we all know it isn't. Essentially, it's more than
just "a web-forum software". That distinction really matters.

Relevant links:

1: [http://blog.codinghorror.com/this-is-all-your-app-is-a-
colle...](http://blog.codinghorror.com/this-is-all-your-app-is-a-collection-
of-tiny-details/) 2: [http://blog.codinghorror.com/code-its-
trivial/](http://blog.codinghorror.com/code-its-trivial/) 3:
[https://news.ycombinator.com/item?id=678501](https://news.ycombinator.com/item?id=678501)

~~~
dikaiosune
> "can we use software to encourage a civilized discourse?"

After carpet bombing this thread with my own criticism, this was a good
reminder of why that was probably stupid. Thanks :).

------
mkozlows
This is a weak article. The author's never worked with Docker before, so got
tripped up by a few gotchas (but seriously, installing a third-party PPA isn't
that hard or weird), but it doesn't take that much googling or trial/error to
figure out how to do it right.

And the point of that is, once you know Docker, now installing just about any
set of dependencies becomes a skill you know. I've never worked with
Discourse. I've never worked with Rails. But because I've done Docker-based
deployments, I bet I could indeed get it up and working in 30 minutes.

~~~
pki
I've used docker before and my install was to a clean ubuntu 14 64bit -
discourse installed + up and running in about 3.5 minutes. I have never worked
with Rails or Ruby, or had to touch rvm for this case.

Incidentally, my hosting machine is basically clean - I don't use it as a
personal or dev box, it does not have a ton of random userspace stuff in
32bit, it just hosts things.

------
markbnj
Ok, I get it. I've felt like writing this article many times. I didn't, but I
give the author props for letting off some steam in a constructive way.
However... this statement drove me a little bonkers...

>> The 30-minute claim came because the software only officially supports
being installed via Docker, the shiny new container gizmo that everyone loves
because it’s shiny and new, which already set off some red flags. I managed to
install an entire interoperating desktop environment without needing Docker
for any of it, but a web forum is so complex that it needs its own
quasivirtualized OS? Hmm.

I understand the reflexive dismissal of things that become popular topics. I'm
guilty of it myself repeatedly, most recently with "microservices." But
sometimes technologies become popular because they are good and useful.

I haven't run into Docker's 32-bit install problem. They should fix that. But
to dismiss the whole technology as some sort of obviously useless quasi-
virtualized OS mumbo jumbo is taking the rant too far. Dependency isolation is
a good thing. Deploying server applications with one command is a good thing.
Knowing that your runtime environment is always the same is a good thing.
Having a simple source controlled script that completely describes that
runtime environment is a good thing. Some people try to use containers for
unreasonably complicated things, or to hide unreasonably complicated software,
but that is not an indictment of the technology. It's popular for a reason.

~~~
csirac2
As a happy Docker user with a lot of respect for what Docker has achieved,
what concerns me is that Docker is clearly an enabler for people to totally
abandon proper release and dependency management along with sane sysadmin
friendly configuration.

I can't tell you the number of times I've hoped to use a public docker image
or just the Dockerfile, only to spend hours futzing with it because I was
unhappy with the grotesquely insecure configuration or because I needed to
work around a bunch of assumptions that are invalid once I've tuned the
Dockerfile for my environment.

~~~
justincormack
Fix the insecurities upstream or dont use the software. Else upgrades will be
a nightmare too.

~~~
csirac2
I agree, and I do (if I end up actually keeping a piece of software running).

It's also quite likely that a pre-built docker image isn't going to satisfy
everyone's deployment requirements.

Starting out with a philosophy that you'll make all the configuration and
security choices for every user of your project isn't a great start, and
that's how I read all these hard-coded assumptions.

------
sytse
GitLab is the other large open source Rails project. We choose to package
below the container level with Omnibus packages. These run without all the
Docker dependencies and install in 2 minutes. We did this because many of our
users could not run a recent/custom kernel with Docker. We're very happy with
the choice and are able to run the Omnibus packages on top of our images
[https://gitlab.com/gitlab-org/gitlab-
ce/tree/master/docker](https://gitlab.com/gitlab-org/gitlab-
ce/tree/master/docker)

That being said, it is still a bit silly how hard a Rails app is to install.

------
dheera
There actually is a point. I'm not a rails developer but as a Python developer
I can definitely say deploying applications is far more difficult than it
should be. To deploy a simple Flask app as FastCGI I had to download some
nasty .fcgi file from MIT's 6.170 course website to detect code changes and
reload the app. To deploy with nginx+uwsgi I spent ages configuring it and
with poor documentation.

Seriously, things like uwsgi should have an "auto-configure" feature where
when an error is encountered with a dependency, it searches the hell out of
your /usr filesystem and caches the resulting configuration. If a module is
missing, automatically try to apt-get and pip install it. Nginx should
portscan the hell out of localhost and detect uwsgi servers. Install uwsgi if
a wsgi app is detected but uwsgi isn't installed. Write the .ini file
automatically. This process is so automatable that there really should be a
"magic deploy" feature where I can just drop a Flask app as
/var/www/html/some_app/index.py and it should be instantly up and running at
[http://localhost/some_app/](http://localhost/some_app/) with zero questions
asked.

------
3princip
Nice article, I can empathize having recently inherited a PHP project which
included a puphpet (vagrant/puppet) VM. It wasn't touched for a some time so
the configuration was a few months old.

After installing the required dependencies (downgrade VirtualBox etc) all that
was left was: vagrant up. My lord, what had I gotten myself into. Que problems
with the configuration, ruby version during provisioning, paths, puphpet had
gone through multiple breaking changes in the meantime, the documentation was
unhelpful. The only inspiration were GitHub issues on tangentially related
projects ... all I know is it was 3am having stared 6 hours earlier.

Then, in a moment of madness I deleted everything and created a completely
new/fresh puphpet configuration using the puphpet site. Again during
provisioning I was met with a problem, there are non-ascii characters in the
php-fpm upstart configuration file (the authors name!). Luckily, this was an
issue already discovered a couple of hours earlier, so a small change to that
and the app was up and running. It was 4am.

Needless to say, I was very frustrated with this state of affairs of these
supposed aids to setting up the dev environment. Granted, my mistake was going
in the wrong direction trying to fix an outdated configuration and all the
problems it generated rather than just generating the config again, but I had
never used this tool before, and hadn't looked at the PHP ecosystem for the
past few years, it seemed crazy at first that a config which worked in the
past, presumably, would not work on my very mundane Ubuntu dev environment.

~~~
mschuster91
Oh gosh, puphpet and vagrant. Had a similar architecture with a Silverstripe
framework recently inherited, it was well documented and all, but due to new
versions everything was broken.

I ended up doing deployment with git pull.

------
dasil003
I've been doing Rails now for _a decade_ , and I love ruby, but Rails is not
suitable for installable open source software. It is one of the worst
languages for that use case. The whole mentality of the Rails community runs
counter to the goals of providing easily deployable packages. There is
certainly a lot of low-hanging fruit to work on these issues, but I don't see
it becoming a priority anytime soon because if you value these things you're
probably already using some other language than ruby.

The sweet spot for Rails is custom apps that is continuously maintained over a
long period, or discardable prototypes. It is not a good choice for deploy-
and-forget, or organizations without any in-house programmers.

------
bhuga
I was going to come comment that there must be a good way to solve this, since
the Ruby community has such a rich ecosystem of tools.

Heroku, for example, has a nice one-click deploy button for Rails (and many
more languages/frameworks). It works straight from the source code, such as
with this open-source rails app, and it's really quite impressive:

[https://github.com/heroku/starboard#deploy-the-
app](https://github.com/heroku/starboard#deploy-the-app)

The author also calls out error reporting as being terrible. And there's also
great tools for managing that, such as newrelic and airbrake.

So surely this author was just unused to the tool ecosystem, I thought. What a
perfect opportunity for a constructive yet snarky comment! But lo and behold,
discourse has deprecated all non-docker installs:

[https://github.com/discourse/discourse/blob/master/docs/inst...](https://github.com/discourse/discourse/blob/master/docs/install-
HEROKU.md#warning-this-guide-is-deprecated)

I was, and am, completely baffled at that decision. And I learned a valuable
lesson about trying to out-snark snarky blog posts.

~~~
dikaiosune
Supporting multiple deployment options introduces a lot of overhead, and
Docker (in particular) seems really good at reducing deployment overhead for
both developers and admins. I'm not sure why it's a bad thing when it means
that the team can focus resources on a single canonical deployment method that
just works. Docker containers can also be run on a variety of public cloud
services, so while Heroku may not be available, pointing your Dockerfile at
EC2 or GCE shouldn't be too hard?

------
jpgvm
If you use 32bit Linux in 2015 you deserve the pain you just endured.

The problem is not the state of deployment (which is still admittedly quite
bad), it's the state of your system.

Install a fully 64bit version of 14.04 and watch all your problems just
disappear.

------
plaguuuuuu
Two things come to mind

1\. Tinfoil hat time! "Pisshorse" make money off of hosting their own
software. So they are financially incentivized to make it as difficult as
possible to install yourself, but at the same time they get to go "woooo, open
source" as much as they like.

I'm convinced Oracle did the same sort of thing by making their DB product
impossible for normal people to understand, in order to charge outrageously
expensive consulting fees to companies. Or so I hear; I've never used it so I
could be wrong.

2\. Your users aren't always who you think they are. I learned this one
switching from back-end/n-tier work over to front-end CMS based web dev. Yes,
the users are the people browsing the website, but the users are also the
people trying to use the damn CMS, so the website UX and design extends to
those people as well.

Meaning, instead of forcing your hapless marketers to use some crazy admin
panel with thousands of options and checkboxes, or even try and edit XML
configuration (I've actually seen this), any time one extends the
functionality of a CMS, creating some custom front-end UI to control it is a
basic necessity.

In the same vein, any sort of software (and hardware! printers, tools, cars,
whatever) has to consider the installation and maintenance of itself as a
UX/design concern and the fact that it has multiple domains of users.

~~~
eevee
I observe that they'll happily charge you $99 to install it on your machine
_for_ you.

~~~
riking
Nope, that's on a 64-bit Digital Ocean machine that we provision for you. The
only secret sauce in the $99 install is the Mandrill reseller code.

------
Myrmornis
Really, 90% of this article is an argument _for_ docker. His docker
installation annoyance was nothing compared to the pain ensuing from his
decision to chuck docker out of the window.

~~~
tajen
This. He says "We need to find a solution to install those apps with a dozen
dependencies, config files, crontabs and daemons". Well, although it may not
be mature security wise, this solution is still Docker.

------
ajhit406
I had the same trouble setting up discourse, so I setup a template on Nitrous
using Docker that you can definitely use to get Discourse up and running in 30
seconds. (I just confirmed, I went from no environment to running discourse in
less than 30 seconds). Just `cd code/discourse && ./start-app`.

This, IMO is where Docker shines. It shouldn't matter if it's setup with a
microservice 12-factor architecture or if everything is setup in a monolithic
VM-like container. I don't have the patience for ops -- I just want something
that works. That's the point of having isolated, replicable containers.

In any case, I encourage you to try out the discourse container on Nitrous. I
was actually surprised it happens to be the least popular container for us. I
assumed because it's such a pain in the ass to get started, that it would be
more popular =p

------
lewisl9029
So I actually remember running into the "Unable to locate package docker-
engine" issue a while ago, and it seemed like an issue on Docker's end because
for me, the issue actually only lasted about half a day before it started
working again without any changes on my part.

So I think in the end this was just a case of _really_ unfortunate timing,
because if it weren't for the Docker installation issue, the only real
complaint left in this post would have been the fact that the official method
of installing Docker was to curl a script and pipe into sh[1]. And the rest of
the post would have been singing praises of how amazing Docker is to be able
to take setting up a Rails app along with all its dependencies and turn it
into such a simple, painless process.

[1] Which is a perfectly valid criticism, by the way. They should really
document the much saner method of installing through their official repos:

[https://blog.docker.com/2015/07/new-apt-and-yum-
repos/](https://blog.docker.com/2015/07/new-apt-and-yum-repos/)

~~~
mkozlows
Yeah, the really weird thing about this article is that it's all snarky about
Docker, and then it goes off into weird manual-install land (which makes about
as much sense as saying "I couldn't get the .msi installer to run, so I
started copying files around and registering DLLs by hand" \-- maybe you'll
get that to work eventually, but it's never going to be pleasant), and then
makes a call for something that... solves the problems Docker solves.

The whole article is basically a "there has to be an easier way!" infomercial
for Docker, only it doesn't realize it.

~~~
Gigablah
It's funny because the project actually provides a Vagrantfile for creating a
regular VM image.

~~~
codinghorror
Vagrant's only intended as a development solution, though, not for production
deployment.

------
raspasov
I have to say that Docker used to be very annoying to work with on Mac OS X
but with the latest release of Docker Toolbox it has a much better "works out
of the box" experience.

The article is pretty ranty but I can agree with the author that many things
nowadays are way more complex than they have to be. As a plug, I'm going to
say that this is one of the reasons why we started the CloudMonkey.io project.
It lets you deploy a docker container with no fuss to production. It's up to
you, however, to ensure that you don't over complicate your system
unnecessarily.

I've done the mistake in the past where very early on in the project I started
using a web server, Redis, ElasticSearch, MySQL, memcached, RabbitMQ, etc.

In most cases, more than three moving pieces only bring you headaches. Now I
always try very hard to keep things simple to at most a web server and a
database, plus maybe a memcached caching layer. If you need to have a queue or
full text search functionality, I'd try to bring it in as an outside service.

------
mschuster91
What's most worrying with all that Docker bullshit is updating. We're going to
end up with physical hosts with dozens of fire-and-forget VMs on them and each
one filled with security holes.

------
reacweb
I agree with most of this rant, but not with this too common mith: "Only one
thing can bind to port 80 and it has to run as root". I generally use the
following command <pre>setcap 'cap_net_bind_service=+ep' /usr/bin/nodejs
</pre>

Just learned the trick to become root when you belong to docker group. Awesome

------
Jerry2
>"Let’s just say it rhymes with “piss horse”

I think he's talking about Discourse [1]. I tried Discourse few years back
when it was released but it was too bare-bones at that time. Haven't tried it
recently.

[1]:
[https://github.com/discourse/discourse](https://github.com/discourse/discourse)

~~~
voltagex_
I really don't understand why a forum needs 2GB of RAM.

~~~
codinghorror
It doesn't; 1GB is the minimum. We have to fit Redis, Postgres, Ruby, Sidekiq,
and 2 Unicorns in there. There are a bunch of fairly active instances on 1GB
RAM, which works fine. If your community grows a lot you'll need more
resources, of course.

------
amelius
Some other sad things, from the top of my head:

\- Besides a web app, we must also create mobile apps, and they usually don't
share much code. It would be much better if apps could just run HTML and
Javascript instead, and still be integrated like native apps are now.

\- Every web app that is deployed now is not guaranteed to work X days into
the future. In fact, it often breaks. We need better guarantees for this.

\- Browsers are incompatible in subtle and not-so-subtle ways.

\- It is way too complicated to write a web-browser. The number of
abstraction-levels combined is too large. Security problems are therefore
unavoidable.

\- On the front-end, React is a suboptimal solution because it requires O(N)
time and energy for simple O(1) updates. We need smarter incremental
computation techniques.

\- The primitives for the web (HTML+Javascript) have been created for novices,
instead of for developers. However, even novices can't read the W3c specs, so
it doesn't really help anybody.

------
gizi
I deploy the basics of the app as a debian package. Debian is very good at
dropping the files and folders where they belong.

There is also a very usable "postinst" installer shell script that can do
additional fixups. It can handle things like "if we are on openvz do this,
otherwise do that" or "if this is debian 7 (wheezy) fix this; if it is debian
8 (jessie) fix that".

If that is still not enough -- which it sometimes isn't for really complex
platforms -- I write an additional shell script that will do the remainder of
the work.

But then again, that probably means that the image will have too many moving
parts anyway. That means it is time to split the image and stuff a network
protocol between the parts. You can use virtual machines for that. The front-
end goes left, the back-end right, and other things go top and bottom. Now we
are back to more simple images to install and to test.

~~~
voltagex_
Where did you learn how to build .deb files? Every time I try it ends up being
horribly complex and I either go back to a shell script or something like
OpenSuse Build Service. Docker is great - when it works, but there's something
very wrong with deploying a whole new OS just because we can't work out
packaging and deployment properly.

~~~
lamby
Jumping in here, but I always suggest just packaging something pretty vanilla
outside of a web app due to all the aforementioned specifics and impedence
mismatches of deploying web apps.

Learning some of the innards of the package management system of your machines
is surely a good idea in itself, if only because you are relying on them so
much. Even if you just say "screw it" and use shell scripts to deploy anyway.

------
jscottmiller
My experience installing Discource was a breeze (aws instance, using external
postgres and redis servers). This included setting up a custom auth endpoint
in my app to allow federated login. I'm amazed that so much effort was made to
simplify this process given that their business model revolves around hosting.

------
LoSboccacc
Yeah everyone up in arm at maven complexity but then you start to use things
built by others and suddenly 'if only there was a better way'

There is, but the influx of hacker types into what's an engineering discipline
just reset what the field collectively knew to do (when was autoconf
introduced again?).

~~~
voltagex_
I agree. I've made an effort over the past little while to learn Makefiles
(after tinkering with waf, ninja, etc) and they can be really good if used
properly.

Hell, even Docker can work well if used properly. The secret here is to keep
your development machine clean and standardised - this is what Vagrant is for,
right?

------
thesnider
This article badly wants a thing that already exists -- sandstorm.io.

Done.

~~~
kyberias
Wow, this is a prime example of a web site not explaining what the hell the
software really does.

"Sandstorm is an open source platform for personal servers."

What is it?

~~~
losvedir
I'm a web developer, but I can't really use my skills to provide an open
source web _app_ the way I'd like to. I'd like to build a small server-side
budgeting app that people can use from their computers or phones to record
expenses, but there's no way I can ask people to find a web host that lets
them run rails, or set up a heroku account or whatever.

So my only alternative would be to run the service _myself_ , but then I'm
storing other people's data, I have to worry about scaling if lots of people
use it, and user accounts, and all this stuff.

The idea of sandstorm is folks run this platform on their personal servers,
and then it lets you browse an app store like interface and one-click install
these server side apps. So I'd bundle up my budgeting rails app as a sandstorm
package, and if someone wants to track their expenses from a variety of
devices, they install the app. Now they're running it so the data is theirs,
there's no scaling issues, and user authentication is provided by sandstorm.

Sandstorm is still in its infancy so there's not a lot of apps available and
the development APIs are being worked on, but I hope it's the future. It would
lead to a more decentralized web with better privacy and users owning their
own data.

I'm hopeful, if not optimistic, for a future where every family is expected to
have their own little server running somewhere. And they access that server
through the sandstorm web interface, and can easily add little apps to it. My
budgeting app, a webmail app, some future federated profile app to replace
facebook, etc.

~~~
kyberias
You know, a picture might really explain it better. If it's something that
sits on top of Linux and manages software installation, it's easy to
illustrate.

------
falsedan
A dev learns that deploying apps is hard, wonders why no one solved this
already, suggests writing more code will fix the problem.

~~~
eevee
Actually I suggest writing _less_ code, assuming that Docker is a whole lot of
code and "what if we write down dependencies" is more interface than code.

~~~
falsedan
Your hypothetical vision of the future sounded like you were suggesting
writing a tool/system to manage the complexity of dependencies/deployment. I'd
recommend changing the thing-to-be-deployed-itself, so it was simpler and had
fewer dependencies (if we're talking hypothetical code changes).

Aside: complex software is hard to deploy because it's complex. Software gets
complex because devs are very comfortable managing large amounts of
complexity. Docker lets you hide that complexity so someone other than the dev
can do the build/test/deploy steps, but it does nothing to address the high
level of complexity in the underlying software. Let's not mince words here:
Discouse is hard to deploy because Rails wasn't designed to make its projects
easy to deploy by someone other than the devs primarily working on it, and no
amount of tooling around it will change that.

------
ufmace
What I think is kinda odd about this rant, and about half of the comments
here, is the implicit assumption that web forum software must be simple and
dumb. "How could it possibly need X, it's just a forum dammit!" Why? Says who?
Do we have some "shit's easy" syndrome going on here?

Meanwhile, the users of the forum that I've seen, mostly at TDWTF, seem to
mostly complain how short of features and functionality it is/was compared to
other BBS solutions.

So which is it? Is it too complex, or not complex enough? It seems to be doing
all right at the current level of complexity. If you're suggesting a different
one, what makes you think that would be better? Better for who, exactly?

Also note that Discourse is trying to take business from a pile of well-
established competitors. To do that, you generally have to be not just as
good, but much better, and with flashier features too, for both the users and
the mods and admins.

~~~
blakeyrat
People at TDWTF don't complain about Discourse lacking features, they complain
about it being overwhelmingly full of bugs and brain-dead behaviors.

Particularly the mobile experience, which is supposed to be one of those
things Discourse was designed to do well.

------
jister
Did you know that in asp.net one can deploy apps with just xcopy? I mean this
is what the whole rant is all about, right?

~~~
jodrellblank
But you can't mandate a particular version of .Net, or configure SQL with SQL
Reporting, or configure add-on services, or request or enumerate firewall
rules, with just xcopy.

Which is what the whole rant is about.

~~~
jister
The rant is about basic web apps like a web forum in his case. your sample
about .net version is a no-brainer in shared hosting and sql reporting is not
for basic apps.

------
otterley
I feel like the real problem here is with the vendor. Had they produced and
tested their product against and packaged it for Ubuntu 14 (which is a very
common distribution), the user would have had a good experience.

Demanding your users manually install prerequisite software -- especially
dependencies that are already included with their OS distribution, and when
it's not clear that it's necessary -- is not good practice. And if newer
versions of such software are needed, that's what "vendoring" is for: you
package all the dependencies along with the software. Sure, it means you might
have a duplicate copy of Postgres or whatever, but at least it works out of
the box.

------
paulmd
Maven is fantastic at handling this complexity. It's a pain to initially
configure, but once you have it set up you can pretty much forget about it.
Similar to the author, my experiences with Gem and Bundler have been rather
painful due to a combination of things like Cygwin/Windows issues,
incompatibilities between Ruby versions, and 32/64 bit problems. Maven isn't
quite perfect, I'd really like the ability for dependencies to have their sub-
dependencies sandboxed in their own namespaces, I've run into some weird
quirks with load order and the API-only JavaEE packages, but once you get a
POM written it's totally reliable.

I bitch just as much as the next guy about the mistakes that have long since
been baked into the Java environment, but that dedication to backwards
compatibility does pay off. Similarly, the fact that it's running on a JVM
rather than bare metal has its performance costs, but it does allow you to
trust that the actual computation will Just Work wherever you run it (although
of course the stuff that touches the surrounding environment will not).

In the comments the author states that he wants something "less invasive" than
Docker. I'm not sure what that actually means - for me, having to actually
install it and deal with the versioning on my real system is more invasive.
Containerized applications that build their own runtime environments is a step
in the right direction there, just like the JVM. Docker has some really dumb
defaults right now, running as root has been known to be retarded since
forever and you should not encourage users to do it. But running as root is
problematic for lots of other reasons, and causes the potential for jailbreak
on other jails too (eg chroot). FreeBSD jails seem to have some good rules on
what you're allowed to do inside a jail.

If you really want "minimal invasiveness" and your software really really
requires that you have superuser privileges then you should probably be
running a full-fat VM with an OS inside, with the network sandboxing happening
at the host. Even then, it's not like even Xen hasn't had the occasional
jailbreak. The real problem is that the program itself should never require
running as root in the first place - but that's a problem with Discourse, not
a problem with jails.

------
jacques_chester
Here's how I deploy:

    
    
        cf push
    

It requires zero configuration on my part.

Actually, I'm lying. Here's how I deploy:

    
    
        git push
    

CI then verifies my work before running this arcane command:

    
    
        cf push
    

It took literally minutes to set up. It'd be nice if it was psychic but I
guess I'll just have to settle for an imperfect world.

------
riffic
Or you could have just installed a Bitnami image until you're willing to jump
through all the hoops necessary to get many multiple moving pieces to work
together happily.

Ops is hard dude. Sorry about that.

[https://wiki.bitnami.com/Applications/BitNami_Discourse](https://wiki.bitnami.com/Applications/BitNami_Discourse)

------
onetom
I find it interesting that no one has mentioned
[http://nixos.org](http://nixos.org) even here on HN...

Isn't it suppose to address most of those grieves the author was complaining
about (and I have also experienced so I have up on Ruby years ago and did some
nodejs but that's not really different either)?

------
tobbyb
We are trying to solve this problem with Flockport [1] so users can at least
get to see and try apps without needing to install and configure tons of stuff
and we use LXC containers which behave like lightweight VMs and provide an OS
environment similar to a VM or a bare metal server that users are more
familiar with. And this also allows us to package the stack and app in a
single container..

With Docker apart from the app you have to deal with the additional complexity
of a single app environment that is not an environment that most users are
familiar with, for instance you need to launch all your apps in 'non daemon
mode' (how many uses know how to do this, and why you would need it?) figure
out how to deal with storage persistence, networking and 'linking containers'
and things like logging, cron and ssh (there is no place for these in single
app containers). Installing something as simple as Mysql with Docker is non
trivial because of storage. So this adds a whole new layer of complexity on
top of the app that's makes it completely unsuitable to package apps if
simplicity and accessibility is the objective. You need to be a real expert to
deal with this, much more than LXC containers or VMs.

We have a guide on building a simple Wordpress stack from scratch with Docker
[2] that gives readers an understanding of the level of complexity involved
with Docker and what it seems to be designed for. It's not for end users and
certainty not to making things simple. It would take a tenth of the effort to
install Wordpress in a VM or LXC container. But its pointless to compare VMs
or LXC containers to Docker, as Docker is intended for a completely different
use case and that's what its users and advocates should highlight and push.

We have packaged over 60 apps in LXC containers at Flockport and Ruby apps are
by far the most complex to setup, usually requiring a complete build
environment. PHP is the simplest both to install and troubleshoot, and usually
just works. The Ruby apps like Redmine, Gitlab are more complex and Discourse
is the toughest to get going. You need to be an expert to install it and an
even bigger expert to run it so much so that I think unless you are a rails
developer the only reliable way to use Discourse is to have the Discourse team
manage it.

[1] [https://www.flockport.com](https://www.flockport.com)

[2] [https://www.flockport.com/a-beginners-guide-to-
docker](https://www.flockport.com/a-beginners-guide-to-docker)

------
chrisra
Corrected title: the sad state of one developer's experience with one web app
deployment.

------
daliwali
I think that the vast majority of web apps could be deployed via shell script
that calls out commands to the particular system. I'm not too optimistic about
Docker since it is vastly more complex than the alternative (VM + shell).

------
franzwong
If a web app is huge, it should have a plugin system. So you only need to get
a basic configuration for basic features (without complicated monitoring, full
text search).

Then you install more, configure more if you want something more advanced.

------
Asbostos
A lot of people are criticizing the author but I know much less about
installing things on Linux and quite quickly managed to install another, more
conventional forum without any of the problems he mentioned. I just copied the
files into a folder, created a database, ran an install script (not as root)
and everything worked. Still not ideal but much easier. Why should installing
software require some kind of expert? Why didn't the developers solve that
problem themselves so every user can just use it easily. This isn't limited to
web apps, it's most open source software. The authors don't put effort into
making it easy even though they're the ones who are best positioned to both
understand it and provide the greatest benefit by saving everybody else from
repeating the same struggle.

~~~
dikaiosune
From the developers' INSTALL file:

 _" Hosting Rails applications is complicated. Even if you already have
Postgres, Redis and Ruby installed on your server, you still need to worry
about running and monitoring your Sidekiq and Rails processes. Additionally,
our Docker install comes bundled with a web-based GUI that makes upgrading to
new versions of Discourse as easy as clicking a button."_

The basic install guide is a bit lengthy, but amounts to writing these
commands:

    
    
      ssh root@machine.address.of.choice
      wget -qO- https://get.docker.com/ | sh
      mkdir /var/discourse
      git clone https://github.com/discourse/discourse_docker.git /var/discourse
      cd /var/discourse
      cp samples/standalone.yml containers/app.yml
      nano containers/app.yml
      ./launcher bootstrap app
      ./launcher start app
    

There's definitely some extra configuration that has to take place between
these steps, but it seems to me that they've gone through a lot of trouble to
make it easy for their users. I'm not sure where you're getting the idea that
it requires an expert to install Discourse -- it only requires one when you
have a _very_ unconventional build toolchain on your Linux box and refuse to
use Docker (the officially supported deployment method).

~~~
Asbostos
OK, I understand now.

------
Tloewald
Ok I'll bite. This is a good reason to use PHP. In fact, I first learned PHP
because I had a specific web application I wanted to write that needed to be
dead simple for relatively tech phobic users to deploy, and PHP won.

For small things it works well and I've never had a single installation as
painful as any of the stuff described in this article. For big things, well
there's Facebook. It may be ugly, but it's definitely doable.

It's not the new shiny, it's full of warts, and it's unfashionable. (If you
tell a fellow programmer you code in PHP you immediately need to rebuild
credibility somehow.) I personally prefer working with JavaScript and nodejs
but I don't think the deployment situation for nodejs is much better/different
from what's described here.

~~~
dmix
Thinking these high level problems can be solved by switching to a different
language is quite a common fallacy in the tech world. Almost always it comes
down to being familiar with the platform/OS/languages you're working with
rather than a few attributes of a particular technology choice. The question
of technology choice in that context is then primarily about using something
with reasonable popularity and an adequete community supporting it - but
usually not much more than that.

------
bfrog
Nix and NixOS as a whole solve this entire problem in a really nice way. Its
still not ready for prime time from my personal experience, but the ideas and
foundation are there... just needs more polishing.

------
xmrsilentx
Another title: Coding Sucks: Why a Job in Programming Is Absolute Hell

------
limaoscarjuliet
Side note: LD_LIBRARY_PATH is bad, do not use it. Among other things it will
crash your 64 bit app if 32 bit lib matches by name (or vice versa). Bad, bad,
bad. Use ld and ldconfig.

------
jimmytucson
I think we've all felt like this before.

Is this one of the reasons people use Java? Is it way wrong to say Docker is
like a JVM for non-Java?

------
btbuildem
Haha, Docker. Few months ago that word started being peppered throughout
conversations in the office, everybody keen to jump on the bandwagon. Guess
what, now we hired a "docker expert".

If this thing is so simple a toothless toddler could use it, how come now we
need experts in it, huh?

------
leke
What rhymes with 'piss horse'? I'm really curious now.

~~~
zifnab06
Discourse?

------
kuebelreiter
This is a story of somebody not really knowing what he does.

------
KhalPanda
I quite like Node BB...

------
devit
The author seems to mistakenly believe that Linux is a multi-user system where
not everyone has root privileges.

The reality is that almost nobody uses it that way, it's not tested for that
and due to the huge API surface area the chance of some kind of exploit is so
large that it cannot be secured even if it people tried.

The only reason "root" exists currently is to make it less likely to
accidentally change or destroy the operating system.

There is no way to provide a multi-user experience on a physically accessible
system (cannot defend against physical attacks), and the way to provide it on
a remotely accessible system is to assign a virtual machine to each user.

