
It's the Future - ctoth
http://blog.circleci.com/its-the-future/
======
akanet
It really is striking how products like Docker, even while delivering
incredible value, continuously fail to message themselves in an intelligible
way. If you go to [https://www.docker.com](https://www.docker.com), you see:

"Build, Ship and Run. Any App, Anywhere."

Jesus Christ. I get that you're The Future, but make the value prop for me
here, at least. Why should I use Docker? What parts of my stack does it
replace? When does the cost-benefit make sense? What new things can I do that
I couldn't do before?

They made a separate page just to address this giant "Huh?":
[https://www.docker.com/whatisdocker](https://www.docker.com/whatisdocker),
which I feel is equally obtuse.

Luckily, the product itself is fantastic, so that gives you a lot of wiggle
room on your website and documentation. Like, a lot.

~~~
shykes
Thanks for the frank feedback. I agree that we have a lot of "wiggle room", as
you so diplomatically put it. Out of curiosity how do you feel about the
current README on Docker's repo?
[https://github.com/docker/docker/blob/master/README.md](https://github.com/docker/docker/blob/master/README.md)

One problem we've encountered is that the audience for Docker is incredible
broad - much broader than you might imagine when reading Hacker News for
example. It is extremely common for CIOs, IT directors, and various business
managers to just pick up the phone to ask us (or our partners) what Docker is
all about. So it's difficult to tell a story which satisfies all audiences.

But I don't think that excuses everything. We are definitely better at
building our product than at explaining it.

One interesting side-effect is that, if you've been exposed to Docker-related
marketing (in a broad sense), most of it probably didn't originate from us,
the creators of Docker. This is sometimes problematic because Docker is so
polarizing: depending on who tells you about it you will get such a different,
often distorted picture of reality. Either Docker is a miracle cure for every
disease on Earth (indicating an over-enthusiastic Docker fan, or a vendor
trying to sell something to Docker fans), or it's a scourge sent by the gods
of the Unix valhalla to punish mankind for techno-hipster false idols such as
javascript and php (indicating a jaded Docker-skeptic, or someone trying to
sell something to Docker-skeptics). Either way, it creates a lot of noise. And
as you point out, there might be less noise if we ourselves did a better job
at explaining when using Docker is, or is not, a good solution.

~~~
benmathes
Big platforms tend to be different things to different people. Might help to
have a different "what is docker?" page for the various roles you encounter.

    
    
      For CIOs: click here
      For developers: click here
      For sysadmins/devops: click here
      For platform providers (e.g. heroku): click here.

~~~
angersock
Whenever I see a product segmented that way, I'm immediately suspicious.

That much work means that a) they're trying to sell something because it isn't
obviously better and b) they're more worried about messaging than being simple
and useful.

~~~
NeutronBoy
> they're more worried about messaging than being simple and useful

But useful means different things to different people. To a dev, Docker might
mean you keep your environment clean, scripting of automated testing, and have
platform agnostic deployments. To the CIO, Docker means your devs spend less
time futzing around with their stuff and more time working on the product. Or
that he can upgrade his infrastructure gradually and not have to worry about
compatibility. Yes, it's the same thing, but different audiences _need_
different messaging.

~~~
wwweston
> To the CIO, Docker means your devs spend less time futzing around with their
> stuff and more time working on the product

The question is whether or not the CIO knows that the devs _are_ in fact
futzing around with ad hoc solutions to problems that Docker solves.

I think the parent is arguing that in the right flow of things, that awareness
is going to flow up to them from the people closest to the problem (the
dev/ops folks working with it) rather than from a vendor with a vested
interest in adoption to an exec/manager whose understanding of the problems
their staff face may well be a high-level view at best. And who are prone to
make decisions off of social proof plus that good messaging rather than
knowing how well the solution fits their problems.

(Not that having the engineering staff involved is any guarantee that decision
won't be mostly made off of social proof plus messaging. It just decreases the
chances. :/ )

~~~
shykes
From my experience, it's not either-or. First you need "bottom-up" adoption by
actual practitioners (in the case of Docker, developers and sysadmins). Then
you also need to understand the constraints and requirements of people who are
in a position to say "no". Those include managers, but also procurement,
architects, network engineers, security teams, etc. Those people aren't the
ones championing your product (most of the time they don't have very strong
opinions either way), but they have a job to do, and if a new dev tool affects
their ability to do their job, they're going to say "no".

It's very common for developers or sysadmin to contact us and ask for a
powerpoint deck, so that they can give a convincing presentation to their
management about the virtues of using Docker in their new project. We even
have specialized teams that go in and do everything they can to help Docker-
based projects succeed.

But as you point out, it all starts with _someone_ inside the organization who
really, really wants to use your product. Otherwise no amount of messaging is
going to save you.

------
Animats
Ah, l33t for system architects.

A few days ago, the CTO of Soylent, the food drink, was describing their
elaborate computer infrastructure. They have one (1) product and a simple web
site. Based on their sales volume, they do about _two sales a minute._ That
could run on a HostGator "Hatchling" account for $4/month, using one of the
seven off the shelf shopping cart/payment programs HostGator offers.

~~~
nether
Soylent had exceptionally poor order handling prior to their latest round of
funding. The way they've run their business and development seems like a total
joke. 6-10 month order delays over a year since they started selling? (Not to
mention charging orders before shipping) "Most scientific food" \- zero
clinical trials? Are you shitting me?

~~~
joezydeco
Might want to read this:

[https://medium.com/@alando46/how-we-spent-500-on-tech-to-
shi...](https://medium.com/@alando46/how-we-spent-500-on-tech-to-ship-2-6m-of-
soylent-3abfdcebee78)

If you knew how most modern warehouse/shipping facilities ran their IT, your
head would spin.

~~~
Animats
Figures. They overbuilt their public-facing web system, and underbuilt their
fulfillment system. Reading that, you can see the problem - their order
system, fulfillment system, and order tracking system didn't talk to each
other properly. You don't just ship stuff blind; you read back the carrier's
online shipping data and match it with the orders, so you know you didn't miss
anything. Otherwise, you don't know you screwed up until the angry customer
calls you.

------
VieElm
Well these technologies are there to help bring Google type infrastructure for
businesses that need it. If you're running a CRUD app just fine on Heroku you
don't need to do any of this and you shouldn't.

When your availability starts having problems or your data is getting too much
for one machine, you start having problems of scale. Where you are in terms of
scaling issues should lead you to the next iteration of technology required to
keep your service up.

Starting out of the gates with that one idea you're not sure anyone will be
interested in? Just stick to deploying something fast and not worry about
Google style scaling solutions.

~~~
jakejake
Seems to be the problem with a lot of technologies. If you're a HN reader, you
can start to get the feeling that you're falling behind if you're not doing
these things. Even though they are totally inappropriate for the majority of
us who are working on smaller systems.

~~~
VieElm
As an engineer I feel an obligation to know about these things and how they
work. It might be appropriate for my work. For my own projects I am just going
to start out with a Django or Meteor app and if I'm lucky enough to have to
worry about scale I'll have some idea of what to do next.

~~~
riffraff
> It might be appropriate for my work.

so might be AS400, but few are trying to learn that I guess.

Let's be fair, it's quite likely that if anyone is looking into this stuff is
more because it's the technology du jour, rather then the off chance that they
end up with 100 million users. It could be useful, and learning stuff is fun,
but that's not the real drive.

~~~
nathan_long
> if anyone is looking into this stuff is more because it's the technology du
> jour, rather then the off chance that they end up with 100 million users

Knowing the technology du jour is a lot more likely to get you a job, even if
an older technology is better in some ways.

------
myth_buster
Sadly I often feel this is the state of most of the web-dev stack due to high
churn. There are extremely smart people working on tools who keep forking out
new frameworks/tools/components/scripts and a bigger pool of smart people who
keep consuming it.

Having seen in enterprise incredible amount of man-hours wasted on migrating,
re-writing and creating POCs on the latest fad, I sometimes wonder whether
this cycle is productive or we keep at it just to keep our brains working and
not go insane thinking of geo-political/financial/meta-physical questions. Its
perhaps Crack for the tech crowd.

~~~
digi_owl
It may have picked up a quicker cadence on the web, but i swear this has been
the case on "desktop" as well.

------
aerovistae
This is exactly how I feel whenever people start talking about this stuff. Is
that bad? I know this is a satirical post but it seems like a pretty faithful
depiction of reality.

~~~
unoti
This is really nothing new. It's a DevOps version of Joel on Software's famous
essay about Architecture Astronaughts from 2001:
[http://www.joelonsoftware.com/articles/fog0000000018.html](http://www.joelonsoftware.com/articles/fog0000000018.html)

~~~
aerovistae
From that article:

"The Architecture Astronauts will say things like: "Can you imagine a program
like Napster where you can download anything, not just songs?"

hahahahahhaha bittorrent.

~~~
digi_owl
Even Napster was used for non-music stuff towards the end, by renaming
archives as mp3.

And the basic protocol was later adopted for more generic sharing systems,
never mind the number of clones that came about after Napster was lawyer
bombed.

Bittorrent is just the latest in a long string of P2P systems, with the
biggest difference being the lack of a central search server.

~~~
Aloisius
Yeah, I remember we had to put an MP3 frame checker in the app to help prevent
them from being shared. We had enough legal trouble with just the music
industry suing us.

------
vbezhenar
I may be old guy but I never get it. Why do I ever need all this
virtualization stuff? I just buy a server, install CentOS, install postgresql,
create another user, install java, download&unarchive tomcat and write some
simple bash scripts. Voila. Everything works, everything is protected,
performance is superb and I can do all that within one day. Good enough to
serve few thousand requests per second. May be not enough for Facebook, of
course.

~~~
imaginenore
And then that data center has a fire, your server is gone, your data is gone,
your business loses X thousands of dollars per day, all because everything was
on one server.

~~~
vbezhenar
I usually have backups to a cloud, so thats's really not a problem.

~~~
imaginenore
Yeah, but it still takes time to set up a new server, launch it, update the
DNS. And you lose all the data since the last backup.

------
bch
Anne > What’s Aphyr?

Bob > -Aphyr is that guy who wrote, ‘Call Me Maybe.’ You know, the distributed
systems and BDSM guy?

Anne > What? Did you say BDSM?

Bob > -Yeah, BDSM. It’s San Francisco. Everyone’s into distributed systems and
BDSM.

This hilarity demonstrates the tone, and alone is worth the price of
admission. Great job with this article.

~~~
ahelwer
"Paxos is like this really old distributed consensus protocol from the 70s
that no-one understands or uses."

I've run into this. In reality _The Part-Time Parliament_ was published in
1998 and _Paxos Made Simple_ in 2001.

~~~
Dewie3
Maybe they got their IT-timescale mixed up with our normal timescale. Things
that happened in IT in the 70s on the normal timescale was like 200 IT years
ago on the tech timescale, man.

------
ninkendo
I've always thought of the whole containerization and orchestration "hotness"
(I guess we're calling it "gifee" now?) to be great for people who want to
write their own PaaS. And not really anybody else.

I mean, it's great that the pieces are all there and open source now. You can
get to a really great place with automation by gluing together Docker/Rocket,
Mesos/Kubernetes, etcd/zookeeper, etc. But for now, a complete solution still
requires you to bring a _lot_ of your own glue.

I mean, by the time you've actually covered enough edge cases to make all
these ops components work together in a meaningful way, in a way that doesn't
doesn't fail randomly or trip over itself when some critical component breaks,
you may as well just start charging other people money to use it.

I say this from the point of view of someone who's done it all (implemented a
private Heroku on the stack described above [1]), and although it works
amazingly well (for literally hundreds of internal apps), it was _not_
trivial... not even close. We're talking probably 1-2 man-years of effort to
get it to the point where it's usable, and that's with leveraging as much
existing tech as we can.

To anybody else, as in, any company who actually wants to ship a product
(where the product isn't just a PaaS), I just don't see how it's worth it.
Just use heroku (or elastic beanstalk, or appengine, or whatever.)

[1] I'd love to make all the glue open source but I'm not really in a position
to do so. But I suspect I'm not the only person who's done this... I really
think anybody who's gotten this whole "the future" stack working solidly is in
a similar position as me: if you really did get the job done for a single app,
chances are you've invented an internal Heroku of your own.

~~~
helloiamaperson
Or use cloud foundry
([http://cloudfoundry.org/index.html](http://cloudfoundry.org/index.html)) if
you have a burning desire to stand up your own paas vs. just using heroku.

------
smadge
Ughhh this.

I just want to give my project to a PaaS and let them figure out everything.

I was looking into Google App Engine, but they didn't support some language
features I wanted (e.g. Java 1.8, Servlet 3.X["I know, programming in Java?
You're stuck in the past"]). So I looked into their new Container Engine. But
like this article points out, it makes deployment 500000000 times more
complicated than it should be.

~~~
kordless
Check out [https://giantswarm.io/](https://giantswarm.io/).

------
noonespecial
Meanwhile, the last two web applications I've consulted on are now more than
12 years old, use SOAP and where written in _Delphi_.

They work surprisingly well to this day and despite being horrified upon
hearing that this was what they were built on, I found them surprisingly well
done.

I wonder what will still be running 12 years from now and what it will look
like.

------
thebouv
Reading this made my brain feel itchy.

Because it's too close to some conversations I've had.

~~~
plaguuuuuu
You might also have parasites :3

------
nchelluri
This is one of my favorite blog posts of all time.

I love it.

Thank you, CircleCI, for posting this!

------
hawski
Reading this I keep my fingers crossed for
[http://sandstorm.io/](http://sandstorm.io/) or another Android-of-services.

------
mmanfrin
_Rails? That 's so yesterday. New hotness is React!_

React? I thought we were doing Angular?

 _Come on man, you 're stuck in the past!_

~~~
canistr
New, _new_ hotness is React Native (or whenever FB decides to make it also
available on Android) and JSBlocks.

~~~
thinkdevcode
New, new, new hotness is Aurelia! Get with the times!

[http://aurelia.io/](http://aurelia.io/)

Edit: just wanted to point out that this post was only partially satirical. I
absolutely am in love with Aurelia atm.

~~~
devuo
atm

------
icebraining
This hits home. While everyone is talking containers, we're running simple
processes with a Linux user per instance, and I feel no need to add more
complexity to our system, except I'm really struggling to automate stuff.

It seems that if you aren't running a full dockerized cluster of services or
outsourcing everything to a PaaS, you're left with building all the
infrastructure yourself. What did people use before this great new wave?

~~~
digi_owl
I think there is a mix of things being bandied about under the "container"
banner.

On the one hand it is about getting more bang for your hardware buck.

On the other it is about someone getting so deep into netsec that they have
developed gills.

In the bang for bucks category you have a chain of one box pr database etc.

Then noticing that the hardware sits idle most of the time, so virtualization
is depolyed to pack more server on a single box to keep it in use.

Then noticing that virtualization comes with a performance overhead, so it
gets replaced chroot/containerization to give the impression of unshared box.

In the netsec category it is really about namespace. Limiting the view of the
world the processes gets.

This has a superficial similarity to chroot, but can go much much deeper.

And if one go deep enough, every server ends up looking like a digital fort
Knox...

~~~
icebraining
Yeah, but none of that actually needs docker and images and such. If you want
to take advantage of the whole server, you can simply run more regular
processes on it, and you can launch them on different namespaces using systemd
or other process manager. You don't need the whole workflow that comes with
these new tools.

~~~
digi_owl
From a traditional sysadmins point of view, thats true.

these days though you are looking at devops...

Never mind that more recent systemd releases can grok docker containers.

------
rehtona
Now do this for the Hadoop ecosystem. I'm ripping my hair out because of the
complexity of it all. I get that distributed is difficult, but this just feels
like too much. Abstraction upon abstraction upon abstraction. Hadoop is this
thing over here, but now we have this other thing built on top that is way
better. But actually that thing also sucks, so we built a whole layer on top
to abstract away the badness of that other thing plus two other alternatives
to it. Oh, but that wasn't good enough, so now you can write queries in this
other language. And on and on and on it goes.

------
msane
To the position that containerization is needless complexity for simple or
non-scaling apps: one of the benefits of containers is it can create
development environments which can be identical to your production
environment(s); no matter what platform you're always running the same code,
same artifacts, same images.

Virtualization does this too, but at great cost. I wish the kinks were better
worked out at this point as well, and hope we start to converge around a few
well-working patterns and toolsets. I expect it to happen. In the meantime it
is chaos and easy to laugh at.

~~~
pbiggar
But (and this is a thing I've been working on) do you really want your dev
environment to be identical to your production environment? I think you don't.

As an example, suppose you use Go. Your dev environment is 500mb of compilers
and toolchain. Your production environment is (hopefully) a container with a
single static binary on it.

~~~
lstamour
The point is not that dev and production environments are identical, but that
dev _staging_ and production environments are identical. Your dev box can have
whatever you like on it, but if it can't fake production behaviours, it's not
going to help you catch and debug bugs in production. Of course there are
limits. You could, it's perfectly valid, buy twice as much capacity in
production and run it twice there. And use remote debugging tools. The point,
especially with ops involved, is that your dev box is a snowflake and you want
the least amount of manual configuration possible. It's not a requirement.
Your app will run fine in production without it simply by testing in
production. But few people willingly recommend that approach, even as pretty
much everyone does it at some point or another. Even the best emulation of
production won't prevent the need for debugging in production when a bug isn't
caught before it gets deployed ;-)

~~~
pbiggar
Yeah, but folks are talking about using containers for actual dev
environments. Because you want to make sure everyone is using the same version
of the go compilers, etc.

~~~
lstamour
Hmm. By that logic, one VM compiles the app and produces or inserts it into
another VM runtime environment....?

~~~
pbiggar
Basically yes. The dev container is used to build the production container's
image.

~~~
msane
I like where you're going with that

------
jbigelow76
Only made it about a third of the way through but it seemed like latter day
Xtra Normal MongoDB/web scale shtick[1].

[1]
[https://www.youtube.com/watch?v=b2F-DItXtZs](https://www.youtube.com/watch?v=b2F-DItXtZs)

~~~
latchkey
Yea, too bad xtranormal is no longer around.

------
sebringj
These frameworks arise from huge corporations then get preached to the masses.
Problem is, they are only useful at the huge corporation level. Ironic.

------
facepalm
A seeming eternity ago I let Maven convert apps into war files that supposedly
could be deployed into any web application server.

What do the new containers add on top of that (or other than that)? Only the
option for more services (not just web app, but database, different languages,
whatever)?

It seems odd having to worry about that kind of thing.

~~~
eurleif
>It seems odd having to worry about that kind of thing.

It seems odd having to worry about things that aren't Web apps? Why?

Containers are also language-agnostic. .war files are only for JVM languages.

~~~
facepalm
Mabye it is either a ploy of admins to make programmers do their work for
them, or a ploy of programmers to put admins out of their jobs?

I don't really know docker yet. So if I need a database, rather than
instantiating a server with a database, I would create a docker image that
runs a database? Then deploy it to some server that can digest docker images
(is there a docker image for that)?

The point of war files was not having to worry about the server.

------
BerislavLopac
In the immortal words of David Wheeler and Kevlin Henney: "All problems in
computer science can be solved by another level of indirection, except for the
problem of too many levels of indirection." Feel free to substitute
abstraction for indirection.

~~~
jimbokun
Just saw this linked on HN yesterday:

[http://www.dwheeler.com/innovation/innovation.html](http://www.dwheeler.com/innovation/innovation.html)

I think Wheeler and Henney's quote applies to a lot of the advances on this
list.

------
bch
This OP gem-of-a-story reminds me of another masterpiece, about Tcl:

[http://www.markroseman.com/tcl/dyingout.html](http://www.markroseman.com/tcl/dyingout.html)
(ref:
[https://news.ycombinator.com/item?id=9689046](https://news.ycombinator.com/item?id=9689046))

Edit: link to submitted story.

------
fredsters_s
hilarious and true.

------
jtwebman
What no mention of running Dokku on Digital Ocean.

------
doppp
What's funny is that CircleCI is one of a million CI deployment services out
there too.

------
crispycret
I read this in the voice of Randy from southpark. Best episode ever!

------
lafar6502
sounds strikingly realistic for a joke

------
serve_yay
So you posited a totally dumb guy to talk to, then you talked to him and he
said dumb stuff. Cool.

