
The Guardian goes all-in on AWS public cloud after OpenStack 'disaster’ - robzyb
http://www.computerworlduk.com/cloud-computing/guardian-goes-all-in-on-aws-public-cloud-after-openstack-disaster-3629790/
======
hoodoof
"That software we've got causes us so many problems, it's disaster! When we
get the new software everything will be fixed."

X to Y Y to X A to B B to C

MongoDB to this MySQL to that Why we switched Why we switched back

blah blah

It's one of the oldest themes in computers - man, this software is crap - that
new software will fix all our problems! Until you find the problems with the
new.

Precisely the same age is the new software vendor harnessing all that negative
energy about the competitor and megaphoning the "man, the new will fix the
old, we're so excited!"

Having said that, I have worked with the compute servers of all the major
cloud vendors and Amazon must be credited for the quality and consistency of
its AWS systems, and Google and Microsoft's cloud computing is in almost all
respects equally good - there is as far as I can tell absolutely no reason to
choose Amazon over Microsoft Azure (yes, even for Linux systems) or Google
Compute Engine. In some areas of functionality GCE and Azure I found to work
much more easily and smoothly than Amazon. When I went to work with OpenStack
I immediately found the lack of completeness, inconsistency and lack of polish
characteristic of many open source projects - trying to do simple things
instantly became hard problems.

~~~
rogeryu
It's a common way to "solve" things. I live in the Netherlands. Ten or fifteen
years ago we tried to cut education costs by merging schools into large school
clusters. Small schools were too expensive. Now they go back and say that the
large schools are too impersonal. In ten or twenty years they will go back to
large schools. You can bet on this.

Is it good or bad? It means that things are changing, and they are not going
back to the same situation as ten years ago. It is a matter of evolving. There
are new insights, new technologies, new people.

~~~
gambiting
Yep, in Poland it's the same with schools. We used to have 9 years of primary
school + 4 years of high school and that was it. In 1999 a new, 3-year long
"middle school" was introduced,shortening primary school to 6 years and high
school to 3 years. Now the new government elected this year has vowed to get
rid of this system and go back to the old one, at some untold cost, "because
old is bad, new will be better, except that in this case new is old".

------
vidarh
It's a bizarre article. On one hand I understand their frustration with
OpenStack. On the other hand, if they expected to pretty much build their own
AWS without any warning bells going off all over the place I find it very hard
to be sympathetic - they're not remotely large enough for that. If they'd set
their goals lower they could have had a very capable system fairly easily.

That they're having problems getting load balancing working properly is a real
sign that their ops team just didn't know what they're doing.

~~~
_Codemonkeyism
"they're not remotely large enough for that."

Second that.

Maybe there are only ten companies in the world large enough to have positive
ROI to build an internal AWS, like Google, eBay, Facebook, ... who don't want
to depend on Amazon.

~~~
leoc
Sounds like there's a space for a Visa- or Underwriters' Laboratories-like
cloud company owned by and run for a consortium of medium-sized publishers.

------
MrTonyD
As part of my job I teach some classes and give the occasional conference
talk. As a result, I've spoken to dozens of people who have tried to adopt
OpenStack and eventually gave up. There are a lot of reasons. But I also
happen to know a very senior technical person with OpenStack - and he
described a completely dysfunctional organization. I wish that there were some
way to stop these organizations that are wasting people's time - maybe
somebody could sue them out of business for the good of the industry?

~~~
johansch
"More than 500 companies have joined the project, including AppFormix, Arista
Networks, AT&T, AMD, Avaya, Brocade, Canonical, Cisco, Citrix, Comcast, Cray,
Dell, Dreamhost, EMC, Ericsson, Fujitsu, Go Daddy, Google, Hewlett-Packard,
Hitachi Data Systems, Huawei, IBM, Intel, Internap, Juniper Networks,
Mellanox, Mirantis, NEC, NetApp, Nexenta, Oracle, PLUMgrid, Pure Storage,
Qosmos, Red Hat, Solidfire, SUSE Linux, VMware, VMTurbo and Yahoo!"

Yeah, I am sure it is quite easy to make them agree on the directions OS
should take...

------
meekins
I'd be really interested in the actual technical problems The Guardian had
with OpenStack and what's the current situation with the OpenStack project.
The company I work for has some interest on the platform but I am not aware
what to expect (problems, new features etc) from it in its current state.

A couple of years ago I experimented with and compared some open source cloud
platforms. Admittedly my knowledge is old, but alredy then I was wondering the
hype and visibility of the OpenStack project. While it has many big-name
supporters that guarantee its lucrativeness for the enterprise, its feature
set and flexibility was seriously behind other open source alternatives.

Back then I fell for OpenNebula because even though it had its warts it
already delivered many features (especially concerning hybrid clouds and
heterogenous virutalization environments) that OpenStack still had on its
future roadmap.

~~~
rkeene2
I also tried OpenStack for a self-assembling cloud product I built. It was a
huge pain and I also switched to OpenNebula about a year ago. It's been
significantly better !

------
alrs
Andrew Clay Shafer called it in 2013:
[https://stochasticresonance.wordpress.com/2013/11/04/opensta...](https://stochasticresonance.wordpress.com/2013/11/04/openstack-
a-plea/)

In a former life, I was frustrated by the futility of trying to be involved
with the Openstack project. My next phase was to warn people away.

Now I wonder why I bothered. Years later it's apparent that I was right, but
what benefit do I see from that? Better to aggressively ignore bad technology
and spend effort on the stuff that looks promising.

~~~
rwmj
He mentions Ceilometer which was recently rewritten
([https://wiki.openstack.org/wiki/Gnocchi](https://wiki.openstack.org/wiki/Gnocchi)).
Of course whether the replacement will work any better is an open question at
this point. Never really understood why they don't just use rrdtool and have
done.

Networking which he also mentions in passing is also being rewritten
([https://www.openstack.org/summit/vancouver-2015/summit-
video...](https://www.openstack.org/summit/vancouver-2015/summit-
videos/presentation/ovn-native-virtual-networking-for-open-vswitch)).

I generally agree with his final points - OpenStack really needs to work for
the end users, not for the many competing vendors working on it.

~~~
alrs
They're not competing, they're cooperatively building a potemkin village so
that they have a cloud story to tell Wall Street and the tech press. AWS is
steamrollering everyone, but most of them incorrectly imagine AWS to be a
mashup between VMware and a payday lender scam. They've been content to put
people on stage every six months to peddle this-actually-works happy-talk.

Dell tells the story. February 5, 2013: Dell announces plans to go private.
May 20, 2013: Dell abandons Openstack.

~~~
sinxoveretothex
Dell abandoned Openstack in May 2013? Better tell ComputerWorldUK then, they
seem out of date by 9 months because they linked me to this story from
December 2013: [http://www.computerworlduk.com/news/it-vendors/dell-and-
red-...](http://www.computerworlduk.com/news/it-vendors/dell-and-red-hat-team-
to-sell-enterprise-openstack-3493664/)

~~~
alrs
[https://gigaom.com/2013/05/20/dells-revised-strategy-
steps-b...](https://gigaom.com/2013/05/20/dells-revised-strategy-steps-back-
from-openstack-public-cloud-spotlights-enstratius/)

------
smcl
There's little substance to this other than "OpenStack is hard, let's go AWS!"
\- I would have liked to hear where exactly they ran into difficulties and
what the problem was in detail. This is for two reasons:

1\. "We have switched some components in our software stack" without much info
is not an interesting story to read.

2\. I have friends who work on OpenStack in RH who would I'm sure be very
interested to know the sort of troubles that The Guardian had so if there is a
usability or functionality issue it can be addressed.

------
DanielBMarkham
You know, there's probably a sweet spot between "AWS all the things!" and "We
will build our own cloud"

I mean geesh, people have been building small clouds since there were servers.
That's the way mom and dad did it, and by gummity it oughta be good enough for
you. The "new" stuff was autoscaling, PaaS, and so forth.

So build out a few servers for content management and publishing, then write
very small amount of code to push what you have out to a CDN. If you want
realtime data capture, capture it using AWS (or whatnot) and pull it back
locally.

I'm not saying that's optimum for every solution, just that the all-or-nothing
kind of thinking is probably what lured them into building their own cloud in
the first place. You need a cloud for some stuff, so use a cloud. But you
don't need a cloud for every freaking thing the company does. Its assets in
the form of text content, internal docs, and branding are probably extremely
small in modern terms.

------
saintfiends
It's a bit scary that Amazon AWS is the only complete solution in this space
now.

There are so many open projects in the cloud space that one would think it's a
solved problem.

Here's a list off the top of my head:

1\. Docker - Container implementation.

2\. Kubernets - Manage a cluster of Linux containers.

3\. Mesos - Manage a cluster of resources (not just containers, I'm guessing
there are some feature overlap with Kubernets).

4\. CoreOS - An OS specialized in running containers.

5\. RethinkDB, CouchDB, Cassandra - Distributed databases

6\. Ceph, GlusterFS - Distributed file systems

7\. RabbitMQ, NSQ, NATS - Distributed queue systems.

8\. Manage VM's in a Data center ?? - I don't know any projects in this space.

What's missing is an interface to manage all these together. Maybe this is the
direction OpenStack should be heading?

~~~
athrun
You are comparing open-source Software against Services.

It's as if you were saying that to compete with Amazon.com (retail) you simply
have to deploy Magento or Open Cart...

------
jdubs
After developing a product on openstack and dealing with so many issues; I
miss AWS so much.

------
jakozaur
Related read: [https://www.packet.net/blog/how-we-failed-at-
openstack/](https://www.packet.net/blog/how-we-failed-at-openstack/)

------
danieltillett
I know cloud everything is the modern way to go, but isn't the Guardian just a
glorified blog? Why could they not just push out their content to a series of
load balanced static iron severs? Call a cron job every 5 minutes and be done
with it.

~~~
illumen
They are a dating website amongst other things. It's a large billion pound
media organisation now with many different units.

~~~
danieltillett
I am not sure why they are running a dating site, but even if they consider
this important why one system to rule them all. Hardware is cheap compared to
developers.

~~~
DanBC
> I am not sure why they are running a dating site

Many newspapers (at least in UK) did. Guardian is one that stuck around to
digital age. It makes them money.

------
jackgavigan
I've always been introgued by the Guardian's focus on building its own
technology. It seems a "bold" strategy, given that tech is not its core
business and the newspaper part of the group (which is pretty much all that
remains now, since they've sold off their other assets) has been consistently
losing money for some time now. I wonder if that strategy would still be in
place if it wasn't owned by a not-for-profit trust.

Other publishers who have gone down the "We're a tech company!" route have
been forced to give up on that strategy[1]. I wonder whether the Guardian will
be able to make that breakthrough, or whether they'll end up migrating to
Wordpress or something similar.

1: [http://digiday.com/publishers/gawkers-kinja-retreat-shows-
fa...](http://digiday.com/publishers/gawkers-kinja-retreat-shows-false-hope-
in-publishers-licensing-tech/)

~~~
gaius
They have a few very charismatic people in tech there, the kind of people who
are very good showmen and salesmen (or women) but they've been "architects" or
similar their whole careers, not hands-dirty engineers, and their grasp of
what's feasible or not is driven mainly by which vendor buys them the nicest
lunch or has the nicest typography in their whitepapers.

~~~
jackgavigan
Do you have direct experience of working there?

~~~
gaius
Back in the day as an expensive consultant, yes, and I know a few people there
now.

If you want to see a newspaper who are _really_ doing something interesting
with technology, check out the Daily Mail, last I heard they were going all-in
on Scala. Newspapers that are still in print are fascinating IT places -
because come hell or high water, the paper _has_ to come out the next morning.
They are the definition of mission critical. Websites are easy in comparison.

~~~
jackgavigan
Interesting. Did any of these showmen find their way into GDS, by any chance?

Would love to discuss over a pint sometime. Do you attend HNLondon?

------
trolla
I’ve used Openstack at my current $dayjob and I’d say that Openstack is
probably one of the buggiest solutions that I’ve ever used. Even simple things
like stopping or resizing VM-s sometimes failed completely and the VM came
unusable after that. Feature wise it’s also lacking. Seems that Openstack
spends more time/money on marketing and generating hype than creating a good
solution. I’d even say that Docker has a somewhat similar problems, they spend
more on hype/marketing than creating a stable and usable solution. However, at
least Docker is more usable right now than a year ago, the same can’t be said
about Openstack.

~~~
mkulke
I guess those expectations are part of the problem, a talk in the recent
openstack conf higlighted this: OpenStack was initially created for what these
days is called "cloud-native" workloads, every VM was considered ephemeral.
companies and users then try to mould it into a cheaper VMWare and are
frustrated how bad it is at this.

Resizing, stopping VMs, while admittedly being a rather trivial tasks, point
to this usage of OpenStack. When I would mourn the loss of individual VMs on
OpenStack (or public clouds for that matter), I would turn gray soon.

------
melted
Sounds like they'll be writing that AWS is a disaster 2 years down the road.

------
dankohn1
"we have had to host it on physical tin". Is this an English English thing? In
America, we would say "host on our own iron". Do other countries use different
metals?

~~~
petewailes
British person: I don't think this is one I've come across.

~~~
f3y67j
Another brit: just once, three years ago by a Programme Director determined to
eradicate our IT dept "We are not going to feed and water our own tin!". I was
equally bemused. However the outcome has been very detrimental to our org.

------
pastyboy
money no object then? Of course not. Safe harbour is a huge problem in EU AWS
breaking the rules. The G could find itself in a sticky regulation mess when
people start suing for data breeches.

~~~
vidarh
Money probably no object given that they were using Cisco UCS and Netapp
storage to begin with. With that as a starting point, a perceived disaster
brewing and what's likely to be fairly steep discounts from AWS to get them to
move given their size I'm sure it looked attractive (if you're paying list
prices for AWS at any kind of size, you're leaving money on the table; if
you're first going to use AWS at least make sure to negotiate).

------
Angostura
Stephen Gran, the senior integrator charged with getting OpenStack to work and
who wrote fulsomely about it left the Guardian in March 2014 according to
LinkedIn.

------
Gigablah
Going off-topic here, but the AWS logo used in the article seriously looks
like a cheap knock-off version.

Edit: apparently they sourced it from Wikimedia Commons, which has a rather...
shoddy replica.

