
The BBC's hi-tech failure: Don't Mention It - chrbutler
http://www.economist.com/blogs/blighty/2013/06/bbcs-hi-tech-failure
======
handelaar
Was wondering if anybody here would notice that the "CMS" being described here
was a project to handle every single second of radio and TV footage from
ingestion through recoding through editing to broadcast, and then beyond that
to transcoding everything for whatever devices want to watch or hear it once
it's finished.

People here hear "CMS" and think we're talking about text.

Incidentally the parts of this boondoggle which actually exist are being used
as components in the BBC. The headline last week was about the decision to
terminate that particular project because it had got far, far out of its
depth. Also to put the highly-charming person responsible for it on gardening
leave indefinitely pending the investigation.

It's a massive cockup, but it has had some useable outcomes, and it is a
product class that doesn't exist outside the world of the Beeb, or other
national/international broadcasters. There's nothing off the shelf to buy
here. In the 90s this sort of thing wouldn't have got cancelled, so I'm seeing
this as a sort of progress.

------
JamisonM
Real headline: Publicly funded broadcaster makes same mistake most large
private enterprises make in more or less the same way.

It seems to me that months or years of auditing and hearings and cries for
heads to roll are probably wasted. Just like a private company a few mangers
should probably get tossed or reassigned and the second version of the same
project should get started to address the businesses needs the first one was
supposed to.

~~~
macspoofing
>Publicly funded broadcaster makes same mistake most large private enterprises
make in more or less the same way.

Actually large private enterprises don't usually make this kind of mistake.
This is a typical big government project. Software development is hard.
Building large, complex, poorly defined software systems is hard.

//EDIT

I should clarify that. I don't think a private enterprise would engage in a
project that would have it build an _internal_ system, at that scale ($100
million), having so little impact on it's primary revenue driver, over so many
years!

Big enterprises fail with big software, that's a fact. But they wouldn't fail
in this specific way. This is a quintessential big government software project
failure.

~~~
andyjohnson0
There's nothing special about private enterprises that makes them immune to
risk. As others have said, many IT projects fail [1]. In the private sector
this is just seen as a business risk, but when a public sector project fails
then its "wasting taxpayers money".

For some reason the Economist article doesn't mention that the project was
originally awarded to Siemens and Deloitte [2], but technical failures and
cost overruns by these private companies caused the BBC to bring the project
in-house. It could be argued that the BBC lacked the capability to deliver the
project on its own, but thats also a mistake found in the private sector.

[1]
[http://www.computerworld.com/s/article/9116470/IT_s_biggest_...](http://www.computerworld.com/s/article/9116470/IT_s_biggest_project_failures_and_what_we_can_learn_from_them)

[2]
[http://en.wikipedia.org/wiki/Digital_Media_Initiative#Initia...](http://en.wikipedia.org/wiki/Digital_Media_Initiative#Initial_impetus_and_relaunch)

~~~
_yosefk
There's something special about private enterprises making them reconsider
their approaches, though - the fact that they lose their own money (which is
often combined with having a smaller pool of money to begin with). Few
enterprises can spend $150M and fewer will keep spending upon signs of things
not working out.

As to private contractors doing a poor job when paid with taxpayer money -
makes some sense, as long as said money keeps flowing. In a different setting
the flow would stop more quickly.

~~~
leejoramo
I know of a private newspaper with a circulation of around 30,000 that spent
$3M on a failed Print software system.

How does that compare to the BBC's viewer & listenership numbers for $150M for
a Audio/Video production system?

~~~
takluyver
According to barb.co.uk, BBC 1 (the flagship TV channel) had some 45M viewers
over a recent week. We could consider other channels, radio stations and the
website, but that's already well over half the UK population. By cost per
customer, that's more than an order of magnitude better than your newspaper
example.

------
Finster
I've personally worked at 3 large organizations that have tried to formulate
and develop an all-encompassing content management system. One of those
organizations is a broadcaster that was also trying to build a digital content
management system.

Every single one of those projects has been a total boondoggle. Always over
budget. Always changing scope. Always missing every single milestone and
deadline.

Now, I know that this is not a statistically significant sample size, but this
sort of thing seems to be more common that I initially thought. It's like the
worst-case scenario of design-by-committee.

------
astrodust
The same organization that couldn't be arsed to host some of their historical
web properties due to "high costs": [http://853blog.com/2011/01/25/pulling-
the-plug-on-the-bbcs-i...](http://853blog.com/2011/01/25/pulling-the-plug-on-
the-bbcs-internet-history/)

~~~
MrDOS
It strikes me that the BBC is hilariously bad at archival. They scrapped
dozens of Doctor Who episodes in the name of saving space
([http://en.wikipedia.org/wiki/Doctor_Who_missing_episodes](http://en.wikipedia.org/wiki/Doctor_Who_missing_episodes)).

~~~
objclxt
The BBC is, today, very good at archiving programming material. All output is
kept for at least five years, and the overwhelming majority is kept
permanently (exceptions are things like long-running quiz shows, where samples
of material are kept post five years, although any disposed material is
offered to the BFI and other third parties before being junked).

Doctor Who wasn't scrapped in the name of saving space, it was scrapped
because nobody thought it was worth keeping. The shows couldn't be repeated
(because at the time, Equity had negotiated limits on repeat rights), and quad
tape was very expensive and easily re-used.

More importantly, nobody really thought that Quad was an appropriate archiving
format. Programs would be tele-recorded onto 16mm for overseas sale and
archive. This is not, in of itself, a crazy idea - try finding a quad tape
player today vs a 16mm projector. The problem at the time was there was no
mandate for archival, so the engineering department were junking the tapes,
and the commercial department (who were selling Doctor Who on to other
broadcasters) weren't retaining the 16mm reels.

Even in the 1970s, many people were still not convinced of the value of
archiving TV output. It's not totally dissimilar to the early days of the web
- it was seen as ephemeral. This was happening at pretty much _every_
broadcaster around the world - Wikipedia has a pretty big list of lost TV
programs, most of which were wiped for space.

------
tehwalrus
Nowhere near as much as the £12Bn the NHS spent on an impossible moving-spec
IT system that they also scrapped(1). Oh wait, it's a tenth of the amount.

(1)
[http://en.wikipedia.org/wiki/NHS_Connecting_for_Health](http://en.wikipedia.org/wiki/NHS_Connecting_for_Health)

~~~
thisone
Saying the program was scrapped is a bit off. It makes it sound like nothing
has come out of the project at all.

Choose and Book is widely in use, the Spine exists, as well as the N3 network.

Data format and transfer standards are coming into play now. So instead of one
system to rule them all, it's working towards interop. Which probably should
have been the overall goal to begin with.

People like to think of the NHS as a monolith, but it's really hundreds of
tiny fiefdoms.

~~~
hahainternet
Tiny fiefdoms all on 172.16

They could easily have picked ipv6 and done it the smart way. Oh how things
could have been.

------
_pmf_
I vaguely remember reading uplifting, grandiose articles and presentations
about domain driven design at the BBC; search for "BBC domain driven design".
From this article, I cannot deduce whether this failed CMS is the same system
that has been glorified in these presentations or something else; maybe
someone can shed some light on this.

~~~
kryten
It probably is. I've experienced the fact that DDD scales up only until the
consultants have gone :)

Same as NserviceBus only works until you've paid for the training.

------
squidi
Previous discussion, including info from someone close to the project, here:
[https://news.ycombinator.com/item?id=5762116](https://news.ycombinator.com/item?id=5762116)

------
pero
Related further reading for trainwreck geeks:

EDS (now HP) infamously botched a 1-year ~50million-turned-4 year-multi-
hundred million dollar CRM project for BskyB a decade ago, and eventually lost
a years-long lawsuit.

[http://www.linklaters.com/Publications/Publication1403Newsle...](http://www.linklaters.com/Publications/Publication1403Newsletter/20100317/Pages/BSkyBvEDSTimetoReassess.aspx)

------
DrinkWater
I had the chance to see some of the work Adobe is planning to do with their
CMS (some actual demos from the "Adobe Labs"), and i cant deny the resemblance
to BBCs case. I am wondering whether Adobe was involved...

------
rplacd
A side matter, but handing the BBC's governance over to Ofcom? They haven't
held the Independent Television programme contractors to their remits since
2001 under the threat of any sanction - or, for that matter, put their
contracts up to tender (not that it worked or ever will work if we want a One
Nation Tory-style public broadcasting system again - and not that I remain
suspicious of their inconsistent singling out of Thames Television) since
1993.

The BBC Trust, at the very least, has an image to mantain.

~~~
arethuza
I did a quick check to see how Wikipedia defined a "One Nation Tory" \- after
all they have been pretty thin on the ground this last 30 years (especially
here in Scotland!).

I'm somewhat amused to see that "One nation conservatism" appears to now be
the policy of the Labour Party:

[http://en.wikipedia.org/wiki/One-
nation_conservatism](http://en.wikipedia.org/wiki/One-nation_conservatism)

------
JonnieCache
Note that the content in this CMS is hours and hours of raw video footage
rather than the usual snippets of text.

~~~
true_religion
Duely noted, but I don't think it ought to make a difference in the price tag
unless the CMS was doing conversion and distribution.

~~~
archivator
IIRC, their CMS was supposed to be the Holy Grail of video management.
Centralized and decentralized storage, processing farms, search-by-image,
advanced tagging, etc. Sounded quite impressive but from what I understand,
there are now commercial solutions that have some of those feature sets. The
way they explain it, when they started no one had these features.

------
bifrost
I'd say "well this is what happens when government runs something technical"
but I also watched similar stuff happen at MSN when I was there. Not to say
that the Beeb is like MSN, but I do know that there were a lot of dim bulbs to
go around...

------
nslocum
As a big fan of the Economist, I'm surprised they didn't catch the typo in the
name of the project they're reporting on. "Digitial Media Initiative (DMI)"
Especially since a rudimentary spell-checker would have caught it.

~~~
shitgoose
I guess Economist's CMS is not as sophisticated, as one at BBC

------
justinph
From what I understand (and I work in the public media sphere), this was a
monster CMS project. A way to integrate audio, video, web content, reporters
notes, etc. Is it any wonder a project like that failed?

------
podperson
The only reason failed content management system isn't an oxymoron is that
content management system is three words.

------
amalag
In the US I think noone even blinks an eye at $150 million anymore

~~~
dopamean
It's funny you mention that. My first thought was, "wow, that's a lot of
money." But then I thought, "but how much is it really?" I actually have no
idea how much it should cost for them to do what they were trying to do though
I suspect it is probably a fraction of that $150MM.

~~~
mikeash
For a more concrete perspective, if you assume that a good developer costs
about $150,000/year, then this is one hundred years of development time for a
team of 10 good developers, which is quite a lot.

I don't know just what this thing was supposed to do, so I don't know what the
scope is.

~~~
realtalker
no one working for the BBC in the UK would make that sort of money as a
developer

~~~
matthewmacleod
I don't know about that. I've seen a few tech jobs around the £70-80k mark at
the BBC, which is about $120k. Throw in the additional benefits and it's
probably not far off.

~~~
IanCal
Hah, I wish. I was a senior engineer there and earned less than half that.

------
kailuowang
$150 million roughly equals to 1000 developer years.

------
jamespo
When I was at BBC Online ~10 years back they wasted a couple of million on a
content management system through a "leading" consultancy that never made it
to production.

------
dmragone
I'm surprised more public projects don't run as bounty-style "build x and get
$1 million" type systems. Rather than picking who's going to do what (or
worse, building all in-house), having many different small times competing for
one pot seems like it might be a better way to get a good starting foundation.
Or perhaps that's a crazy idea for something as big as this.

~~~
Finster
That's spec work.

~~~
tomjen3
Yes, that seems quite obvious, which is why it puzzles me that you took the
time to post it.

------
Aqueous
Article doesn't actually tell us what it is, beyond a traditional CMS.

~~~
taylorbuley
More important, _what is in it_. A large difficulty with content management
systems are the data types; pairing down or normalizing metadata while trying
to map old to new.

------
clientbiller
They were trying to build netflix?

------
caycep
Sounds like a typical hospital EMR/EPIC/Cerner/Meditech deployment!

------
ttrreeww
Now they didn't. They "transfered" $152 million of public money into private
hands.

It's a success. Plus, they get to do it again.

