
Bug Out: Troubleshooting Software Delays on London’s Crossrail Project - zeristor
https://rail.nridigital.com/future_rail_dec19/bug_out_troubleshooting_software_delays_on_london_s_crossrail_project
======
zeristor
By the sounds of it they didn’t appreciate the complexity of the software
problem.

Perhaps with hindsight how could this problem been avoided before time, or
flagged up as a key issue.

It seems the delivery of large infrastructure projects is a dark art, a
civilisation that can deliver that without huge issues has a distinct
advantage.

A large amount of the science in project management, as I understand it,
stemmed from Britain not wanting to repeat delayed and overrun projects from
several decades ago.

I doubt those lessons were forgotten, but did they just reveal deeper issues?

~~~
zig-zag
It seems to me that when project management and politics mix the words rock
and hard place are appropriate.

In the case of Crossrail the original management team also ignored the on-
train software because the trains were bought directly by TfL and it was
treated as someone else's problem.

When the delays to construction started to happen Crossrail management started
to squeeze the later parts of the project and because the software testing was
someone else's problem that was squeezed the most. They probably did not
understand how critical it was to the whole project either.

The CBTC signalling system (Siemens Trainguard MT) has been in use elsewhere
but it may be that some features have never been properly tested because they
were not needed in other locations.

I don't know if the autoreverse feature is used elewhere. Does anyone else
know if they use it in Hong Kong?

~~~
vertex-four
A large part of the problem, at least a few months ago, seemed to revolve
around failures when switching between signalling systems.

~~~
thu2111
There have been a lot of problems with the physical layers too. In the
Heathrow tunnels there need to be multiple signalling systems operating
simultaneously and apparently the radio signals have to be on very close
frequencies. This in turn requires quite advanced signal processing software
in the trains to clean things up and ensure reliable communication.

The political problems are more straightforward. It's a repeat of the TSB
problem. Non engineers often cannot psychologically accept that a project has
an unknowable deadline and cost, so try to force people to estimate and then
stick to their estimates, reality be damned. Crossrail's former management was
so delusional, so psychologically unable to tell politicians and the public
the truth, that they actually started shutting down the project a few months
before construction was due to complete, e.g. by letting go of their
external/internal communications teams. When Mark Wild took over he found he
couldn't even update the project website or email all the contractors because
the people who managed that had been let go - although many stations still
hadn't even been built yet! Further delays occurred because contracts started
expiring and scarce construction teams got reallocated to other projects in
the capital. The entire effort was set up on the assumption that they could
estimate project time and budget a decade in advance!

