
How a Single Mechanical Failure Sparked 625 MTA Delays (2016) - CaptainZapp
http://nymag.com/daily/intelligencer/2016/02/mta-one-day-625-delays.html
======
THE_PUN_STOPS
“‘For 67 percent of the railroad’ — that is, every lettered train line except
the L — ‘we don’t actually see train movement or control any signals and
switches from the control center.’ Instead, they do it the same way they’ve
been doing it for decades: train crews communicating by radio with a
dispatcher. If there’s a delay, the dispatcher phones it in on the ‘6 wire,’
an open party line, and awaits instructions.”

Wow. I knew the system was old and rickety but I didn’t realize it was this
old and rickety.

~~~
lb1lf
The tram line in Trondheim, Norway (orders of magnitude smaller than anything
operated by the MTA, but still...) had a signaling system operated by the
drivers on a single-track line - on a pole next to the track there was a small
bucket.

When you rode uphill, you grabbed a token as the tram passed. Empty bucket
meant thou shalt not pass.

When you came back down again (leaving a vacant track behind), you dumped the
token back in the bucket.

This system was in use at least until I left town in 2005.

~~~
jacquesm
I love the simplicity of that solution, but the hacker in me wonders how hard
it would be to duplicate the token...

~~~
TeMPOraL
A cracker in me wonders what would happen if some random person just came up
and stole the token...

~~~
lb1lf
I believe the assessment at the time was that the speed was so low on this
track anyway that a missing token would be an inconvenience more than an
outright hazard - tram drivers getting into staring contests and the like.
("You back off!" "No, you back off!")

------
quietbritishjim
> “Their nightmare scenario,” says Gene Russianoff of the Straphangers
> Campaign, the venerable subway gadfly group, “is there are these people who
> are like bouncers who are standing in front of the staircase at Grand
> Central and put up a felt line and say, ‘You can’t go down for 15 minutes,
> because it’s too crowded down there.’ And it’s not an insane, paranormal
> phenomenon. It happens in London. And it would be terrible.”

I've had this happen while commuting in London, and it is definitely annoying.
But it is not "terrible" and certainly not "insane" or a "nightmare scenario".
Surely it is better than the vicious cycle of overcrowding forcing longer
dwell times forcing more overcrowding? And that's not even taking into account
the obvious danger of overcrowded platforms, with people at the back wanting
to get to the front, and people at the front teetering over electrified rails
as a train approaches.

~~~
Jesus_Jones
I've always wondered, why don't they have fences a few feet from the edge of
the platform, with many open gates? Then you could stand at the fence without
being able to be pushed into the train's path. Since they aren't on the edge,
you could get off the train okay. It would reduce free movement but would not
matter unless there was heavy traffic - and in that case, it would be
welcomed, by me at least.

Let's see if I can do this with ascii art, just for fun:

    
    
       [trains, edge of tracks below]

==========================================================

\--- -------- -------- -------- ------- fence line

    
    
       (people, crowds .................................)

~~~
avianlyric
In London TFL is looking to deploy more track side doors, which are similar.

I can’t speak for other cities, but in London many platforms are simply too
narrow for a few feet of the platform to be cordoned off.

Additionally the other issue with overcrowding is emergency and disaster
response. If the fire alarm goes off, you need to get all those people too the
surface, without anyone being crushed or trampled (look at the Hillsborough
disaster to see what happens when you get to wrong). I suspect that people
being pushed onto the track is minor concern compared to what happens if
people start to panic.

~~~
CaptainZapp
This article would justify a submission of its own but is so spot-on in reply
to your comment; here we go:

[https://www.newyorker.com/magazine/2011/02/07/crush-
point](https://www.newyorker.com/magazine/2011/02/07/crush-point)

------
Animats
There are only gap fillers at two NYC subway stations, Union Square and old
South Ferry. Since the new South Ferry station is open again (flooded in
2012), the old curved platform at South Ferry isn't being used any more.

Those gap fillers are a tough piece of mechanical engineering. They have
nearly a foot of travel, because they're on a sharply curved section of track.
Some other subway systems use gap fillers, but they're usually much smaller,
and are sometimes just passive rubber blocks.

Gap fillers get banged by the train as it leaves the station, rather than
being retracted before the train moves. That's to prevent dumping people onto
the tracks. So gap fillers have to be very tough, and they have to be
pushable. That means no electric ball screws or hydraulics; they have to be
pneumatic. They have to work at temperatures from below freezing to above
100F. They have to resist ice. They're stepped on by people and people can
drop things through the slats, so they have a potential jamming problem.
They're long sliding devices, always a headache. They're long enough that
being pushed near one end when the air cylinders don't respond fast enough may
force them back misaligned and jam them, which is probably the failure mode
that caused the problem here.

Union Square's gap fillers are a one-off; there are no other copies of that
design. If there were lots of those things, different designs would be tried
and tested, as is normal for rugged mechanical design.

------
nerdponx
There are lots of small improvements that could be made to this system.
Passengers are often left in the dark about what is happening, and so can't
make informed decisions about which train to take, or whether to take the
train at all. Announcements are rare except when trains are to be rerouted.
The overhead signs that report arrival time estimates and delays are not
always visible from the turnstile. And on the Lexington Ave line local track,
most of those signs do not show the express train times, so transferring from
the local to the express is always a gamble.

~~~
luckyt
There's the Transit app, which tells you how long you have to wait for the
next train. If more people used it, people would know about delays before
entering the platform, and only take the train if they absolutely have to.

~~~
nerdponx
The Transit app appears to use the official train schedule, which is useless
in case of delays. The real improvement is the MTA's own Subway Time app.

------
bob_theslob646
>MTA executives are naturally defensive about the criticism. They argue that,
unlike in the ’70s, the current problems are a result of their own success —
the subways are more popular than ever and therefore more crowded.

Aren't these the same people that got caught destroying data so that people
could not check the efficiency of management?

~~~
LeifCarrotson
The current problems are a result of the city's success _in spite of the MTA_.
Tourism and commuting population have both increased more than the ridership
on the subway, which shows that the MTA is holding the city back.

------
dfraser992
I have recently been doing a literature survey of research into train
timetabling and re-scheduling for a potential contract (using machine learning
et al). Dealing with train delays in real time is not an easy problem to
solve; it's pretty much a NP complete problem to find the most optimal
solution. So 'the best so far' is the goal given time constraints. So I'm not
surprised at all about how one issue went and cascaded. Managing real world
train networks is pretty complicated because of the interconnection of so many
things - the train schedules, the crew schedules, managing the rolling stock,
the passengers... and the profit/loss of the train company.

~~~
jloughry
If you've not already found it, _The Visual Display of Quantitative
Information_ by Edward Tufte (Graphics Press, 1991) talks a lot about the
representation of graphical train schedules by the slope of a line between
stations; the intersection of two lines on the graph shows the time and place
where two trains meet going in opposite directions. This was in the nineteenth
century.

------
pavel_lishin
I've had a lot of luck using this unofficial web-app:
[https://wheresthefuckingtrain.com/](https://wheresthefuckingtrain.com/)

~~~
iooi
Interesting, is it using different data than the official subway time app[1]?

I use a script to check the times the trains arrive so I can leave the office
at an optimal time.

Something like: echo " #!/bin/bash

curl -s [http://traintimelb-367443097.us-
east-1.elb.amazonaws.com/get...](http://traintimelb-367443097.us-
east-1.elb.amazonaws.com/getTime/W/R18) | jq '.direction1.times' " >
/usr/local/bin/train && chmod +x /usr/local/bin/train

It works really well.

[1] [http://subwaytime.mta.info/](http://subwaytime.mta.info/)

~~~
pavel_lishin
Good question - I have no idea! I did find it funny that it's almost always
returning data, even when the overhead signs are blank.

(Also, howdy, neighbor - I leave the office and go to the 23rd street stop.)

------
dkarl
What strikes me is the massive difference in functionality between the system
when it is running smoothly and when it hits a snag, and how people consider
the very best case to be baseline against which performance should be judged.
Perhaps operations can be improved, but pending those improvements, the system
is essentially running overoptimistically, pretending that its performance
when everything is working right reflects its true capability more than its
usual performance does. Many people have made this point, but the quote I have
to hand is from Rich Hickey: "In the real world, failures are all the time. A
large system is in a state of partial failure almost continuously." In a
system like this, there's no point in improving the best case or even the most
common case. In fact, it can be counterproductive, because it makes it hard
for riders to figure out how to plan their trips, and when they do figure out
the best way to use the system (getting to work an hour early many days to
ensure you're consistently on time) they resent it. Also, if you perform
acceptably most of the time, people are going to assume that you're just a few
tweaks or a modest increase in capacity away from performing acceptably _all_
the time, which can be very far from the truth.

------
closeparen
One thing I don’t see acknowledged in the enthusiasm for rail-based public
transit is its propensity for cascading failure. A road network has the very
nice property that a disabled vehicle can coast off the road, or only block a
subset of lanes, leading to a “graceful” degradation and creating backpressure
that encourages other drivers to take different routes, which exist. It isn’t
fun, but it’s a pretty amazing resilience story compared to alternatives. Many
commodity parts, failing in ways that the system as a whole recovers from. You
would think software engineers would be more positive about that.

We build train systems in such a way that if any component fails, the entire
network is fucked. There’s only one way to get between A and B, and if it’s
blocked by an emergency situation or the failure of some bespoke part, no
trains will pass until the situation is resolved. From a distributed systems
perspective, that’s horrifically bad engineering.

Maybe there’s some way to make rail networks more road-like in this way, or to
build better bus systems that people will enjoy and embrace, but as it stands,
the differences in failure modes motivate drivers to stay in their cars.

~~~
avianlyric
Rail based transport on cities makes a lot of sense. In cities usually the
only way to significantly increase transport capacity it to dig tunnels.
Building on the surface is just too difficult.

When you build tunnels your cost proportional to the volume of earth you
remove, which means your cost is the square of your tunnel diameter. Double
the diameter, quadruple the cost.

This is where rail shines. Rail let’s you run faster, longer, vehicles in much
smaller spaces, because the rails guarantee you won’t collide with the walls
or each other. Just look at the London tube, with trains only slightly smaller
than the tunnels. This means smaller _cheaper_ tunnels, so you can build more
of them.

In this system the tunnels already eliminate any real ability to reroute, so
you may as well put the biggest vehicles in there and maintain them properly.

~~~
closeparen
A system experiencing cascading failure has no capacity, and at this point it
should be considered engineering malpractice to imagine that American
democratic institutions will do the maintenance on _anything_ , in the face of
overwhelming evidence to the contrary.

~~~
avianlyric
I’m not sure what your point is. It’s not viable to build more roads, so
tunnels it must be. Tunnels are expensive, so rail is a requirement.
Maintenance is must, regardless of political will.

Saying that US institutions can’t/won’t maintain things, thus don’t build
things that need maintenance is just unhelpful. You might as well say we
should all give up now and go home.

