
Massachusetts gas fires: Another technological tragedy - greeneggs
http://bit-player.org/2018/another-technological-tragedy
======
acidburnNSA
"All of these events feel like unnecessary disasters—if we were just a little
smarter, we could have avoided them—but the fires in Lawrence are particularly
tormenting in this respect. With an aircraft 35,000 feet over the ocean, you
can’t simply press Pause when things don’t go right. Likewise a nuclear
reactor has no safe-harbor state; even after you shut down the fission chain
reaction, the core of the reactor generates enough heat to destroy itself. But
Columbia Gas faced no such constraints in Lawrence. "

Pretty interesting comparison. Arguably airplanes can coast down and nuclear
reactors (especially Gen-IV low-pressure ones like sodium metal or molten
salt) can passively cool their nuclear decay heat post shutdown without
external power. But it is true that TMI and Fukushima were not chain reacting,
they were both decay heat cooling problems. For those unfamiliar with this, a
chain reactor immediately drops to about 7% of full power after shutdown and
then exponentially decays to 1% after a day and half a percent after weeks.
Turns out 1% of a gigawatt is still a lot.

On the nuclear topic, I'm annoyed that natural gas accidents that cause deaths
get way lighter media treatment than nuclear accidents like TMI that cause no
deaths.

~~~
t0mas88
Airplanes coast down fine after a (double) engine failure, but they don't
coast nicely when your primary instruments aren't working. The Air France
flight didn't just loose airspeed indication, they got invalid readings and
all associated warnings. Unfortunately that is the common failure mode for
flight instruments, they don't just show nothing, they show wrong values.
Fortunately pilots are trained to know how instruments work on the inside and
what the most likely wrong indications and their causes are.

The problem the author points out is that you cannot "pause" to think about
what information may be correct and which instruments to follow. You have to
react immediately, which is much harder than considering the same decisions
sitting at a desk reading the NTSB report.

If the low speed indication/alert is correct you need to push the nose forward
immediately to keep the aircraft out of a stall, if the high speed
indication/alert is correct you need to pull power to flight-idle and nose up
carefully but immediately to avoid overspeeding the aircraft and risk
structural failure.

~~~
Gravityloss
AFAIK the Air France flight would have been ok if they had not even touched
the controls. They had a lot of altitude. Of course easy to say from the arm
chair.

I'm not sure how easily a regular airline overspeeds or stalls if the controls
are close to a middle position. They should be aerodynamically stable (if
loaded correctly).

For a fighter jet it's a different situation.

~~~
kpil
They were initially close to a high altitude stall, the margins are not that
large at that altitude, so just dropping everything isn't really an option.

The report highlighted training, which I guess is always possible to point
out, but looking at the event I think they could have made more conclusions
regarding how the control system performed counterintuitive by emitting stall
warnings at the time they actually did the right thing, and provides little
feedback when it drops into alternate law making a standard full pullback into
a potentially dangerous move.

------
Animats
There are modern controllers which will detect this situation:

 _" When Open Loop Detection is enabled L.dE, the controller will look for the
power output to be at 100%. Once there, the control will then begin to monitor
the Open Loop Detect Deviation L.dd as it relates to the value entered for the
Open Loop Detect Time L.dt. If the specified time period expires and the
deviation does not occur, an Open Loop Error will be triggered. Once the Open
Loop Error condition exists the control mode will go off and an Open Loop
message will be display. If the process value goes in the opposite direction,
a Reversed Loop message is display. The sensor is likely wired in reverse
polarity."_

If this crew was replacing an old cast iron gas main, it probably predated
such controllers. That's not necessarily a bad thing. A mechanical controller
can work for many decades. It's hard to get that level of uptime from
microprocessors.

The real problem here is "maintenance induced failure". Especially on a system
which is not fully shut down.

[1] [https://www.watlow.com/-/media/documents/user-manuals/pm-
pid...](https://www.watlow.com/-/media/documents/user-manuals/pm-pid-1.ashx)

------
taliesinb
The Chemical Safety Board produces some pretty fascinating videos that
reconstruct the chains of events that led to large industrial accidents in
chemical plants. The videos normally conclude with recommendations for
voluntary actions industry can take to prevent similar incidents in future.
They basically summarize the results of detailed investigations by CSB agents.

The videos are well produced and, apart from those accidents that involve
injury or death, they are quite entertaining if you happen to be in a forensic
frame of mind.

Here’s an example of one of their videos:
[https://youtu.be/_icf-5uoZbc](https://youtu.be/_icf-5uoZbc)

~~~
Scoundreller
I like their videos. Unfortunately, this incident will be investigated by the
Transportation Safety Board.

The report will be thorough, but we may not get a great video out of it :(

~~~
hexane360
The TSB does do videos, but you're right that they're not as polished as the
CSB's.

------
hyperion2010
Oh man. My PhD work is focused on helping people communicate clearly about the
measurements they are making and how to interpret them and where the pitfalls
are in a research setting. Every time I read about something like this I
wonder if I can use what I am working on to help people automatically detect
when the assumptions of their measurement model could be violated. For
example, if you can explicitly model the fact that the measurement is not the
actual value and that they can become decoupled (and can thus brainstorm and
model potential reasons), then you can start to help people realize that the
map (measurements) are not the territory (reality), in ways that aren't just
cute phrases without guidance for practical implementation.

~~~
spongo
I'm curious: within which field are you doing this PhD? I could see you as a
computer scientist, systems designer, political theorist, perhaps an economist
or philosopher.

Is any of your work available to read online?

~~~
hyperion2010
I'm technically in neuroscience, neuroinformatics more specifically, but was
raised by a political theorist. Unfortunately I have a bad habit of writing
code rather than prose, and don't keep a blog or similar, so the first thing
that I could point you to would be my thesis, which is still months away.

~~~
YeGoblynQueenne
Well, hurry up then :)

Seriously, this sounds like a very interesting subject. Best of luck with your
thesis writing and submission (mine's still a couple of years away).

------
jt2190
While they were replacing an old system with a new system, they had
inadvertently created an interim (hybrid?) system that wasn't as safe as
either the old or new:

> The cause of the accident was not a leak or an equipment failure or a design
> flaw or a worker turning the wrong valve. The pressure didn’t just creep up
> beyond safe limits while no one was paying attention; the pressure was
> driven up by the automatic control system meant to keep it in bounds. The
> pressure regulators were "trying" to do the right thing. Sensor readings
> told them the pressure was falling, and so the controllers took corrective
> action to keep the gas flowing to customers. But the feedback loop the
> regulators relied on was not in fact a loop. They were measuring pressure in
> one pipe and pumping gas into another.

------
chb
"I admit to a morbid fascination with stories of technological disaster. I
read NTSB accident reports the way some people consume murder mysteries. The
narratives belong to the genre of tragedy."

The author is not alone. Over a decade ago, there was a theater piece called
"Charlie Victor Romeo" that consisted solely of actors re-enacting cockpit
voice recordings of aircraft that crashed.
[http://charlievictorromeo.com/](http://charlievictorromeo.com/)

------
patcheudor
What I still don't understand is why the pressure regulators and over pressure
valves on the house gas meters didn't kick in? Were the homes involved so old
that the meter protections failed or weren't there in the first place? It
seems that ultimately you've got to have failsafes at the homes themselves.
Without such failsafes, what would stop a bad actor from purposely over
pressurizing a residential branch?

~~~
jacquesm
Those regulators work within a set range of input pressures, and need a
certain delta to work properly. Over-pressure them severely and they will
simply fail, and most such failures will lead to gas escaping from the
regulator body. That is why you will usually find them outside of the premises
they protect so they vent into the outdoors rather than into some enclosed
space.

As for your bad actor: what would he pressurize your lines with? An air
compressor? He'd have to dig up the lines or disconnect them first and gas
from the line would likely escape in quantities large enough to discourage
such tricks.

Just like in theory you could disconnect the mains from a house and then send
a high voltage pulse down the feed lines, in practice pranksters and
miscreants tend to avoid doing stuff that might get them killed instead.

~~~
patcheudor
>What would he pressurize your lines with? An air compressor? He'd have to dig
up the lines or disconnect them first and gas from the line would likely
escape in quantities large enough to discourage such tricks.

On the gas meter on my house anyway, the underground pipe mates at a valve. It
doesn't seem hard or particularly dangerous to shut that valve off, disconnect
the meter, connect whatever, then open the valve again.

~~~
jacquesm
If you leave the valve attached to the mains line then you can access the
house lines, usually there is even a special port for this that you could use
without disconnecting the mains line that is used for leak inspection (they
evacuate the lines and measure the rate of seepage). Once you connect
something in the line and you open the valve again all pilot lights will have
been extinguished and won't re-light due to safeties.

Anyway, if you want to destroy someone's house there are much quicker, less
obvious and easier ways to do so.

------
joe_the_user
_I had believed such a catastrophe was all but impossible. The natural gas
industry has many troubles, including chronic leaks that release millions of
tons of methane into the atmosphere, but I had thought that pressure
regulation was a solved problem._

Yeah, natural gas tech is old and over time has been worked into a multi-layer
system that is close to "inherently safe". I remember years ago trying to
light an old heater the most stupid way possible and only singeing my eyebrows
in the resulting explosion.

But these events just show that if a company _skimps enough_ on needed
maintenance, ignoring constant smaller leaks and the similar despicable
maneuvers done by this gas company, then the overall potential of piping an
explosive gas into a city can be realized and people die and are injured,
building are destroyed and so-forth.

In this context, it seems pretty obvious this isn't a "technological tragedy"
but a "people doing bad things tragedy" and I hope it's obvious people deserve
to go to jail for this and if they don't then other problematic aspect of this
society are then to blame.

~~~
JoeAltmaier
The gas company leaks may be insignificant compared to the way natural gas is
obtained. In particular, the way it isn't. Dakota fracking just burned off the
natural gas, because in the USA its politically incorrect to build pipelines.
Result: terawatts of natural gas burned off at the source or released into the
atmosphere. More damage to the environment than a century of leaks. The
burning flumes could be seen from space, appears as a megacity of light for
years.

I think our political process has completely lost touch with science and
pragmatism.

~~~
joe_the_user
Your argument connecting blowing-off natural gas and the current antipathy to
pipelines today is false and disingenuous.

Essentially, it follows the logic that the only way to keep corporations from
devastating the environment is giving them some positive market incentive to
not do so. But of course, given zero regulations or morality, there's always
going to be an economic incentive to toss some poisons in lakes and some
pollutants in the atmosphere - because some things just aren't useful and if
you have zero-cost disposal, that the (amoral) path of least resistance.

I'd go the opposite rout. Polluting in whatever form should be illegal.
Polluters should be fined or go to jail, whatever is appropriate. "I could
make money with my stuff so I tossed it/burned it/whatever" should never an
acceptable explanation or excuse.

~~~
JoeAltmaier
Uh, its true and the direct cause.

Billions of cubic feet of gas is not a reasonable disposal problem - there's
no place to put it, that isn't worse than burning it. Except a pipeline, where
it would be ultimately burned anyway, but now yielding energy.

------
userbinator
I suspect the remote-sensing aspect of their regulation system was not well
known, and thus the consequences of possibly opening the control loop weren't
taken into consideration. It's a subtle but important point --- and I remember
a friend telling me a story of destroying some very expensive electronics
because the sense line of the power supply came loose.

------
scotty79
What I see in those incidents is severe lack of backup sensors.

Why after switching to new pipe nobody checked/could check the pressure in it?

Why there were no other kind of speed sensors on a plane, simpler, less
accurate, slower but not that vulnerable to icing?

Why in Three Miles Island incident why people couldn't double check if reactor
has water in it or not?

------
dsfyu404ed
Properly installed residential gas piping should be able to handle triple
digit pressures. Appliances should be able to handle double digit pressures
because almost all appliances that can run on NG also can be equipped to run
on propane which outputs at up to 30psi depending on the regulator.
Manufacturers slap a safety factor on top because it's a consumer product and
you don't want someone to blow up their house and sue you because they used
the wrong regulator.

There's a reason some houses went bang and some houses didn't.

I know it's fashionable to blame the megacorp (and it sure looks like they
have plenty of blame in this case) but there's plenty of blame to go around
here. There's a reason some houses went bang and others didn't. I've worked
residential construction in MA and plumbers as a group have a pretty bad
reputation (well earned, at least based on my experience, you should see some
of the corners these guys will cut). I would wager that the houses that went
bang weren't the ones that had their gas pipes done by the apprentice who
still does everything by the book but the "seasoned professional" who knows
exactly what you can get away with when your work only needs to hold 1/2psi.

Like any large accident there are many dominoes that need to be lined up
before they can all be knocked down at once. I think it's foolish to act like
some contractor dutifully carrying out Colombia Gas's faulty work order is the
only cause of this.

------
jrjrjrjr
Even my air compressors have blow off valves, did something get lost in the
advancement of technology from the boiler days ???? it seems to be a pretty
stupid design that doesn't have relief valves.

~~~
wyattpeak
I don't know much about the specifics of gas distribution, but I came across a
comment on an entirely different discussion recently which I think is apropos:

> you should be very suspicious of any conclusion that requires you to assume
> that all the world’s experts have missed something extremely basic.

It seems to me most likely that the answer is simply that releasing natural
gas into the air is an even more dangerous failure mode than overpressure -
natgas is not air in a compressor. But regardless I'd bet you dollars to
doughnuts that, for one reason or another, blow-off valves are a bad idea in
the context.

~~~
drb91
I’m not sure I agree with the statement—humans overlook basic details all the
time. Most computer bugs are “extremely basic”. Multiply times possible
failure points and it’s easy to see how basic flaws can easily cause systemic
failure. Experts DO miss extremely basic things daily. You have to actively
build proccesses to avoid this. Expertise is not enough!

------
LawfulRewrite

      Next year we may see the first models without a 
      steering wheel or a brake pedal—there goes the 
      option of asking the driver (passenger?) to take 
      over.
    

This is bad news for everyone. This is the network effect exploitatively writ
large, such that, surely you can prevent yourself from being inside such a
thing, but as a lone individual, without laws in place to ban such a thing,
you, your children, your family, your friends are all endangered by a
blameless force that everyone can simply shrug at, and point to statistics
claiming that there is even just an incremental improvement over ordinary
human performance.

You can stop yourself from posting selfies on social media, under your real
name. You cannot stop other people from taking pictures, which your
acquaintances discover and tag with your real name.

This is kind of horrific. At least as horrific, or moreso than, current
traffic statistics, because the moral hazard in play is abysmally worse.

There was an effective arms race that took shape with SUVs and road rage in
the late 1990's. I think something similar will take shape, as self driving
cars ramp up. It may surprise some, to find that a self driving incident won't
be accepted as blameless, glitchy software errors. Owners may see themselves
villified directly, for things a car they chose to own, had subsequently
carried out.

I think there are three turns of consequence to a botched self driving car
deployment.

 _One:_ some will choose harm the legal owner as an individual, lawfully or
lawlessly.

 _Two:_ others will harm dealerships, mostly through sabotage.

 _Three:_ overt action against the manufacturers. At all levels, and not
limited to ordinary civil disobedience.

These consequences are nothing to be sniffed at. Aviation shows us that spotty
disasters don't result in civil unrest, but with a human in the loop survival
was incentivized. Automotive deployments like this will be a fire and forget
scenario, and the corporations loosing the reigns, have a demostrable history
of neglect. I wonder if they anticipate, in the rush to market, just how
severely the general public might react to finding their roads on the
receiving end of software glitches that kill their firends and relatives like
deer?

------
ekke
This is fascinating. For anyone interested in a slightly odd but unique and
in-depth view of Systems design and failure, would like to recommend "The
Systems Bible: The Beginner's Guide to Systems Large and Small" by John Gall.

[https://www.amazon.com/Systems-Bible-Beginners-Guide-
Large/d...](https://www.amazon.com/Systems-Bible-Beginners-Guide-
Large/dp/0961825170)

------
jacknews
I realise this is beside the point of the article, but gas distribution seems
like an obsolete and dangerous technology to me.

For cooking, induction, for heating, heat pumps.

~~~
jzwinck
Induction stoves are unpleasant and ineffective to cook on, not to mention
inefficient. And if you want to use a wok, well, you can't.

They are definitely better in appealing to first time buyers who may not know
or care about their shortcomings. They look great and are easy to clean. But
if you are into cooking, gas wins.

~~~
toomuchtodo
> Induction stoves are unpleasant and ineffective to cook on, not to mention
> inefficient. And if you want to use a wok, well, you can't.

With induction, ~90% of the energy from the electricity is used for cooking
while only 40% of energy is used using a gas cooktop. Induction is more
efficient, and you can use a wok if one is purchased that is designed for use
with induction. Induction units can also regulate themselves based on feedback
from the cooking device on the receiving end.

Induction is arguably superior to cooking with gas.

[https://www.consumerreports.org/electric-induction-
ranges/pr...](https://www.consumerreports.org/electric-induction-ranges/pros-
and-cons-of-induction-cooktops-and-ranges/)

[https://www.reviewed.com/ovens/features/induction-101-better...](https://www.reviewed.com/ovens/features/induction-101-better-
cooking-through-science)

> On almost all counts, induction is faster, safer, cleaner, and more
> efficient than either gas or electric. And yes, we've done exhaustive oven
> testing in our labs to support that claim.

~~~
jzwinck
What about the 45% efficiency of a natural gas power plant?

Yes induction is efficient if you only look at the part inside your house. But
most of the losses are outside your house.

If you live in a place with advanced wind or solar power like South Korea then
electric probably is more efficient.

~~~
toomuchtodo
The electric grid tilts cleaner every year. Your stove will burn natural gas
forever. A combined cycle natural gas generator is upwards of 50% efficient,
already more efficient than your gas stove.

It's wasteful to spend money on gas infrastructure when it's clear electrical
distribution is the future of home energy use. Infrastructure dollars are
already in short supply. Just my two cents.

EDIT: You can stockpile energy with batteries, which is the likely outcome
based on how much battery manufacturing capacity is coming online to build
hundreds of thousands of EVs a year (which are also a great buffer for
renewables and electricity in general).

~~~
PhantomGremlin
_It 's wasteful to spend money on gas infrastructure when it's clear
electrical distribution is the future of home energy use._

Maybe for cooking with a wok. But around here most homes are heated with
natural gas. And that takes a lot more energy then heating chicken and
vegetables.

New natural gas furnaces are up to 97% efficient. They're no longer allowed to
sell furnaces that are less than 78% efficient.

[https://www.consumerreports.org/cro/gas-furnaces/buying-
guid...](https://www.consumerreports.org/cro/gas-furnaces/buying-
guide/index.htm)

~~~
toomuchtodo
You're only going to need a gas furnace (vs an efficient electric heat pump)
in climates where it dips below 10F and an air or ground source heat pump
would need to call on auxiliary heat to keep a dwelling warm (unless you live
in a newer home with a very tight envelope and little unassisted air exchange
occurs).

To your point, you can purchase very efficient gas furnaces, but they're more
expensive and require a retrofit of the flue pipe to PVC due to corrosive
properties of high efficiency furnace exhaust (not a concern in new
construction).

[https://www.consumerreports.org/cro/heat-pumps/buying-
guide/...](https://www.consumerreports.org/cro/heat-pumps/buying-
guide/index.htm)

Air source only heat with a heat pump is a slam dunk anywhere south of the
36°30′ parallel, and all of California.

[https://www.currentresults.com/Weather/US/average-state-
temp...](https://www.currentresults.com/Weather/US/average-state-temperatures-
in-winter.php)

------
amluto
I find it bizarre that it’s not required by code for a pressure regulation
station to have a sensor in that station.

I also wonder if the utiliy’s connection between the sensor and controller is
on the internet.

