
The Internet of Gas Station Tank Gauges - fla
https://community.rapid7.com/community/infosec/blog/2015/01/22/the-internet-of-gas-station-tank-gauges?hn-repost
======
alexcason
In my first job as a Software Developer about 5 or 6 years ago I worked at a
company which connected to forecourt controllers (which managed ATGs, fuel
pumps, etc) on petrol forecourts in the UK, retrieved the data and presented
it to a team of analysts for reporting back to the forecourt owners.

To this day, the most difficult software challenge I have had to tackle was to
retrieve the data from the serial port of a controller called the DOMS 2000.
It had been replaced a number of years earlier by a new model but was still in
wide service.

We had very limited documentation and I spent weeks observing the data being
transmitted from the serial port while performing actions using a simulator.
Eventually I managed to reliably intercept and correctly interpret almost
every command.

I believe, to this day, I may well be one of the foremost experts on the DOMS
2000 on the planet.

~~~
saganus
What would you say was the biggest cause for this to be so difficult? is it
"just because" you didn't have the proper documentation? or was the nature of
the communication protocol itself hard due to its design? or was it perhaps
the fact that both factors would compound terribly?

------
toyg
This is what the NSA and all digital counter-terrorism theatre players should
_really_ do with public money: take a good hard look at the network
infrastructure underpinning critical business segments, and force them to
clean up their act. Be "the common man's pentest agency".

~~~
jessaustin
According to _@War_ , which I just read, NSA has been poking into "vital
national interests" like fuel distribution for years. That they haven't
discovered (or having discovered, urged people to fix) this issue, is kind of
an indictment of their abilities. Of course, having their dirty laundry aired
by contractors isn't a great endorsement either. I wouldn't actually mind if
they helped the various insecure agencies of the federal government improve.
At this point, however, it doesn't seem reasonable for them to bother "the
common man".

~~~
otakucode
The NSA is a bit crippled. They are so intensely afraid of someone (anyone)
figuring out what they do or how they do it that they can not act on any
information they gather. The 9/11 Commission report showed that the NSA had
full access to all of Mohammed Attas communications, knew exactly where he
was, had translated his messages, etc, yet they did nothing about it. When the
CIA requested access to the information they had, the NSA refused. When the
FBI requested access to the information, the NSA refused. The degree to which
they are obsessively paranoid can not be overstated. It's quite possible that
they are very well aware of these weaknesses along with multitudes of others,
but they will never say a word about it for fear that someone might figure out
what they do.

~~~
Aloha
This is possibly the most important comment on this thread.

------
EvanAnderson
I have a Customer who uses this equipment (the Vedeer-Root gauges that the
article describes) in unattended fueling stations. They moved from dial-up
modems in the stations to Internet connections a couple of years ago (to
facilitate modern credit card processing) and, at that time, the vendor who
handles the tank gauges asked me to forward a port through each of the the
stations' firewalls to the gauge. The idea that I was going to terminate a VPN
in each station and accept no unsolicited traffic from the Internet was
completely foreign to them. With that in mind, I'm not at all surprised by
what rapid7 is reporting here. It's likely that those IP addresses where
they're finding tank gauges also have other targets (like video surveillance
systems, environmental monitoring systems, automated fire suppression systems)
receiving Internet traffic on forwarded ports, too.

~~~
jauer
This sort of thing happens everywhere. Grain dryers, wastewater treatment
plants, etc.

You get industry-specific vendors that believe their solution is opaque to
anyone that hasn't received their specific training and they run loose. The
end-user doesn't care because their expert in [whatever automation] tells them
it has a password or proprietary protocol and can't be accessed without a
program that's only given to trusted (industry) insiders.

~~~
otakucode
There are some great presentations on YouTube from the Chaos Communication
Congress and similar conferences about the abyssmal security of PLC-controlled
industry stuff, including prisons. It is exactly as you say. They think their
systems are too obscure for anyone to figure out. Of course, they hire the
cheapest talent they can find to bang the things together in the first place
and constantly look for ways to cut staff and replace developers and security
personnel with newer (cheaper) people all the time, so they never end up with
anything of any significant complexity. Even many places that think their
networks are airgapped turn out not to be. In one prison discussed during one
talk I watched, the prison had a commissary or something run by McDonald's...
someone had connected to the commissaries Internet connection giving the
Internet a direct route into all the insecure systems controlling the prison.

------
jrochkind1
How do you do "responsible disclosure" of a multi-vendor multi-customer
vulnerability like this, with vendors not from the IT industry (or at least
they don't realize they are in that industry, heh), who almost surely have no
easy way to contact them to report vulnerabilities.

Real question. Was this irresponsible disclosure? What should they have done?
(Are those doing the disclosing risking criminal prosecution?). I don't know
the answers.

------
solve
I've been kind of wondering what the business case for "internet of things" is
for years, and now I realize. It's a modern SCADA. No more, no less.

~~~
jfoutz
There are a few different parts. I keep thinking about amazon, facebook and
google's programmable infrastructures. Control needs to be secure, but data
acquisition could be public. I can imagine a gas company that supports surge
pricing like uber.

If all this stuff is visible as a web service, you can take a regular old
programmer and optimize inventory and delivery schedules. I'm sure that
happens now, but i'm not sure how sophisticated it is. There's probably some
value in a car being able to access the octane rating and price of gas on
fillup, which would update your phone with price and performance metrics. I
think there's also an aspect of service discovery you're overlooking.

I guess the point is, the words "internet of things" are goofy and overused,
but it implies being more pervasive, open and standardized than scada.

------
Joona
/r/netsec discussion:
[https://www.reddit.com/r/netsec/comments/2te5h8/the_internet...](https://www.reddit.com/r/netsec/comments/2te5h8/the_internet_of_gas_station_tank_gauges_thousands/)

------
driverdan
Access to 3% of US gas station real time data should be enough to give you an
edge in open markets. It might be enough to find mispriced stocks before
earnings. Not that I'm suggesting anyone should do such a thing.

------
emilburzo
shodanhq search:
[http://www.shodanhq.com/search?q=INVENTORY+port%3A10001](http://www.shodanhq.com/search?q=INVENTORY+port%3A10001)

telnet, ctrl-A, I20100

------
coldcode
You wonder how many other internet connected things are exposed this way. I've
always wondered if there are internet connected traffic light systems (I know
most systems are ancient technology) are vulnerable in the same way.

~~~
tomswartz07
They are. And some Wind Farms (windmill generators).

[http://shodanhq.com](http://shodanhq.com)

Here's a fun search:
[http://www.shodanhq.com/search?q=Anonymous+access+granted](http://www.shodanhq.com/search?q=Anonymous+access+granted)

~~~
jessaustin
That looks like FTP-type sites? What's the point?

~~~
tomswartz07
Sure, the link I had has FTP sites, with absolutely no login required.

What can you do with an FTP site? Host your own phishing site. Modify the
owner's own webpages.

Related to this article: if you can narrow the addresses down to ones that
also have these gas meters, then you can modify their website to say "Hey;
free gas!" and mark their tanks as 'always full'.

The key take-away here is that there is a literal TON of unsecured sites and
multiple ways that something bad could be done.

------
userbinator
If this was only read-only access, it could be useful - especially in times of
shortages.

------
hedgehog
Fun fact: One of these stations can have almost 100,000 gallons of fuel in it.
My cab driver this morning used to be a mechanical engineer working on gas
stations in the US. Apparently modern filling stations typically have 2
(gasoline) or 3 (with diesel) 30,000 gallon tanks. The gauges by themselves
might not be able to cause a big problem but I would hope anything that
involved in the safety of the system is well-built.

------
jdalgetty
I think it's been like this for a long time. Before things like port scanning
there was wardialing.

------
ricardobeat
Broken link. Redirects to [https://securitystreet.jive-mobile.com/#jive-
document?conten...](https://securitystreet.jive-mobile.com/#jive-
document?content=%2Fapi%2Fcore%2Fv2%2Fposts%2F6980) with a blank grey page.

~~~
fla
The problem seems to be on your side. Maybe check your browser plugins ?

    
    
      Request Method:GET https://community.rapid7.com/community/infosec/blog/2015/01/22/the-internet-of-gas-station-tank-gauges?hn-repost
      Status Code:200 OK

~~~
eterm
The ?hn-repost seems especially brazen. And indeed the submission history
shows it was posted 5 days ago and received 2 points.

Timing, a better title, or both?

~~~
fla
Both I guess. A mail from @ycombinator.com asked me for a repost.

------
delinka
"...approximately 5,800 ATGs with TCP port 10001 exposed to the internet and
no password set."

Are we talking about simple monitoring information here, or some kind of
administrative access?

"In our opinion, remote access to the control port of an ATG could provide an
attacker with the ability to reconfigure alarm thresholds, reset the system,
and otherwise disrupt the operation of the fuel tank."

Remote access to a control port would indeed conceivably allow these
activities, no "in our opinion" required. But you haven't told me whether this
is indeed remote access to a control port.

Also, the ISP pie graph just looks like a typical distribution of customers
across ISPs. How useful is this supposed to be?

~~~
bchociej
Well, the docs for these things are online and there are hundreds of opcodes.
Some are less benign than "inventory status". One hopes that any of the "S"
opcodes ("set" as opposed to "inquire" or "I" opcodes) are protected by the "
System RS-232 Security Code", but I'm not about to try to find out.

~~~
jschwartzi
One would also hope that the System RS-232 Security Code is actually unique to
each device and is not simply specified in a manual somewhere.

~~~
jrochkind1
I'm guessing the "Systems RS-232 Security Code" is a 4-digit numeric PIN, or
at any rate a fairly low-entropy password normally, and that there is
absolutely no rate-limiting or monitoring or other guard against a live brute
force attack.

It looks like the documentation's been taken down from the URL included in the
OP. I'm sure it's available elsewhere, but I haven't tried to find it.

------
NKCSS
And now we wait...

While I like stuff like this, it's basically an open invitation to start
messing about.

------
jacquesm
The world of SCADA is terribly insecure, and has been pretty much ever since
the leased lines got replaced by internet connections to save some costs.

If 'the terrorists' really would like to do damage this would be an excellent
target.

------
profinger
The pie charts remind me of this article:

[http://www.businessinsider.com/pie-charts-are-the-
worst-2013...](http://www.businessinsider.com/pie-charts-are-the-worst-2013-6)

------
CheckHook
Seems irresponsible to release this publicly.

~~~
nhaehnle
How else do you get these people to change? We're not talking about a problem
at a single company here. It's a wide spread issue of people apparently being
either unaware or just not caring enough about security.

A well-connected industry insider may be able to get things moving. But if
industry insider exist who care enough, why hasn't this been fixed a long time
ago?

For an outsider, bringing the issue up in public is probably the most
effective way to get this fixed. And it's not like they published a script to
exploit this.

~~~
irq-1
Obama just doubled-down on the CFAA; our government and society doesn't want
to know, and they don't care about 'the most effective way to get this fixed.'

------
PaulHoule
I was in on a conference call last week where they said there are currently no
crypto protocols which are safe and effective for small footprint iot
applications. Of course the NSA is trying to fast track not one but two new
ciphers for this application.

~~~
azdle
I don't totally buy that conclusion. Yes, you won't get any useful security on
an Arduino, but there are plenty of small footprint devices that will do full
TLS. I'm using TI's CC3200 right now with TLS 1.2 using
ECDHE_RSA_WITH_AES_256_CBC_SHA to connect to our APIs and this thing is just a
Cortex M3 doing all the crypto. (There's probably AES accelerator, but I doubt
there is any hardware doing the PKI stuff.)

I mean, sure, it's a more expensive chip, it's two ARM cores plus a WiFi MAC
in a single package, but it's pretty much all the computing and connectivity
you would need for a normal iot kind of application in a less than 1 cm^2.

------
mkramlich
I once wrote software for a Gilbarco gas station pump, basically a "next gen"
model of the kind used by ordinary folks all across the country daily. I too
was surprised then by how much Internet related tech we were expected to use
and interface with. (I was not the overall architect. And I saw the practical
advantages of those choices.)

Let's just say I could smell the security holes.

