
Vulnerable industrial controls directly connected to Internet - wglb
https://arstechnica.com/information-technology/2018/01/the-internet-of-omg-vulnerable-factory-and-power-grid-controls-on-internet/
======
donquichotte
PLC manufacturers and the controls community in general is lagging behind
technologically by twenty years. It is very frustrating for a security-minded,
computer-literate controls engineer to work with decade old hardware and
software.

In a company I worked at, there were PLCs controlling sensitive processes,
including heaters for welding plastic, connected to the company network via
Modbus-over-TCP. A visitor with a guest access, nmap and pymodbus could read
AND SET temperatures of any of the devices. A black hat could easily have
started fires or leaked acid in our production floor.

~~~
WillReplyfFood
<Rant> Worser still, they have even trouble adopting object orientation. The
whole area is stagnant in a Win95 Programming kind of way. Its truly
horrifying.

Worser still, even the people producing the newer tools to allow for OO, do
not really test these tools, so you develop in a environment where you are
basically a beta tester on a budget (TC3 - looking at you).

Nobody dares to move, to not break precious backwards compatability and
devalue the experience of "experts" who for several generations simply did
fall into every trap state-chart based design can provide. Means, every
project runs into complexity walls at the end, there are no automatic proofers
to check the state charts, there are no unit-tests (If i here the poor excuse
-"But there is no hardware to test against" one more time).

In my opinion, this industry is as ripe for disruption as it gets- if you can
generate and update code from a high lvl object and process description here,
and proof reliability and speed + allow for user-code - the industry would
throw money at you.

Half of the projects in this area are not ond budget and not on time. There is
actually not even planning for that- just a factor, which usually is bigger
then 2.0.

Im so sick of the excuses. You cant compare conventional coding and PLC
programming. Yeah, you cant, one moved forward and the other got stuck with
the lowest common denominator - due to lots of non-software guys trying to get
a food in. All these fb lego-players, with no clue what they do...

</Rant>

~~~
taneq
I hear ya. The issue, though, is that the end product has to be at least
readable, if not maintainable, by any half competent electrician, so lowest
common denominator (ie. usually ladder logic) is as abstract as you can go.
Also for most industrial controls, the system is "wide" rather than "deep", in
the sense that almost everything is pretty trivial ('read inputs X and Y, set
output Z if X and not Y') but there are thousands of X and Y. And finally, X
etc. are tied so intimately with the hardware, which by the time the site is
actually built and all the vendor packages integrated is only loosely related
to the original design. So while I absolutely agree that it could be improved
on significantly, the end result will still look (on the surface) quite like
what we have now.

~~~
marcosdumay
> the end product has to be at least readable, if not maintainable, by any
> half competent electrician

You can get that by using Python or similar languages; you can get that by
programming in something like a Haskell DSL and hiding the underlining
libraries; you can get that by creating an actual DSL and interpreting it on
the fly.

But C or C++ state machines manually encoded into structured code is one of
the things that won't get you there.

~~~
taneq
Absolutely not. Most industrial automation engineers, let alone electricians,
are unable to handle Structured Text (the standard IEC text-based programming
language for PLCs), let alone Python. Haskell is not even on the same planet.

And a DSL is the exact opposite of what we're trying to accomplish: An
electrician _with no prior knowledge of the system_ needs to be able to read
the code and understand which switches and sensors need to be turned on in
order for that motor over there to start. Having to learn an entire new custom
language at 3AM when you're losing $700,000 an hour in lost production is not
an option.

(Edit: I should clarify that I'm talking about mining here. High end factory
automation engineers may very well be more familiar with 'traditional'
languages.)

------
9214
My graduate project at university was about air-gap malware, soft TEMPEST and
hacking on industrial PLCs (S7-300, Stuxnet and all that :P).

What terrified me most is not only the state of ICS nowadays (it's pretty much
fucked), but also the fact that cyber-security vendors try to make a profit
from that by shoving their products down the throats of scared managers.

And what @donquchotte says is pretty much true - on facility there I
practiced, all PLC were connected via TeamViewer to company network, passwords
were written on stickers and slapped on monitors. They used Modbus over TCP,
plain-text packets easily viewable with Wireshark.

On the first day I accidentally broke my S7 because of wrongly uploaded
configuration, it took them 3 days to find some special Siemens laptop to
connect to this PLC and reset it. Also tried to play around with Metasploit
and follow the work of Beresford [1,2], but the most trivial attack of mine
was just flood on PLC's IP with random junk - it stopped responding to any
commands.

[1]:
[https://www.youtube.com/watch?v=33kouEKm0zo](https://www.youtube.com/watch?v=33kouEKm0zo)

[2]: [https://github.com/moki-ics/s7-metasploit-
modules](https://github.com/moki-ics/s7-metasploit-modules)

~~~
lima
The first time I tried to automate the S7-1200 web UI, it crashed hard when I
sent a POST requests with an empty body.

These things do not belong anywhere near the internet, patched or not.

~~~
daemin
Pretty much built for someone that knows exactly what they are doing. Like
they've read the manuals cover-to-cover and taken an exam.

~~~
blincoln
An industrial control system that crashes when it receives an unexpected
network request isn't built for someone that knows exactly what they are
doing. It was built _by_ someone who wanted to expend the minimum effort
possible and make contingency-handling someone else's problem.

This isn't just about hacking. A device that crashes hard when it receives a
POST with an empty body may do the same thing if there's a physical glitch
with its connection, or if a cosmic ray flips a bit somewhere. Devices that
control industrial machinery should be more reliable than that.

If this were a hobbyist project, or a toy, that would be one thing, but these
are specialized devices that cost a _lot_ of money. They're fragile
liabilities that don't belong anywhere near a shared network.

~~~
folkhack
> An industrial control system that crashes when it receives an unexpected
> network request isn't built for someone that knows exactly what they are
> doing.

Yeah - I can see even the simplest of network vulnerability scans wreak havoc
on a device like this.

~~~
Parcissons
Its enough to just plug in a cable into two switch ports.

------
rukenshia
It honestly doesn't surprise me that so many PLCs are exposed directly and are
not up to date. Having worked in maintenance at a large factory where we
directly worked with Siemens PLCs, nobody is really prepared to make their PLC
secure. Most people handling and programming those PLCs do not have a deep
understanding of "Internet of things" (in Germany, the term "Industrie 4.0" is
used a lot) but management is happy to jump on that topic.

You usually don't worry about anything other than the task your PLC is
supposed to perform (we're in a factory after all and need to make money, so
better do it quick). I don't think I can remember ever updating the firmware
of any of our PLCs, most of them even rot in a storage room for a couple of
years before even being put in use.

Everything is connected to the local network for monitoring purposes and if it
weren't for another department strictly managing our network I reckon there
would be a lot more of them accessible directly from the internet.

------
upofadown
It's hard to make money out of attacking stuff like this because you can't
normally keep the system broken in a way that is easily fixable when the money
shows up. Once you contact your victim they just fix whatever is broken and
then ignore you. Industrial processes are unreliable enough that one extra
outage might not make that much difference.

It's kind of like making programs to run on Windows 95. You don't have to
worry that much about making your program reliable because no one will notice
a few extra crashes.

~~~
cryptonector
Making money is not the only possible motivation for exploiting vulnerable
industrial systems, though of course as the other reply says, there are ways
to make money indirectly anyways.

First off, the damage to third parties might be extremely severe, especially
in a targeted attack. Think of long-lasting power outages due to damaged
equipment. Or think of environmental damage (e.g., suppose you got a nuclear
power plant to vent radioactive gases, or worse, to have a meltdown).

Second, the damage to the owners of the industrial facility might be severe
even if not to third parties. Here the attacker might benefit if, e.g., they
are a competitor or have shorted the victim's stock.

The first case is of extreme interest to the public, don't you think?

Let's not dismiss concerns about industrial security.

~~~
upofadown
Of course damage is bad but I was specifically addressing the apparent lack of
interest from the criminal set.

For your first case, you failed to provide a possible. motivation. I will
assume "terrorism" as that seems to be brought up as a motivation for most
everything these days. There is more than one problem with this.

First, it is pretty much impossible to prove that an attack over the internet
was actually carried out by any particular entity or even any entity at all.
Things fail on their own. Second, it is usually easy to entirely prevent
another attack. So no ongoing sense of vulnerability.

For your second example, it has always been fairly easy to damage industrial
infrastructure but there have been almost no attacks of this kind. Stock
market movements are not that severe in the case of even serious catastrophe
much less the sort of trivial problems you could cause by attacking industrial
controllers on the net.

------
bluedino
Put this crap all on a segregated network. It's how we did it 20 years ago,
are people still making the same mistakes? Were firewalls not common even 30
years ago?

~~~
snerbles
I'm an engineer for an industrial integrator. No matter how often we tell our
customers to keep the industrial control networks separated, they'll still
find ways to lazily screw their networks up.

A couple years ago at a Fortune 100 company, I dealt with a WinCE panel that
began randomly spewing millions of broadcast UPnP packets - enough to make
Wireshark lag badly. Took us about fifteen minutes to find and unplug the
offending item, and the recommended mitigation from Rockwell Automation was to
just disable UPnP. No patch was available for a problem that's been known
since about 2006 or so, on an HMI product that is still being sold today.

Due to some terrible switch configurations on the part of the customer, the
traffic made it out to the campus network and DoS'd most of their intranet
servers. About 3,000 employees were sent home early for the day because none
of them could read their email.

------
golergka
> A 2013 attempt (attributed to Iranians) to "hack" a dam in upstate New York
> failed, largely because the dam's flood control systems had been broken for
> some time.

This seems more like a satire than real world.

------
sschueller
That reminds me of VNC roulette.
([http://5.230.225.107/](http://5.230.225.107/))

It however appears to be offline but it isn't hard to create your own.

~~~
jakecopp
I just read up on this, it's absolutely terrifying.

So far the news stories seem to have shown unsecured x ray machines, farm
equipment, cctv, maybe oil wells, bank computers, cashflow software, waste
treatment plants, parking lot systems, petrol station storage tanks [1] and a
Swiss water reservoir [1].

It shows everything wrong with the lack of incentives for security!

[1]: [https://securityledger.com/2016/03/vnc-roulette-feasts-on-
in...](https://securityledger.com/2016/03/vnc-roulette-feasts-on-insecure-
industrial-control-systems/)

------
snowwindwaves
I would like to see some authentication and encryption in the communications
protocols like modbus and ethernet IP. Could be as simple as entering the same
phassphrase on the client and the server. If you don't have the passphrase
your requests are dropped or you can't decrypt any packets you might get your
hands on.

I can't see why that wouldn't be achievable. Yes it would take some extra
processing power on the PLCs and would generate a bit more heat.

It would have to be implemented by Schneider and Rockwell, who have been
pushing all responsibility for security to the network level.

At least the latest schneider PLC, the m580, allows for requiring a password
to connect using the programming software[0], and also some ability to
restrict what client IPs can connect[1]. The m580 was released in the last two
years, so everything prior to 2016 doesn't have those features and comes set
up to allow anonymous upload of new firmware by TFTP.

[0]: [https://www.schneider-
electric.ca/en/faqs/FA238215/](https://www.schneider-
electric.ca/en/faqs/FA238215/) [1]: [https://www.schneider-
electric.com/en/faqs/FA282558/](https://www.schneider-
electric.com/en/faqs/FA282558/)

~~~
jdblair
The problem with requiring a plaintext password is that it may slow down
attacks when a device is simply connected to the open internet, but it is
trivial to defeat in all other cases. Modbus RTU is broadcast, so you would
just listen for the password. Modbus TCP can be defeated with ARP stuffing and
man-in-the-middle attack (I've seen this automated for taking over SCADA
systems).

A better solution would be a shared symmetric key that is manually programmed
into all devices. However, this complicates install and maintenance. Its a
hard sell when your controls are installed by contractors paid by the hour.
Pretty soon everyone is setting their 128-bit key to 0x12345678 so nobody
forgets it during emergency maintenance.

~~~
snowwindwaves
Substitute passphrase in my post with shared symmetric key. It is still a
string you have to enter on both ends.

if the underlying mechanisms just work in every case except for key mismatch
and give you a good error message in that case, requiring your contractors to
enter the same string in two devices that they already have to program is
going to be hard for them to justify as a great additional expense.

------
scardine
When I was younger I had this friend (cof, cof) who used to war-dial 0800
numbers (telephones beginning with 0800 are national toll free numbers in
Brazil) looking for modem tones.

The biggest share of toll free numbers with modem tones were fax machines and
but many were 9600 baud terminal for some CLP. I guess these were remote
maintenance accesses for isolated machinery - I wonder if people still do this
(may be an updated version with 3G or 4G).

------
gjvc
As mentioned in the article [https://www.shodan.io/](https://www.shodan.io/)
is in my book, the go-to site for this kind of thing. Anyone aware of any
similar sites?

~~~
iancarroll
[https://censys.io/](https://censys.io/) is also very cool.

------
arca_vorago
I can speak first hand about this. You want to know what the real problem is?
Cheap motherfucking executives and boards. Full stop. I was at one time a one
man miracle show for a company that had 6 branch locations, and was supporting
at any given time ~50 operational PLC's in the field, among other PLC's
internally for other operations...

Now, I've learned a lot about mistakes I've made, and I was never a perfect
senior sysadmin, mostly because I wasn't playing the meeting room
presentations/reports and politics game. So I'm not trying to make myself out
to be perfect in any fashion.

That said, when I requested a full time t2 person. Denied. Requested a full
time t1 person. Denied. I literally had to poach some part timer who was
sweeping floors and train them myself... which did nothing to alleviate my
time crunch dealing with desktop users in order to tackle the substantial
infrastructure security issues, which takes time and thought (did I mention it
was an open office area, so good luck getting good thinking work done...)
Requested contractor structured cabling job (just horizontal)... denied...

Us sysadmins can secure this stuff, mostly through really good network
policies and firewalls, etc, but if management is cost-cutting corners left
and right and would rather hire some contractor-MSP type that does 1/4 the
work at x3 the cost, while refusing to support internal IT teams with the
tools and funding they need to get their job done properly, it's no fucking
wonder insecure systems abound in the industrial world.

This is managements problem, and until the public or others start punishing
them for not seeing IT and security as an investment instead of just a
janitorial cost-sink, it will continue to happen.

That's the reason behind my last burnout. Now I'm happily on break pursuing my
data science degree. I've been on the contractor side too, (dropped out of
college after the Marine Corps to start an IT support company that is still
alive today even after I left) so I've seen the inside of hundreds of
companies, from fortune 500 oil to 20 man lawfirms, so I'm not just pulling
this info from a couple of jobs. I saw it _everywhere_.

A good CIO or CTO should be able to address many of the technical-political
disconnects, but they hardly exist. Go try asking /r/sysadmin how many of them
even get a real budget. Half the time they just have to "ask" and hope they
were convincing enough that management approves. Usually to a CFO or CEO since
the CTO/CIO doesn't exist.

Oh, and of course they all want to immediately jump on the IoT bandwagon but
still don't want to hire the people to get them out of their old technical
debt, much less the new massive technical debt many IoT devices will bring!

Don't even get me started on the "big data" issues IoT and many PLC type
devices bring.

edit: The key to the increase of this issue is the direct LTE connections on
the PLC's, whereas you used to have a satcon to a router and everything was
internal from there, that's less true these days. Even so, you still have
issues with the security of that edge.

~~~
junkcollector
To touch on this a bit. To be a good controls guy that does PLC programming
you need to have some mechanical background to understand the equipment you
are controlling, you need to know enough electrical to do the wiring and parse
what your sensors are telling you and you need to be a fair shake at
programming/comm protocols/logic. Oh, and most control loop implementations
are actually sets of differential equations with DSP inputs so having a
knowledge in that stuff helps a lot when you get into the precision stuff.

If you go look at the current postings for these roles you'll find that the
average salary is between $15-$25/hr, you'll be on call 24/7, and you have to
work physically demanding jobs in relatively harsh environments. I've seen a
few outfits that don't even offer to pay their employees for commuting time
between sites, only for when they are actually sitting down and working on the
boards.

------
kuon
What is the opinion on IPv6 on this kind of problem? I mean, IPv4 being often
behind a NAT, I guess a LOT of devices that would be exposed are protected by
a NAT.

But in an IPv6 world where NAT is no longer required, don't you fear that a
lot of device might get exposed?

I know it will be impossible to scan the whole address space, but I am sure
other means of discovering IPs will arise.

~~~
zAy0LfpBZLC8mAC
NAT does not prevent inbound connections.

A stateful packet filter does.

Stateful packet filters work with both IPv4 and IPv6.

~~~
kuon
I know, but I meant how NAT with IPv4 was deployed by ISP (at least in
europe), which is a single IP with all inbound connection blocked.

I guess it will depend on how ISP will deploy IPv6, but where I saw it
deployed, the ISP did not provide NAT by default and all allocated IP were
routable and inbound traffic was not blocked.

~~~
zAy0LfpBZLC8mAC
> I know, but I meant how NAT with IPv4 was deployed by ISP (at least in
> europe), which is a single IP with all inbound connection blocked.

The single IP does not prevent inbound connections. The stateful packetfilter
does. This has nothing to do with NAT.

Also, NAT is not usually done by the ISP, but by the customer's router.

~~~
kuon
Most of the time the customer router is controlled by the ISP. And I don't see
why you are so pedantic on this, the point is about how NAT is generally
deployed, and for consumers, this is with a single IP, stateful firewall and
UPnP for port forwarding. And this might change with IPv6.

~~~
zAy0LfpBZLC8mAC
No, that is how IP(v4) is commonly deployed: With a single IP, NAT, UPnP for
port forwarding, and a stateful firewall.

Yes, that could change with IPv6. But it still doesn't have anything to do
with NAT. Just because stateful firewalling happens on the same device as NAT,
doesn't somehow make it a property of NAT. There also usually is a web
interface on the same device. Now, with IPv6 that could change as well. Does
that mean that web interfaces are a property of NAT? Or that it would be
sensible to say "NAT is commonly deployed with a web interface"?

------
daemin
I guess there have not been any large scale or very public attacks against
these computers because the people that would do this would be an enemy of
every state that has manufacturing.

The disruption may be temporary as things would recover quickly, and the
perpetrators would be hunted down by law enforcement and thrown into jail for
a long time.

------
splike
I wonder if this was in response to the top comment on this HN post from 3
days ago
[https://news.ycombinator.com/item?id=16237026](https://news.ycombinator.com/item?id=16237026)

------
jboxee
Is this related to this recent finding?

[https://news.ycombinator.com/item?id=16237026](https://news.ycombinator.com/item?id=16237026)

------
chaz6
How do they know they aren't honeypots?

~~~
paulie_a
Simple answer because they are not that clever

~~~
pmlnr
Speaking from experience, Siemens is the company where they ask you what is
SQLite when you mention that a half-binary xml is not necessarily the best
format these days.

~~~
CapacitorSet
That sounds weirdly familiar. (Hello S7 project files?)

~~~
pmlnr
Something related, still ACX.

------
m3kw9
I wouldn’t be surprised if there are some controllers work from home and use
VNC to access controls!

