
Unauthorized code in Juniper ScreenOS allows for administrative access - tshtf
https://forums.juniper.net/t5/Security-Incident-Response/Important-Announcement-about-ScreenOS/ba-p/285554
======
adrtessier
Hoo boy, most of these things don't worry me, but this one does.

I'm semi-responsible for some Juniper gear, thankfully all Junos (BSD) based,
but I no longer trust any of it if this is malicious injection vs. a bad
review. However, what the hell can I do? I can't audit the code. I trusted
Juniper, and now I'm stuck with that trust being burned. Running to any other
proprietary network vendor is just as uncertain.

If Junos gets a bulletin, I have a lot of work on my hands very soon, as do a
good chunk of service providers. I remember there being rumors of a certain
three-letter agency saying they had some type of exploit for the Cisco ASA as
well; I wonder if it was something this deep, vs. just a run of the mill RCE
vuln.

This is one more reason to use open-source products for actually security-
sensitive systems, maintain a good amount of defense in depth, and do a little
bit of auditing of the code you're using yourself. More often than not these
days, it sure pays to be paranoid.

EDIT: At the same time, this also really makes me respect Juniper more than I
have previously. A company that finds this internally, on their own audit,
could have patched it silently and said nothing about it to anybody. It
probably would have been better for them PR-wise. The honesty is worth me not
jumping ship to another (probably compromised) proprietary vendor, but you
betcha if I can get away with it, I'll run something open-source and community
audited when I can.

~~~
lmm
I've always regarded networking equipment as outside my security boundary. All
it does is forward packets to the right places; an attacker can deny service
by shutting that down or sending them to the wrong place, but nothing else.
All my connections are encrypted and authenticated at a higher level.

Do you terminate SSL or something on yours? Or have open unauthenticated
services running on your internal network? If not, what's the actual threat
here?

~~~
technion
\- Plenty of services that can't adequately be secured any other way are often
mitigated by restricting access to VPN users coming in over a network device.
I appreciate that "just secure the service" is supposed to be the best
practice, but when you're talking about things like IPMI interfaces or SCADA
devices the alternatives approache zero

\- Controlling the networking equipment can open you up to things like
sslstrip

~~~
rsync
"things like IPMI interfaces or SCADA devices the alternatives approache zero"

My strategy with IPMI has been to assign IPMI non-routable, private IP
addresses, then block that address space at the interior of the network (which
is sort of redundant) and then require folks to SSH onto an interior host and
connect to IPMI that way.

I would be very interested in, and receptive to, criticisms of this model ...

~~~
lmm
The argument there would be that it's very hard to secure all access to a
network - anyone compromising any network device, or e.g. physical access to
the cable runs in the building, then had access to IPMI.

In a high security situation I'd keep the IPMI network physically segregated,
with a small number of machines acting as access to it. Or maybe connect IPMI
only within each (locked) rack, and require using something like ansible if
you want to perform an operation across more than one rack. Whether the
cost/benefit fits for your circumstances is another question of course.

------
nathanb
I work for a company which makes network devices. We've detected many hostile
intrusions in our network. If you make hardware or software that runs in
enterprise datacenters, someone is surely going to be trying to steal your
source code to find exploits and possibly put backdoors in.

We use multi-factor authentication just to get in the corporate network and a
separate, airlocked engineering network to store our IP. From what I've talked
to from my colleagues at other major device manufacturers, this is becoming
the industry standard (seven years ago I scoffed at Ericsson's paranoia for
having a sequestered engineering network. Turns out they just saw the attacks
earlier than we did).

In our case, doesn't seem to be the NSA. Looks more like China. Could easily
be either one, or yet another party. This is the world we live in.

~~~
ghshephard
When I set up the Stock Options system at Netscape (as the Desktop Support
guy) back in 1997, It consisted of two computers, connected to each other via
a switch, in a Locked room, with a wall all the way to the ceiling to reduce
false-ceiling access, with that room also located inside the Secure Legal
Office Space. Systems were backed up daily by the users, using encrypted
backups to Zip Drives.

It's interesting how when you don't know what the hell you are doing, you
sometimes do something reasonably secure by pure happenstance. (Also, I had
probably read too much Bruce Schneier when I was a teenager.)

~~~
beagle3
What exactly did the Stock Options system do? Was it the registry of options?
Did the accounting department have such a secure setup?

~~~
ghshephard
I'm not 100% familiar with what precisely they were tracking. The software was
called "Equity Edge", and it involved employee stock options. I do recall
contacting their support organization when I realized the data files they were
storing on the hard drives didn't seem to be encrypted (the systems were
Windows 95). Netscape had two employees whose sole job seemed to be the care
and feeding (and data integrity) of this system.

Data was sent to the Accounting Department (and other Lawyers) on Printouts.

------
tptacek
Holy shit. You thought you'd read the important security news of the day on
HN, but, nope: Netscreen VPNs were backdoored!

~~~
dsacco
Yeah but Thomas, this isn't a million dollar bug.

~~~
xer0x
Points.

------
snowpanda
This might not mean anything, but NetScreen-5GT 6.2.0r15 (The first affected
version) was the first release with a SHA-1 sum. April 2015 is the first
archive of this page I could find.[0]

I wonder if the reasoning behind the SHA-1 is (possibly) that they were
starting to notice some strange activity.

I applaud them for disclosing all of this. That could not have been an easy
thing to have to do.

[0]
[https://web.archive.org/web/20150422145246/http://www.junipe...](https://web.archive.org/web/20150422145246/http://www.juniper.net/support/products/screenos/ns5gt/6.2/#sw)

Archive link:

[https://archive.is/jdw13](https://archive.is/jdw13)

------
late2part
Your government is illegaly modifying commercial software so they can spy on
you without warrants. Your government is doing this through illegal breaking
and entering, or by paying people to defraud their employers, or by using
extortion to force people do these illegal acts.

Your government put CISA (Warrantless Wiretaps) into the budget bill.

Wake up.

Vote against any elected official that supports these things. Tell your
elected officials you want your privacy and you will work to put them out of
office if they don't defend it.

~~~
rmc
They aren't _my_ government. This is the actions of a foreign government.

~~~
awjr
How do you know?

~~~
rmc
Because my government is bad at technology.

------
reiger
[http://arstechnica.com/security/2015/12/unauthorized-code-
in...](http://arstechnica.com/security/2015/12/unauthorized-code-in-juniper-
firewalls-decrypts-encrypted-vpn-traffic/)

~~~
frik
_" It's not clear how the code got there or how long it has been there. An
advisory published by the company said that NetScreen firewalls using ScreenOS
6.2.0r15 through 6.2.0r18 and 6.3.0r12 through 6.3.0r20 are affected and
require immediate patching"_

If the use a Subversion/Git code repo to maintain their codebase, they should
be able to track down who wrote the code and when.

~~~
creshal
export GIT_AUTHOR_NAME="Barack Obama"

export GIT_AUTHOR_EMAIL="potus@whitehouse.gov"

~~~
zaphirplane
That's the who not the when or how

~~~
radialbrain
For the when:

GIT_COMMITTER_DATE="Tue Dec 8 12:33:03 2015 +0000" git commit --date="Tue Dec
8 12:33:03 2015 +0000"

That will change the commit and author dates.

~~~
mikeash
Shouldn't the server log incoming pushes so you can find when the commit
arrived, not just when the commit claims to have been made?

Maybe they _don 't_, but this seems like something they ought to do.

~~~
beagle3
You could have them do it, but it's just going deeper down the rabbit hole.
The eventual question is "who/what do you trust?" \- maybe it was the git
server that got pwned?

A PGP-signed commit with a key generated on a smartcard (and never exposed) is
a little better, but ... Someone pwned RSA before, and I'd be surprised if
Gemalto and Yubico (just two examples) don't have some Three-Letter-Agency
backdoor (and .. I'm sure those TLAs have equipment that can read modern
smartcards).

------
geofft
Should we take that as a dysphemism for "code that wasn't security-reviewed by
someone who should have been Cc'd on the review" instead of the much more
obvious "malicious commit, either from an employee or an attacker"?

~~~
hcf
The former would be authorized but unreviewed, unauthorized seems to mean they
got owned.

~~~
mhurron
I doubt they would be so open if that were the issue. My bet is unreviewed
code or including an open source library without proper oversight.

A revelation that their code repositories have been compromised and injected
code has been released would call into question all of Junipers products.

~~~
dogma1138
If it was an open source library that was imported there would be a link to
the CVE affecting that library most likely and that CVE would've been updated
to announce that it affects additional systems (JUNOS/ScreenOS) this would
usually not trigger a completely new CVE from being issues (i.e. Heartbleed
and Shellshock which got updated for weeks and even months when new systems
were discovered to be affected). The "unauthorized code" also introduced 2
separate and unrelated vulnerabilities one which allows you to bypass the
authentication by some means (logs you in as a SYSTEM user), and another which
allows you to decrypt VPN traffic. The overall phrasing (knowledgeable
attacker), the fact that a fresh CVE was issued, and the fact that 2 unrelated
but very specific vulnerabilities were introduced into the system makes me
think that this was more intentional than just an issue with importing code
from a 3rd party.

~~~
mhurron
Then all Juniper code should be thought of as tainted. It's really as simple
as that. Juniper has announced that everything they have released can not be
trusted.

EDIT: > it was an open source library that was imported there would be a link
to the CVE affecting that library

That would only be if it was a error in the library that caused this and not
the way it was used.

I just do not see Juniper coming out and so casually saying, "Our source code
was clearly compromised, and this is the one instance of them changing our
released code that we found."

~~~
dogma1138
If it was poor implementation that's not unauthorized code. Also I don't
remember the last time that "unauthorized code" was used to describe the cause
of a vulnerability, and code being committed without undergoing the full code
review and compliance process is quite a common occurrence and also a common
cause for some security vulnerabilities, especially ones that are easily
caught by static code analysis.

The phrasing, the very specific nature of the vulnerabilities, the
"knowledgeable attacker" requirement which means that you can't just fuzz your
way into it just like any other zero-day and the fact that some of the Snowden
documents that were published mention an NSA specific backdoor for Juniper
firewalls means make me think that this wasn't an internal process failure.

If the process would've failed we would've gotten an advisory at the most
without any specifics, the fact that they've intentionally mentioned that
unauthorized code managed to get there is almost like a canary, they said that
they've been breached without effectively saying that.

------
yuhong
An attempt at a diff:
[https://gist.github.com/hdm/107614ea292e856faa81](https://gist.github.com/hdm/107614ea292e856faa81)

------
marshray
Isn't this somewhat unprecedented: a major vendor announced their source base
has been actively compromised by a malicious party?

If so, this is a potentially industry-changing event.

~~~
nostrademons
Not really:

[https://en.wikipedia.org/wiki/Operation_Aurora](https://en.wikipedia.org/wiki/Operation_Aurora)

[http://www.nytimes.com/2014/03/23/world/asia/nsa-breached-
ch...](http://www.nytimes.com/2014/03/23/world/asia/nsa-breached-chinese-
servers-seen-as-spy-peril.html)

~~~
DCoder
There was also
[https://lwn.net/Articles/57135/](https://lwn.net/Articles/57135/)

~~~
cypherpunks01
That is a certainly a compromise attempt, but I wouldn't call that 'actively
compromised'—looks like a secondary CVS mirror repo was pushed to, and it was
noticed "quickly". No damage done there.

------
blacksmith_tb
"Unauthorized" seems strangely vague - does that suggest something was
released without code review, or that an attacker actually managed to get
something into their codebase?

~~~
mpeg
Lots of network devices don't do any kind of real validation of the updates
they get from a remote server, beyond using SSL which only checks that the
origin is correct.

A while ago I accidentally stumbled upon the file servers that served updates
for a major router manufacturer. The host vulnerability itself was reported
and fixed, but I doubt they told their clients (the router company) anything.

In that case, anyone who found the vulnerability could have modified the
binary blobs, I wonder if this is a similar case.

~~~
jlgaddis
It sounds much more like someone made their own "contribution" to the source
code.

For JunOS, updates are digitally signed (and verified when installing) so any
modifications would have had to occur before the images were built, but I
really can't remember if ScreenOS updates are signed or not (it's been a long
time).

------
firebones
Sounds more like an insider committing an obfuscated exploit, or a plausibly
deniable bug.

Props to Juniper for owning up. Unless that is part of the con...

------
kccqzy
Can someone explain how the code is able to decrypt VPN traffic? I'm no expert
on VPNs but I thought they provide end-to-end security and the protocols could
detect tampering?

~~~
cheeseprocedure
There is speculation it compromised the cryptography used for VPN traffic,
enabling someone with access to that traffic to decrypt it through brute
force:

[https://twitter.com/matthew_d_green/status/67787100435437158...](https://twitter.com/matthew_d_green/status/677871004354371584)

------
chiph
There's no proof at this stage that a government agency is behind this. It
could easily have been an employee inserting this code in an attempt to
blackmail the firm, or perhaps to gain financial advantage by learning
corporate secrets that would allow them to beat the stock market.

Hopefully there are source-control logs that show when this alteration was
made and by whom, but given how hardware companies treat software I doubt it.

------
hdmoore
If anyone is interested, I have been working on diffing the code for the
backdoored vs patched versions: [https://github.com/hdm/juniper-
cve-2015-7755](https://github.com/hdm/juniper-cve-2015-7755)

------
lifeisstillgood
tl;dr Project aurora was a series of attacks in 2010/11 where Chinese attacked
the SCM of major companies. Juniper _may_ have had its SCM polluted without
going through normal review processes

And so signing code patches looks like a good idea.

~~~
fabulist
Juniper hasn't released nearly enough details to conclude that this was
related to Operation Aurora.

~~~
lifeisstillgood
I meant that the implication seems to be the code got there not by an
authorised committer adding bad code, but by external party adjusting the SCM
- the conversation seemed to be heading off into wilds of "code review
practises"

------
secfirstmd
I think a lot of the focus here is on technical penetration of organisations.
Much easier in many cases to just do a human intelligence penetration of an
organisation to put the code in place.

------
rmdoss
TLDR: ScreenOS was backdoored.

------
ossreality
Can someone detail how widely Juniper/ScreenOS is deployed in the wild?

~~~
NetStrikeForce
It was a best seller before Juniper started using JunOS on their new firewalls
(SRX) and it continued for a while due to being rock solid and their ability
to do pretty much anything in a fairly easy way.

I remember working with my first SRX in 2009 and nowadays I still talk to
customers rocking ScreenOS devices. I've lost any trust on them due to this
news, of course, but boy their OS and feature set was awesome... I did like
them so much that I bought a tiny NetScreen 5GT a few months ago for £10 only
to gather dust on my desk for weeks, until I gave it away to a colleague that
had a better use for it :)

Anyway, I derailed. They were everywhere and they're probably still around in
many SMBs.

