

The World's Worst Penetration Test Report by #ScumbagPenTester - zebra
http://it.toolbox.com/blogs/securitymonkey/the-worlds-worst-penetration-test-report-by-scumbagpentester-58747

======
runjake
This is the world's worst writeup of the world's worst penetration test
report.

First, this article is on some linkbait site and it gets posted to HN.

Second, the author complains that these kinds of outfits haven't been "weeded
out" of the industry. Anyone with half a mind in the security industry knows
that it is filled with charlatans and snake oil. That the author finds this
surprising seems to indicate they're a bit green. Nobody's been weeded out
from existence. Thus, a little caution and common sense is required.

I suspect the author's associate probably did a Google search for "pentester"
and stumbled upon this outfit. Or worse, he read the stunning Google reviews
for this outfit.

Also, as a US company, don't use an Indian company for a task so sensitive,
and where it's vital that language be precise. Maybe that's politically-
incorrect, but its the truth. If you're an Indian company, by all means ignore
this advice.

------
alcari
> Microsoft IIS susceptible to CVE-XXXXXXXX. Recommend applying accordingly
> patch.

> Another almost good finding - but according to the appendix, this host is a
> RHEL 5.x box. Those sysadmins - finding ways to run IIS on linux!!
> Brilliant!

I used to see crap like this all the time coming out of PCI compliance audits.
The hosts I dealt with ran Apache on RHEL, too. We'd point this out, and
they'd "accept" that "fix."

~~~
louthy
I've seen similar to this with our pen-test firm (who apparently do pen-tests
for the UK NHS) - we're a C#/ASP.NET shop, running on Windows servers and they
know this.

For some reason when I examine our logs after each test I see attempts at
access to /usr etc.

It does make me wonder if all so called pen-test companies are just one
massive scam.

~~~
octo_t
Those parts are typically automated. Also even if you're a .net shop, you
might have your stuff placed in front a linux/bsd based router which responds
to certain requests (it does happen).

A pen tester can't know what infrastructure changes you've made, and should
work from scratch each time, IMO. It might not have even been your company
which made the change, hoster might have made change to the environment etc
etc.

If it doesn't cost you any more to have them to try and scan /usr,
../../../../../../etc/passwd and so on, why not?

When I did pentests, we encountered a customer who, via a misconfigured puppet
manifest, installed WordPress on a public facing server with a known-
vulnerable plugin, which we found by _always_ scanning /wp-admin/ even though
the customer was a strictly RoR shop.

------
incision
Sadly, relative to what I've seen, the examples here really aren't that bad.
There are plenty of outfits based right here in the US who will happily sell
you the output of a default scan from an outdated version of Nessus.

Also, you've not seen cut-n-paste, search-and-replace garbage until you've had
to sift through the mountain of responses to a public sector RFP.

My best theory is that there are a large number of companies which simply
shotgun shoveled together lowball responses to every posting. The lowball
number virtually guarantees consideration and people who either don't really
understand what they're procuring or don't actually read the responses let the
stuff slip through.

I always made a point of requesting that the companies which submitted crap
like that be banned from future solicitations - it never worked.

------
_delirium
If this is a baseline for "very bad" penetration testing, it makes me think I
could start up an "almost competent" low-end penetration-testing business just
by blindly following a checklist found on the internet, as long as I used a
checklist for the right operating system and proof-read the final report.

~~~
wikwocket
There are many, many niches where you could do well by downloading free
software from the internet, reading free literature/guides/checklists from the
internet, applying a tiny amount of common sense and competency, and then
offering related services in exchange for money.

That so many companies manage to screw this up so badly, and still stay in
business, indicates that all you need to succeed is some business acumen, the
basest level of technical competency, and a little luck! :)

------
slowmotiony
"Their chief pen testing monkey couldn't get into the USA for whatever reason,
so he managed the test from India."

Well, there you go.

------
n1ghtmare_
I'm a developer and we outsource some of our projects (we used to anyway) -
I've got to tell ya - I've seen worse. Oh, the horrors ... the mess. I
remember once when the "consultants" deployed for the first time on our
staging env we somehow ended up with 15 databases some containing credit card
info and all kinds of transactions (I guess from previous clients), the reason
I know is because I had to examine this madness. As far as I could tell their
software was dependent on most of the DBs. By the way the software was a
recruitment web app, nothing to do with payments what so ever.

------
tptacek
Just as a note to startups here considering "penetration tests":

"Penetration test" is a term that means wildly different things depending on
who you talk to.

The kind of test discussed in this post is the most common kind. People in the
field call them "network penetration tests". These are the projects where
someone runs nmap and Nessus and Metasploit against your network, dumps the
Nessus results into a Word document, and calls it a day.

I'm not wild about these kinds of projects, and even less wild about the firms
that specialize in them. They may find things on your network that you need to
know. But they generally involve people just running some tools and
interpreting the results, and then, if they find something blatant, spending
the balance of their time using that finding to pry their way into the rest of
your network.

The latter part of the project --- the part where they get to your database,
dump your hashes, pivot from machine to machine, &c --- is not a great use of
your security dollars. It's generally always going to be the case that if
someone finds a way to run code (or SQL) on one of your servers, you're done
for. The important finding is the flaw that gets attackers into your network.
The findings that come after that look scary, but since there's not a whole
lot you're going to be able to do to reliably lock down your internal servers,
they aren't very useful to you; the next team that finds some other way onto
your servers will embarrass you just as badly even after you "fix" the
internal flaws from the first team.

You can get a license to run Nessus pretty cheaply. You can download nmap and
Metasploit yourself. If you can build a product, you're more than qualified to
run them yourself. If you don't have the bandwidth to do that, don't pay too
much to have someone else do it. Also, demand that the team that does the
netpen breaks out the findings that actually get them into your network,
versus the less valuable findings like "older version of OpenSSL detected that
we don't actually know how to exploit" or "customer records recovered after we
took control of your database", and make sure the team concentrates on finding
new ways into your network, rather than on extending their access into your
network once they do find a way.

You'll need to ride netpen people not to waste time extending access, because
the Fortune 500 companies that are the bread-and-butter clients for network
penetration testers actually do want people to spend time extending access and
finding "shock and awe" internal findings --- they're doing these tests for a
different reason (to justify security budget), not for the reason you're doing
them (to make sure it isn't easy to break into your servers).

~~~
trackerbri
I agree to a large extent. I've been asked for 'a non-intrusive penetration
test'... Uh, what? Then you hammer it out and they're looking for what we call
a 'vulnerability assessment'. The pen-test without the pen- phase.

There's a large number of people running those tools and doing exactly as you
describe. I call them, 'the competition'. They bid low and ship a canned
report with little to no analysis or follow-up. I highly recommend everyone
scan themselves regularly. None of the tools you mentioned are too scary. I do
tend to find the severity ratings to be out of whack with the real world
impacts a lot of times, so if you see a bunch of red in the report don't
panic. Read the text and figure out what it means to your network. The
Metasploit Framework has a bit of a learning curve, but nothing too daunting
and really it's not necessary for a maintenance scan. You can always invest in
a licensed version if the Framework scares you.

Those tools don't provide meaningful coverage of web applications but they
would give you a decent idea of the security posture of your network
perimeter.

Many situations require the disinterested third party to perform the
assessment or audit though. Sometimes customers/partners want to see your last
report if you're dealing B2B and then there are the compliance requirements
(PCI, health or personal information).

The one thing about that report that really bothered me was the remediation
price tags. I know a lot of companies do similar things but we never offer
remediation services since it would put us in a conflict of interest. I turn
down security product installs regularly on our assessment and penetration
testing customers because I don't want us in a position where we're auditing
our own work. It's a point of contention sometimes but its an audit
independence requirement and I won't budge on the issue.

------
madaxe_again
Eh, we get pen-testers regularly telling us that we have to turn off HTTP and
HTTPS (we had one who helpfully suggested serving the site over a VPN) in
order to be secure.

This was on retail eCommerce websites.

------
homakov
All you need is to check references & testimonials of a company you want to
hire. Why pay for some random crap from no-name "craporation"?

------
king_magic
Does anyone have any recommendations for reputable penetration testing
organizations in the US?

~~~
zvarnell
I work for VerSprite. I'm new to the company but like what I have seen so far.
We tailor our services to the needs of each client, verify everything we find,
and the pentesters write their own reports.

------
izzydata
where do you even find people so unreputable? This seems beyond unreasonable
to the point of it being illegal.

~~~
epochwolf
Any number of countries in Africa, Asia, Europe, North America, and South
America. All you need is to operate from a country that doesn't honor the laws
of your target's country.

This kind of fraud happens with outsourced manufacturing in China all the
time. Not limited to china but it's just the first thing that came to my mind.

------
unfamiliar
Is it me or have the articles here gotten significantly worse over the last
month or so?

