
Security Certifications Are Causing More Harm Than Good - ginasilvertree
https://www.tacnetsol.com/blogs/news/security-certifications-are-worthless-and-causing-more-harm-than-good?utm_source=hackernews&utm_medium=gp
======
e79
The thing with infosec is that no matter if you're a consultant pen tester or
an in-house member of a blue team, a high proficiency in technical writing is
required. And few certs demonstrate that the person is a good technical
writer. It's not enough to know the answers to multiple choice questions. It's
not even enough to know how to exploit things. If you don't understand
something well and can discuss it in technical detail to a number of different
audiences, I don't believe you'll get very far in the industry.

There are a couple of exceptions, of course. OSCP is a good certificate to
have. To pass the exam, you are required to not only demonstrate proficiency
in several areas (i.e SQL injection, buffer overflows), but you must also
write and submit a technical report to a review team. The technical report
must address vulnerability overview, impact, risk rating, reproduction steps,
and more. Of course the exam isn't perfect, but it's probably the biggest test
of real technical understanding _and_ ability I've ever seen.

~~~
tptacek
It's unlikely that typical OSCP-holder could write a modern buffer overflow
exploit, or even judge exploitability of a memory corruption flaw given the
source code and a traceback.

Equally importantly: memory corruption exploit development and SQL injection
_are different skills_ , and most people who do SQL injection don't need
proficiency in "buffer overflows". Why is superficial coverage of "buffer
overflows" part of the rubric for that certificate? I don't know, and I don't
know that anyone else does either.

Is there a single coherent security certificate anywhere in the industry? I'm
interested in examples.

~~~
616c
This is why when I was in an infosec bootcamp I begged you to talk to my class
and give a dose of reality.

The leet kids, a minority few and the rest naïve, I would bribe while they
whittled away at online CTFs and MicroCorruption and irritate them with
mediocre questions until they tuned me out. I did not care for tools; approach
and mindset are order of magnitudes harder to explain.

I thought you could be a wakeup call had you told them all the certs pro se is
a waste in a program that pushed that nonsense. I talk trash of my certs and
skills the whole time and they did not get why.

I know you're busy, but I've read your blog and you're preaching to the choir.
Starfighter folded, but I would pay for you to test me as a customer and find
a mentor to answer my stupid questions that I would pay handsomely for the
privilege. I feel I'm not the only one, if you get tired of NCC, that would be
amazing!

~~~
tptacek
I'm not at NCC!

------
orev
There is a huge problem in IT. It's not certifications. It's the totally
illogical bias _against_ certifications. There's no reason someone can't have
both skills and certifications, but everyone treats them as mutually
exclusive. Certs help with administrative things like HR requirements,
contractual obligations, audits, etc... No, those things do not make one
secure, but running a business is not only about being secure. The problem
most IT people have is thinking that certs address technical issues, when in
fact they address business issues.

~~~
pmichaud
> There's no reason someone can't have both skills and certifications

Of course you're right that it's not impossible. But here's why it happens
anyway and why the heuristic of them being roughly mutually exclusive is not
insane:

1\. There's a certification that's nearly meaningless because it's so easy to
obtain without also having the relevant expertise that the certificate is
supposed to represent.

2\. People who are actually good at the thing will notice the certificate
doesn't measure the skill correctly, and will also note that there are people
in the world with this certificate who don't know the skill.

3\. Those experts will not use the certificate when they hire people, and will
not get the certificate since it doesn't work and no one who is an expert is
using it to hire anyway.

4\. Meanwhile, there are basically only two groups that care about the
certification:

> a. People who are clueless: a clueless hiring manager who doesn't understand
> the domain they are hiring for so they are looking for cheap proxies for
> skill and experience. Clueless wannabe professionals who don't understand
> the domain or the industry enough to have been in the expert group above but
> who are still looking for jobs in the field. Clueless clients who are
> impressed by the certification because they don't know any better.

> b. The people who are taking advantage of the clueless clients:
> Professionals and hiring managers who know very well that the certification
> is worthless, but who use it for sales and marketing anyway because it
> mollifies clueless clients.

If that trend holds, then you have a signal (the certification) the repels
experts, while attracting the clueless and those who would exploit them.

 __So when you find someone in the world with that certification, you should
expect on average for them to be clueless or preying on the clueless __. That
's why treating it as mutually exclusive isn't insane heuristically, even
though it's not impossible that someone who is good also has a certification.

~~~
ABCLAW
You've ignored the point of the post you're replying to. You're looking at the
credential as a employee signalling tool, not a tool for other parties to
satisfy a business need.

Your HR department needs avenues to sift through referrals and comparison
points. If an individual has the certificate and compares equally with a non-
certificate candidate, the first individual has signaled, through the
certificate, that he is interested in the field, as well as willing to invest
time and resources into advancing in that field. This is the flipside of the
employee signalling.

If an applicant applies without the certificate to a post which requires it,
he will recognize his cover letter/interview should make a point of
demonstrating competence in the area of the certificate. This is an instance
of employer signalling.

If an applicant fakes having acquired the certificate and there is an easy way
to determine whether or not he is faking, then HR now has an easy sieve for
removing employees willing to misrepresent their skills to obtain the job.
This is a step in the HR QA process.

Some employers work outside the 'work for hire' states. If they contain a
certificate requirement in a job posting with enumerated skills, they can
point to ineptitude in the certificate skillset as reasons for dismissal
depending on the jurisdiction they're in. This is a tool in legal's toolbox.

Perhaps the certificate system is the result of the collaboration of a number
of employers attempting to pool credential and training requirements for an
industry, then create requirements for credentialing into legislation to
control the industry pipeline of workers. This is a method of signalling to
legislators or restricting competitors by indirect means.

And so on.

Additionally, your fact pattern has a small problem: expert acting as hiring
managers who have knowledge of the problems with the certificate are free to
adjust their screening procedure to obtain more fully vetted candidates. The
only time avoiding the certificate entirely is when the signal it provides is
negative.

This doesn't mean that certs are useful, merely that they might be useful to
stakeholders you aren't considering.

~~~
pmichaud
I did ignore that mostly, you're right. My basic claim is that on average this
is actually true:

> The only time avoiding the certificate entirely is when the signal it
> provides is negative.

And further, I think all your examples are perfectly valid, real-world
examples that I don't dispute exist and that lots of people find important. I
also think perfectly good and reasonable people operate in the reality of
their industries and play ball with these things when necessary. AND I think
all the use cases you mentioned are bad for the system overall. As in, they
are real, and in a practical sense we can't just ignore them, but ideally we
wouldn't have them.

HR using negative signals for filtering is bad. Job requirements tailored to
the actual job are good. Broad and mostly arbitrary requirements from a third
party are bad. Applicants to a well specified job also know to address
weaknesses in their cover letter. Specific and well-specified requirements are
also useful in legal disputes. Using arbitrary requirements to provide legal
for firing people is bad. Employers colluding to control the training pipeline
using arbitrary requirements encoded in law is bad.

I agree with you that they useful in the real world, but I'm arguing that that
usefulness is evidence that the system is worse than it could be.

------
leoscuro
Articles like this one frustrate me.

I'm 30, and am essentially starting life over after finishing my military
enlistment a couple years ago. all the experience of setting up shops and
drafting reports meant nothing with out a degree. So I start working on my
degree, and I am absolutely miserable. My love of learning was sucked out of
me because I wasn't learning: I was working towards an extra line on my
resume.

Right now, I am in a jr. sysadmin position making minimum wage, but I was
selected for a SANS scholarship where they pay for your GSEC, GCIH, and one
elective cert. My friend bought me a decent laptop, so I could experiment on
virtual machines. Another registered me for the NCL so I could access their
gyms to spread my legs a bit with more powerful tools. I READ SECURITY
WHITEPAPERS FOR FUN NOW. I love trying to figure out how to best balance
company workflow and security best practices.

I know at the end, that the three certs are not going to make me a SME, but at
the very least I hope that this particular extra line on my resume can help
get my foot in the door somewhere I can be mentored and develop my base. A
salary that can actually pay my bills would be nice too.

Then I read articles like this, and wonder if I'm going to be sidelined again.
I feel like at that point, my life is worthless.

~~~
unhinged_wanker
This guy is flat-out wrong. He bags on the CISSP - thats a friggin management
cert, not a technical cert. Like bitching the CEH doesn't go hard into
Opex/Capex.

Meanwhile on planet earth "draw up an inter-agency security agreement
compliant with all local jurisdictional laws and industry regs" is also
infosec and command line kung-fu will do fuck all to help you get it done.

This guy just drinks "unicorn" piss - he didn't get "trained", he's just so
darn smart and hardworking and special. I bet his business card says "lead
ninja" or some other IT fuckboy bullshit.

~~~
coverband
Exactly... CISSP shows that you have an understanding of risk, numerous
compliance requirements, and how much basic housekeeping activities like asset
inventory management or having proper data classification/access controls help
in maintaining security. The title of "Information Systems Security
Professional" suggests that you're knowledgeable enough to speak intelligently
in all of the ten domains, but your everyday job might be in a single
relatively non-technical domain, like "Business Continuity and Disaster
Recovery Planning".

I wouldn't expect anyone with a CISSP to be an expert in "tech ninja" stuff,
but he should be able to assess whether overall security is better served by
investing in the "ninja work" or, for example, additional phishing training
for employees, at a given point in time. This is certainly not a deficiency in
CISSP, and I don't think anyone with enough experience in the infosec industry
would have such an expectation.

~~~
tptacek
I've been in the industry since 1995. I've worked for Fortune 500 companies.
What's the experience I'm missing to appreciate the CISSP? Because from where
I stand, it seems mostly like a scam to me.

~~~
coverband
Then by definition you don't have any expectations for "a CISSP to be an
expert in 'tech ninja' stuff", as I was saying... ;-) I'll agree with you
that, to an extent, all certifications are a scam, especially those with
artificially high sit-down fees. My point is that, CISSP does not claim to be
a gauge for whether you are a crypto expert, just that you should know the
difference between basic types of encryption and when it makes sense to
encrypt your company's data, so that an accountant in one of those Fortune 500
companies you mentioned doesn't make a costly mistake. In short, it's not
about "how to trigger an RCE", but, if you're in an Ops role, about "how can I
ensure my users are patched without delay, so that I can minimize the impact
of an RCE". Does that make sense?

~~~
tptacek
Roles I've held:

* ISP network security engineering

* Network penetration tester

* Software developer for network security products

* Application security assessor

* (Most recently) Security team lead

I've had these roles for small companies and for very large ones.

What experience am I missing that would lead me to change my mind about the
CISSP? I don't think attempting to pigeonhole me as a "crypto expert" is going
to persuade me, because that's not the span of my professional experience.

~~~
intern4tional
That's an impressive resume of roles, but security is more than just those
areas.

I think the grandparent is trying to say that the CISSP is largely for non-
technical security roles. People that manage large security organizations are
generally believed to be the ones that benefit from the CISSP as they are not
interested in the details and more on a 1000 foot strategic view.

Without knowing more details about the your specific expertise, I would say
you probably haven't been in a role that would benefit from the CISSP by just
looking at your list. If you've been the CISO for a large company with 400+
people reporting to you doing IS work, having a CISSP should at least help you
prioritize the work that needs to be done. Likewise for many companies that
have non-technical management in security organizations, a CISSP helps provide
some background for them.

~~~
spydum
Have you actually looked at the CISSP material recently?

It's a hodge-podge of everything under the sun. The only thing it's able to
prove is that

a) you have endurance and spare time to sit for a 4-6 hour multiple choice
test

b) you can commit to rote memory a bunch of meaningless material which you are
unlikely to encounter in real security/risk management role

It truly is the worst of the bunch, but for reasons yet explained, it's the
defacto "must have" by bigCorps - which is why it gets picked on by so many
folks: everyone knows it's bad, yet most people end up picking it up.

~~~
intern4tional
I haven't looked at it in years, but that hodge-podge of material was more
than enough to provide an executive with the basics that they needed to know
to manage an IS organization which IMO is the goal of the certificate. As
others have mentioned, it is a management cert, not one for normal use.

There are plenty of worse certificates out there - I would argue that the CEH
is probably the worst one at the moment (although they are making some changes
to improve)

------
w8rbt
There's 'compliance security' and then there's 'street-smart security'. They
are very different things.

Most organizations aim for compliance (it's cheap and easy). They base
security on contracts, certs and insurance policies.

Street-smart security practitioners are appalled by this. And, management
doesn't understand why the 'security people' aren't on-board with
'compliance'.

It's a lot like the old west with Cowboys and Indians. Two totally different
world views.

~~~
ianai
Pretty sure you hit it on the head. Over time, security breeches should
alleviate this gap.

~~~
spydum
IMHO, I dont think it will pan out this way. Companies are being breached, and
neither their marketshare nor profits are being significantly affected. Aside
from [https://www.dailydot.com/layer8/code-spaces-
hacked/](https://www.dailydot.com/layer8/code-spaces-hacked/) \-- how many
other companies do you know which really paid a significant price for being
hacked?

Yes, Anthem/BCBS, Target, HD, Sony, etc, etc have all had losses.. but they
really havent been long-term impacted it seems.

I dont know what the answer is, this sucks hard as both a consumer and an
infosec person. I tend to view security as a "hidden performance" factor. As
long as the security flaws don't inconvenience the paying customers too much,
they simply don't care if they exist or not.

------
sathackr
I was involved once in a criminal forensics case. The defense's "expert"
witness was a one man computer shop. He had created his own "certifications"
and listed them on his resumé as indications to the court of his suitability
as a witness.

It was literally "person's-company-name Certified Forensic Examiner".

He had created about 6 certifications, all of which he held.

It's kinda funny, but also kinda scary that the court accepted this as proof
of his qualifications. The prosecution never raised an objection to it either.

~~~
logfromblammo
So he got the court to accept a self-signed cert as a trusted root.

I don't see how that's much different than asking someone to solemnly swear
they are telling the truth, when most humans are as capable as lying about
whether or not they are truthful as they are of lying about anything else.

If the court has no one capable of gauging the expertise of a witness, it has
to trust in _someone_ to do that for them, and if neither party in the case
objects to the witness certifying himself as an expert, it has no reason to
gainsay that assertion. It's really the prosecutor's failure alone, for
letting that detail slip past.

~~~
cortesoft
There is something very different about telling the truth and having
knowledge.

Everyone can tell the truth, but not everyone knows technical details about
something. He is using his certifications to show he knows technical details;
he might even believe he knows those things, but he very well could be wrong.

~~~
cestith
Hell, he may even be right. He might be a real expert. He doesn't make his
claim to know things any more credible by way of vouching for himself even if
he's an actual expert.

Certifications sometimes set a terrible baseline, but at least it's an
independent baseline.

------
raw23
I think soon that this sentiment will start to apply to Universities. It seems
inevitable at some point in the near future there will be an online
'university' (for lack of a better word) who's graduates will be considered
equal or even better than a standard university education, particularly for
tech related degrees.

Universities have been a centralized source of accreditation for a long time.
All it takes is for someone to figure out how to restrict graduation and
filter good candidates using testing or some other means to gain accreditation
and acceptance of its graduates by industry.

~~~
Maven911
That's exactly why a lot of bootcamps have come into existence, along with a
guaranteed job in the industry at the end of it. Though many employers hire
university grads as a sort of "signal" for people who can work hard, think
critically, finish what they started etc. And I'm not saying one is better
then the other, just noticing this trend of bootcamps popping up everywhere to
replace university CS/CE education.

~~~
jtmcmc
anecdotally most people I know who have gone through bootcamp have had major
issues getting hired and the ones that did were hired into support/saleseng
rather than software engineering.

~~~
ganoushoreilly
I've seen the same as well, more so that most of them were taught one
structured way to look at problems and only how to use specific tools, rather
than why. There's definitely tradecraft learning necessary beyond a bootcamp.

------
VLM
Certs show the candidate is in the upper 90% of the group. Its no different
than fizzbuzz. Its very much like stack ranking and tossing out the bottom
10%, those being the cert-less.

My day job maybe 15, 20 years ago was basically Cisco CCNP Routing test. It
was kinda useful to study and pass Cisco Switching test because switching is a
different world of networking. Probably I was in the top 10% of router ops,
but I was only in the top 90% of switch ops. For many jobs thats perfectly OK.

Something very few people like to talk about is self inflation of company
requirements. Top 90th percentile is frankly more than good enough for most
companies. Yes lots of self important strutting about rockstars and ninjas but
all they really need, often all they can get, is top 90th percentile, and it
works out fine.

A cert is not a Nobel prize or Congressional Medal of Honor. Its not even a
PHD. Its kinda like graduating middle school, or having a clean-ish criminal
record. Maybe the best example is its like passing a drug test for a job,
having the self control to not get high for a whopping two or three days
before the test is kind of a minimum display of self discipline to get a job.

------
strictnein
> "Recruiter Thomas Ptacek, whose Chicago-based agency Starfighter specializes
> in recruiting security folk"
    
    
       NET::ERR_CERT_DATE_INVALID
       Subject: www.starfighters.io
       Issuer: Go Daddy Secure Certificate Authority - G2
       Expires on: Nov 13, 2016
       Current date: Apr 12, 2017
    

Is there some infosec version of Muphry's law?

~~~
phonon
Starfighter is defunct.
[https://twitter.com/tqbf/status/771533037666390017](https://twitter.com/tqbf/status/771533037666390017)

~~~
harperlee
Was there ever a post detailing why? I searched for it several times but found
nothing...

~~~
phonon
I think this will probably be it.

[http://www.kalzumeus.com/2016/12/30/kalzumeus-software-
year-...](http://www.kalzumeus.com/2016/12/30/kalzumeus-software-year-in-
review-2016/)

------
ryan-c
My pet peeve with CISSP (and all other ISC^2 certifications) is this:
[https://www.isc2.org/candidate-
background.aspx](https://www.isc2.org/candidate-background.aspx)

It appears to say that if you ever hung out on IRC and tried to keep your
handle private, you're ineligible.

~~~
spydum
That does sound pretty unrealistic, but doesn't this consider that scenario:

"Omit user identities or screen names with which you were publicly
identified."

------
mi100hael
From most people I talk to, the exception is the OSCP since it requires you to
actually pop real, live boxes. Anyone holding that cert has actually exploited
a buffer overflow, escalated privileges, etc. CEH, CISSP, etc are just too
theoretical with no hands-on requirements.

~~~
lawnchair_larry
It's a joke as well, and it just means the holder could copy and paste an XP-
era exploit, which has roughly no relevance today.

~~~
amckenna
Don't know if you have taken it in the last year or so since they updated it,
but it's pretty tough. You may be able to use a public exploit to elevate your
shell once on a box, but getting code execution was the difficult part. One of
the challenges involved fuzzing, writing custom buffer overflow exploits, and
dealing with weird stack pivots. That only got me about 20% of the way to
passing the test. All in 24hrs. My girlfriend was taking the GPEN at the same
time. While I was banging my head against a debugger she was making flash
cards. I think that highlighted the difference between the certs.

~~~
tptacek
Describe the overflow exploit you wrote. What was the vulnerability, and what
did the exploit look like?

~~~
amckenna
Unfortunately I can't get into too much detail because I had to sign an NDA
(to prevent cheating). But the process was similar to when I have found them
in the wild: identify the app, install it locally, fuzz various parameters (it
was a real application, albeit an old one), find the crash, figure out stack
space, figure out bad characters, find the right JMP ESP or equivalent
instructions in a loaded library, write shell-code, encode shell-code, slap it
all together, hope your hex math doesn't suck, run the exploit. No DEP, ASLR
bypass, SEH manipulation, use after free, or heap related work - I learned
that on my own.

Their web app challenges were fun too. LFI to code execution, SQL injection,
things like that. They have a bunch of network related recon, standard red-
teaming stuff.

The OSCE involves ASLR bypass, AV bypass, and using egg hunters.

The big thing about the OSCP, OSCE, OSEE certs is that you actually have to
_do_ all of the stuff they teach you. Not a multiple choice or written
question in sight. For the test they drop you in a network with vulnerable
machines and you have 24, 48, and 72 hours (depending on the cert) to get code
execution on each through various techniques. It was challenging, interesting,
and satisfying.

Edit - it's worth mentioning that I still find vanilla buffer overflows on
projects. These days most thick-client applications that I see are old as hell
and are still vulnerable to exploitation techniques from decades ago. So while
the skills that the cert makes you prove are cursory and introductory, they
are still useful. In any case it's a good starting place for those that want
to learn stuff on their own but do better when they are given the push to
prove it.

~~~
tptacek
In what way did the exploit they had you write differ from the kind we wrote
in 1997?

~~~
alltakendamned
It doesn't. The OSCE targets are Vista and 2K3 Server.

That being said, the nice thing about OSCP imo is that it gives you some
structure and a well set up environment to play in. I think OSCP is a great
entry-level certificate and serves as a good filter to interview junior
candidates.

Does this help at a more elite level, nope, but that's also not the purpose of
it.

------
HenryBemis
Having a 1-2 certifications on a specific domain means that we speak the "same
language" regarding our work.

Red flag: someone that has an email signature with 50 letters next to his/her
name,there is NO WAY someone is spent enough time on each: coding, security,
audit, accountancy, at the age of 30 AND be proficient in all these domains.

~~~
neshibble
I run into this issue on my resume. Do i throw random skills i spent 4 months
learning for some project and never used again? I feel like overloading these
things devalues the skills i actually AM exceptionally competent at, as
opposed to just capable.

~~~
ergothus
My advice? Remember that your resume goes through at least 2 filters, HR and
IT.

What happened was:

IT boss: "we need to hire a coder"

HR: "What skills?"

IT Boss: "Oh, Foo language. But if they're a good coder they can pick it up,
so just a good coder"

HR: "...you're kidding, right? There are millions of resumes out there, most
from people with no skills just trying to land a great job. Give me enough to
filter"

IT Boss: (provides list of three things)

HR: "This still isn't enough. Practically EVERYONE will have these. Give me
years of experience, skillsets, processes, etc.

IT Boss: "Fine, here" (gives long list of things that MAY be useful)

HR: (starts filtering resumes based on these words, removing lots of good
people and including lots of bad people)

IT Boss: (looks at resumes) "These people are clearly lying and all over the
place, I'm going to focus on one or two things to decide who to interview"

So in writing your resume, you want to make sure you have the buzzwords for
the job to get past HR. These buzzwords are pretty much guaranteed to be on
the job listing, even if they end up not being very essential to the job. Did
they mention Scrum? Better have it on your resume, because you may be filtered
out if it isn't, even if it's something you'd not consider worth listing.
Also, use the same words. I once was asked if I had "shell experience", even
though BASH was on my resume. I assume "Agile" and the various implementations
are the same. If they mention XP, you better mention XP.

BUT when your resume then makes it to IT, who (1) know what these words mean
and (2) aren't looking for the same things at all, you need to have what they
want. I tend to use a sidebar on my resume to capture the HR buzzwords, and
emphasize my work experience in the main body, so an IT person skimming it
will see what they want to see.

One technique I've taken to handle HR buzzwords on things that I don't think
are actually a big deal: If the job listing says "Must know React, Angular,
Backbone, or other JS frameworks" and I wasn't really strong in any of them,
I'd do enough research and testing coding to do a Hello World in them, then
add "Exposure to Foo, Bar" on my resume. It tends to get through HR (word is
present!), and I'm not lying to the IT people - they understand that I'm not
claiming expertise, but I'm also saying I'm willing to give it a go.

As a corollary to all of this, you need to tweak your resume for every job
posting, to match their buzzwords and remove ones they didn't list that aren't
really core to your skills.

------
unethical_ban
I find value in the SANS courses I've taken. Not hat the certs mean too much,
since the tests are pretty easy... but the concepts, common jargon and best-
practices discussions that take place can be useful.

It's kind of like college. It's about what you get out of it. From a hiring
perspective, though - if a course provides utility, perhaps if one of the
interviewers also has taken the cert course, they can probe the candidate more
thoroughly on their knowledge from the course.

~~~
spydum
Nobody is knocking security training -- just the value of the certificate
process.

We all know far too many people who have mastered the test, but not the
material. This makes the signal from the certification unreliable.

------
infosecdude64
Being a manager of an InfoSec team I agree with this, especially the CISSP and
CEH.

I've seen a few folks get a CEH and then they're off to App testing land, but
the funning thing is, none of them has ever written an app, some not even a
script, and they are now doing security testing on mobile apps. Basically they
just push a button on an app scanner and pull a report, it's sad.

The folks that do succeed in security are the ones with curiosity, experience
and drive to learn.

~~~
ZeroManArmy
As some one who is looking to get into security, I have that drive of why. At
work I hate when the senior engineers close something without explaining it.

I like to know how something broke and why it broke. I understand programming
and can read about any normal language to a basic degree and lightly
troubleshoot.

Your absolutely right, about those kinds of people too. Some get the
certification and stop there. Others get it and use it as a foundation and
build on it.

------
abraves10001
It doesn't seem like the certs themselves are causing more harm than good, it
seems like the lack of evaluation employers do for potential employees is the
real issue. Giving too much credence to essentially any degree is problematic,
I don't think security certs are exceptional, in this case.

------
peterwwillis
I joined the workforce at the same time that people were realizing that certs
were a joke. That both hurt and helped me, as people hired me based on talent,
and not a piece of paper.

But I wasn't in management, and I've since learned how very little technical
skill you actually need to be an effective manager, and now I realize that
certs are a great way for a manager to understand a complete baseline of the
concepts needed for a particular field.

A manager does not need to be a hacker, but they need to understand a baseline
of security concepts. That's what certs are really useful for.

------
amckenna
I think the point is more that security certifications CAN be worthless and
you don't NEED them, but that doesn't make them inherently bad/worthless. I
think the author's argument should be that the industry has begun to rely on
them too heavily for vetting. That makes sense though because it can be very
difficult to vet the skills of a client. The hiring process is very time
consuming so if you see two candidates and one has "proven" they at least have
some baseline skill in an area then they will lean on that for decision making
in the same way they look at education or self reported experience.

Experience on a resume is self reported so that is an even worse indicator of
skill than a cert. At least one of those two involved external validation by a
3rd party.

I think there are a few good ones out there and getting them ensure the person
has at least a baseline knowledge of some subject. I have worked in the
industry for years as a pentester, but I still went and got my OSCP and OSCE
for fun. A lot of it was review, but it was nice to fill in some gaps and
practice things I hadn't had as much experience with.

Certs are like college degrees, you can get by without them, but it can be
easier if you have them. You will probably learn some things along the way and
the provide a foundation for later studying or pursuit. You don't NEED them,
but you don't need a lot of things in life, that doesn't make them worthless.

------
Maven911
So the article alluded to reading books and hacking on your own. But for those
who need some sort of curriculum, progress bar, or structure, what would HN
recommend to get to some sort of level of competency in the infosec field
(like intermediate level/beginner-advanced).

~~~
strictnein
Others will likely have more informed opinions, but here's some stuff:

Book: Web Application Hacker Handbook
[http://www.wiley.com/WileyCDA/WileyTitle/productCd-111802647...](http://www.wiley.com/WileyCDA/WileyTitle/productCd-1118026470.html)

I've seen it highly recommended and if you're not familiar with the field it's
a good overview of exploit types for web apps.

Online training for free or cheap: Cybrary - mostly okay, but free.

PluralSight - [https://www.pluralsight.com/browse/it-
ops/security](https://www.pluralsight.com/browse/it-ops/security)

Coursera has a Cybersecurity Fundamentals specializationd that's pretty good -
[https://www.coursera.org/specializations/cyber-
security](https://www.coursera.org/specializations/cyber-security)

Other books, if you wanted to go down the reverse engineering route:

Assembly Language Step-by-Step: Programming with Linux

The IDA Pro Book (for the strangely hard to buy IDA Pro, but the free version
is pretty good)

Practical Malware Analysis

~~~
Spearchucker
Bear in mind that IT security goes far beyond something with a processor in
it.

There are physical access controls, personnel assessments, probability and
impact assessments, budgeting, people-monitoring, process analysis and
modelling...

Computers are a tiny part of it. This being HN I have understanding for the
bias though.

------
phaus
Many certifications are worthless, but I would argue that not all of them are.

I learned more useful, practical concepts and skills by taking a couple of
SANS courses than I did in four years of a CIS program at a University. Both
my university classes and the SANS courses consisted of books, presentations,
lectures, and individual or group assignments/labs. If they are taught the
same as a university course, why is it automatically considered inferior? SANS
teaches a lot of tools, but they also teach the underlying concepts to prevent
people from becoming dependent on tools. In some subjects, it would be insane
to not teach tools. For example, I took the GREM (Malware Analysis) course.
Its an very basic course, but it would be foolish for anyone to teach a course
about reverse engineering or malware analysis without using IDA or OllyDBG.

While the class won't (and doesn't claim to) turn someone into a professional-
level reverse engineer, this course helped me understand a few things about
assembly that I just wasn't comprehending when I used other sources.

Would I attempt to use my GREM as justification for applying to a malware
analysis job? Of course not, but the course has helped put me on the right
path. Its possible many people learned through another, far less expensive
method, but that doesn't mean the training was worthless.

If I were hiring someone, I wouldn't use certifications as a sign that they
are qualified. However, I would use the certifications listed on their
resumes, combined with their work experience, to figure out what kinds of
questions I should ask them.

I also wouldn't use certifications or a lack of certifications to disqualify a
person. Using your anecdotal evidence of bad experiences with certified people
to label all of them as incompetent is ignorant.

------
rbc
I don't really go along with this post. I think the title is a good eye-
catcher, but inaccurate. Terry seems biased toward working on the cutting edge
of embedded firmware and I agree that is really important work. I also agree
that a certification like the CISSP (I have one of those) won't prepare you
for that sort of work.

I'll wildly speculate that some potential Tactical Network Solutions customers
are asking about DoD 8570.1 security certifications. That may be a mistake on
their part for something as far down in the weeds as embedded firmware
solutions or the Centrifuge IoT Security Platform.

On the other hand, some of Terry's customers likely hold those 8570.1
certifications, so he might want to be careful about rubbing in how they
wasted their time in acquiring them.

If he takes a look at SI-7 in the System and Information Integrity control
family (found in NIST Special Publication 800-53) he might find some selling
points for his products. A certified security engineer who had done a RMF
audit would know that ;)

------
sxldier
@tptacek and everyone else,

What are your opinions on colleges/Universities with degrees focusing on
Cybersecurity?

Such as Utica and there Bachelor/Master degree in Cybersecurity. [1][2]

[1] Program info: [http://programs.online.utica.edu/programs/online-cyber-
secur...](http://programs.online.utica.edu/programs/online-cyber-security-
degree)

Curriculum: [http://programs.online.utica.edu/programs/bachelor-cyber-
sec...](http://programs.online.utica.edu/programs/bachelor-cyber-security-
curriculum)

[2] Program info: [http://programs.online.utica.edu/programs/masters-
cybersecur...](http://programs.online.utica.edu/programs/masters-
cybersecurity)

------
cmdrfred
I feel this way about certifications in general. I work with tons of Microsoft
and Cisco certified people and the basic computer science errors they make has
be doubting the value of those certifications.

~~~
lawnchair_larry
Why would cisco people need to know CS? Different fields.

~~~
cmdrfred
I mean simple mistakes, like not understanding that information on an air
gapped machine is unknowable to another machine on the network.

"If you took the tax id and social from 'airGappedMachine'"

"It has no connection to any other machine"

"Just query the database"

"Store it on a thumbdrive and walk it over? It changes quite often I don't
think that's a good workflow"

"No, just query it"

"How?"

"SQL!"

Or the classic:

"That password only has N characters, I can crack that in Y seconds"

"You are the Administrator of that box, you can simply reset the password and
do whatever you like."

"Yeah but the hackers.... Y seconds"

"It's not Y seconds for them, if they already are an Administrator or have
read database access it's game over anyway they have to use the API and that
locks you out after 3 attempts. Also your are assuming a much easier hashing
method than is actually in use."

"Yeah but my calculator says Y seconds so it's Y seconds"

~~~
emmelaich
Classic. This could be the script for one of those animated videos that
[https://www.youtube.com/user/gar1t](https://www.youtube.com/user/gar1t) did.

Like the "Mongodb is web scale" one.

------
technion
I had this comment from a recruiter on LinkedIn:

    
    
        The CCNA Security only costs $180. It's a very small investment and I don't see why anyone serious about security wouldn't spend the money to get certified
    

It's like they viewed it as being completely outside of any actual training
and development time - even the person hiring saw it as a sticker you were
meant to buy.

Edit: Interestingly, this vendor specific cert is far more valuable in my
country than anything like CEH if recruiters are anything to go by

------
gwbas1c
“Would you feel comfortable letting a doctor be your primary care physician if
all it took was to pass a written multiple choice exam?”

Doctors have to go through extensive certification in order to be hired, and
constantly have to re-certify. The difference, though, is that a doctor's
certification is very rigorous and well-designed.

The difference is that a medical certification proves competency.
Certifications in our field do not.

~~~
FLUX-YOU
Why is the medical industry able to make decent tests of competency but
devs/security groups cannot?

------
unstatusthequo
Absence of them when you're a consultant is the issue. Its not that it wins
you clients by having them, but not having them might lose you opportunities.
Also, not obtaining them (especially if you know what you're doing) shows
either potential laziness or "better than everyone" attitude that also is
negative. The thrust of that article was exceptionally tilted to that
attitude, and I would think twice about hiring someone with such a huge head.
I'm sure those guys are good at what they do. If so, take the minimal effort
to credential yourself so when people see your name on paper, they have some
reference that you know what you're doing. Without it, is the readers
imagination.

I have CEH, EnCE, and EnCEP. Doing CISSP this year. Why? Because it makes me
stand out regardless. And I've landed clients who were amused by the "ethical
hacker" destination. So don't undervalue cert just because of some cocky
nerds.

~~~
dguido
It depends on the market you're after. Sophisticated buyers won't ask for
certifications. They can look at your work and understand your value. Those
are the clients my firm is after. Leads that ask for certifications drop out
of our sales process, and I refer them elsewhere.

Part of the problem with certifications is that lots of students look at them
as a means to an end. This is wrong and counterproductive. Learning to pass a
certification is the laziness, most counterproductive exercise you can do to
learn security. Yet this is common. Learn by doing. Then get a cert if someone
demands it or offers you more money for one.

If more people approached certification that way, there would be less
industry-wide pushback about it.

------
secretsinger
There is a similar issue with technology certifications (e.g. FIPS 140-2).

A lot of companies treat these as some kind of mystical incantations that will
protect them if sufficiently invoked. Case in point: being mandated to switch
from one OTP generator app to another because the latter is "FIPS-Compliant"
\- regardless of the fact that both generate the exact same set of OTPs.

This cargo-culting is not inherently harmful, but it leads to magical thinking
and a false senses of security, as well as diverting time and energy away from
more productive avenues.

I suspect that the CISSP-genre of certifications suffers from a similar
pathology: intrinsically they do function as at least a partial indicator of
some type of competence. The problem is when actors with a financial incentive
to game the system meet up with bureaucracies: the less defined but more
accurate metrics are thrown under the bus in favour of something that is easy
to quantify and sell.

------
gumby
Every comment seems to be about whether the certs demonstrate anything but
this _article_ says, _" a job description requiring a CISSP was a warning flag
to industry elite not to apply."_

In other words: if the company _asks_ for certs it's the equivalent of wanting
"6 years of react.js experience". I completely agree.

------
fosco
I am familiar with people who have purchased undergrad and grad level degrees
in various fields as well.

The reality is recognizing the importance of a foundation of education is
critical. there will always be shortcuts that people take in every imaginable
part of life. With that said, people who have a firm education or knowledge no
matter where it is from (institution and/or self-taught) will be able to point
out people who took short ccuts fairly quicky. The challenge is knowing the
right course of action to take, firing or other knee-jerk reactions can result
in more harm than good in some situations.

------
lazyant
I had a CISSP certification, I let it expired, I couldn't afford traveling and
going to conferences to get the Continuous Education points.

If you do the math, attending webminars, reading books and writing reviews
don't get you all the points you need every year (I can't imagine the fraud
that must be going on for people trying to get those points). So I said "fuck
it".

Funny thing is, I'm way more experience in security now (with CISSP expired
due to stupid points) than when I got it and was certified. Joke and money-
grabbing scheme no doubt.

------
strictnein
It's the oldest story in tech, certs are "worthless" but look at almost any
infosec job posting and you'll see:

    
    
       Ideal candidate will have CISSP, OSCP, CEH, SSCP, WTFBBQ, etc etc

~~~
_pdp_
That is because most of these job postings are written by HR who have no other
tool to filter successful candidates.

~~~
strictnein
Oh, I know, it's a sick loop. I guess the only way to play is to have two
versions of your resume, one with the certs and one without.

------
homakov
Btw, I conducted pentests for over a hundred of companies (sakurity.com), and
none of them ever mentioned any certifications. Maybe 1% of deals failed
because of government-specific clearance.

------
Zork212
I heard this back in the day when the CNE was "money printing machine, " and
the MCSE just kicked off. So nothing has changed from the complainer side of
things.

Certs have value depending on the person's skills. Never hire just because
they have a CISSP or XYZ.

Sometimes it's just the key to the door to the interview. From there it's up
to Employer to properly vet the candidate. If they hire an idiot with a paper
cert, then it's their fault... and then they write an article. LOL

------
godzilla82
My experience with all the certified security consultants is that they refuse
to put any effort to understand the end risk of a security vulnerability on
the product that is being audited. For e.g. for many sites, it doesnt make
sense to implement security features that are required by online banks.

As an aside question for web developers: How many web developers
encrypt/checksum all fields on the client-side?

------
throwaway2016a
I wanted to get my consulting company into PCI auditing and you need
certifications to do that. One problem with the certifications is they aren't
actually skill based.

I wouldn't be able to get one despite having experience because:

For several of them it requires years of work experience with a specific job
titles (your job needs to be security, it can't just be part of your job) and
the continuing education credits are expensive and largely not helpful.

I would love if there was a recognized certification that was actually
interested in proving your skills not for making money.

I know other disciplines are the same way.

They don't really serve to keep unskilled people out, they serve as kind of an
elite paywall where you need time and money to break through. And breaking
through is largely an exercise in brute force not in skill.

~~~
rsync
"I wanted to get my consulting company into PCI auditing and you need
certifications to do that. One problem with the certifications is they aren't
actually skill based."

[http://www.rsync.net/resources/regulatory/pci.html](http://www.rsync.net/resources/regulatory/pci.html)

------
austincheney
I am wondering how many of these wise commenters have actually taken the CISSP
exam, or even know what it is? It costs $600 and only has a 60% pass rate even
after it requires certain validations in order to achieve a test appointment.

In discussions like these I really wish people would post up front if they had
taken the test (regardless of pass or fail). This lets me know which comments
I can ignore as completely ignorant.

------
abarringer
I once tried to hire for a MS DBA position. The certified MCDBA's could not
tell me what an index was or how it would be used. I was shocked so I went and
took the tests myself. There were a few questions on indexes but I could see
how someone could answer those questions and forget the exam cram by the next
week. We ended up hiring someone with no certs and no degree and he was
fantastic at the job.

~~~
devnullmonkey
Ah, yes... the "paper tiger". While I do have certs myself, it's only because
my employer has required them. Left to my own devices, I would never bother.
I've been in IT for almost 20 years and I agree that the "no cert/no degree"
guys usually work out.

The team lead whose feet I studied at in my first few years in IT had a high
school diploma and he could run rings around the guys from Carnegie Mellon and
RIT that worked with us. He was a deep diver mentally. I watched this guy drop
awk, sed, bash strings a mile long, write Perl scripts without consulting a
single web page. He knew iOS (Cisco), Perl, TCL/Tk, Sun, BSD, and about
everything else. He could configure a HA UNIX servers and Oracle DB backends
without consulting a manual. It was most impressive. He was let go because he
was a team lead (middle manager). The people that replaced him--yes, people--
knew nothing in comparison.

------
sathackr
I'd venture to say most certifications in general are junk, CYA processes. If
you are basing your hiring decision on the letters someone puts after their
name, you're doing it wrong.

The same could probably be said for a degree, particularly with the current
snowflake "everyone must pass" mentality at so many schools.

Certifications and degrees are nice, but neither guarantees a person is
competent in any field.

------
raesene6
I'm going to partially disagree with the article. the problem with the
approach of "just learn to be a good security person" is that it doesn't
scale. Sure back when I, and a lot of other people who are a bit older,
learned security that was the only option, there weren't structured courses
and certifications.

However when we're working at scale, certifications can be useful as providing
a demonstration that the holder has some level of exposure/knowledge of the
arena in question.

what complicatates this quite a bit is that some of the more popular
certifications are, rightly, not considered that good as the process to get
them lends itself to rote learning. So things like the CEH and CISSP where the
exam is multi-choice, not so great.

On the other hand things like the CREST CCT definitely require a decent level
of knowledge to pass and you need to be able to apply that knowledge in a
practical situation and in time limited conditions.

I find the anti-certification bias in IT and security a bit odd really. If you
look at other professions (e.g. law, accountancy, architecture etc etc) it's
recognised that these things are required to get a minimum level in place in
situations where you have a large body of people, I don't really see why IT
Security should be any different.

To me, the problem of "these certifications are bad" should have an answer of
"lets make better certifications" not "lets not use certification"

~~~
zeta0134
The only certifications I've picked up so far along my industry journey (I
have no college degree and don't plan to get one, so these are necessary) are
the Redhat RCSA and RHCE. Both of these certs can't be solved with rote
memorization, and required me to log into a virtual machine and solve problems
in a live environment, running through a plethora of common systems
administration tasks that were then graded by how well I accomplished the
requirements. Often the requirements were vague enough that I had to do some
digging.

There was no real memorization needed, and I had the full man pages of the
operating system at my fingertips, but the time limit ensured that I needed to
have at least a certain degree of proficiency with each tool to finish all of
my tasks before the end of the exam.

I assumed this was the norm with certifications, but the comments here
strongly suggest that it is not. Are Security Certifications really multiple
choice questions without any practical applications? That seems like it could
stand to be improved greatly.

~~~
raesene6
some are bad certs (rote learning, multiple choice), others are not (they have
good practical elements).

One problem is that some of the "bad certs" e.g. CEH, CISSP , are well
established.

My feeling is that the answer isn't "don't have certs" but "have better certs"

------
jstewartmobile
A lot of the bigger certs have all kinds of high-dollar prerequisites and
accounting that only larger firms are going to bankroll. Perhaps the certs
aren't so much an expression of how "good" you are as much as they're a token
of how much faith "the man" has that you will stick around long enough to
recover the investment.

------
foo_bar_baz_quu
Disagree with the sensationalism of the article but for anicdata: I
interviewed 41 security engineers last year. Gave offers to 8, hired 5.

A few, less than 10, had certs listed on their resumes and no one in the
interview loops gave weight to those certs or even talked about them from what
I recall

------
jvehent
True infosec l33t$ know the only real certification is being able to recite
from memory the lyrics of
[https://www.youtube.com/watch?v=FoUWHfh733Y](https://www.youtube.com/watch?v=FoUWHfh733Y)

------
doke01
Certs value, if any, are to show a very basic skill level, or starting point.
That's it. If you are hiring someone just because they have a CISSP where
another individual does not, you don't know what you are doing either.

------
_pdp_
While I agree that many of the certifications are worthless they do help
newcomers to get into the security field.

I was lucky enough to start early when the only thing you had to do was to
show your skills and willingness to learn. Those days are long gone but you
can still prove yourself by delivering awesome security research or commentary
- and people will hire you based on that.

That being said, any IT field which requires some sort "technical"
certification to get hired is probably not the field you would like to get
into. Why? Too much competition and race for the bottom line.

Luckily this is not exactly the case with Information Security - yet. In our
line of work there are people who do a lot of the uplifting where
certifications do matter. I would not say these are very technical jobs and
they are totally dispensable. Other professionals acquire specific skill sets
which are rare or difficult to obtain and as a result they tend to have the
upper hand. There is of course a lot in between.

I do not mean that everyone who has certifications next to their name sucks.
Not at all. But unfortunately, unless you are applying for a commoditized IT
security field or security manager type of role, it will raise some questions.

Keep in mind that HR is also partially to blame. Many people do not know how
internal HR teams typically work which is essential to understand why certain
things happen the way they do when hired or when progressing through the ranks
of a company.

HR are typically not technical and they are not experts in security either so
they do not know what exactly they should be looking for. Of course they are
not completely clueless but a seasoned hacker can smell bullshit a far while a
well informed HR cannot. HR's filter is your CV and the job spec and these two
are simply not enough to evaluate a successful candidate. Some job specs are
written by HR themselves which is crazy because they are not in the position
to formulate the role so they have to stick to what is known - i.e.
certifications.

That being said there is a shortage of IT security professionals. As a result
of that almost anyone can get hired if they show the right attributes. It does
not take a long time.

The only thing that bothers me with the InfoSec industry these days are not
the certs but that companies tend to have really bad hires (maybe due to bad
certifications) who due to lack of deep understandings of the subject work on
things that practically do not matter. As a result of that these companies
have the illusion that they are doing something while the fact is that they do
not in a practical sense. This is why in many places no one knows what the
security department does - I am not kidding.

------
konceptz
"If you must obtain a security certificate for compliance or regulator
reasons, so be it."

It's interesting that his argument is against certs for companies but then he
looks at regulation as unchangeable.

------
Scuds
If I'm a competent enough services and web developer wanting to move into
infosec, what else could I be doing to get my foot in the door besides
collecting certs, as ostensibly shite as they are?

~~~
dsacco
Bug bounties. Andddd that's it, you're done. Find a few in recognizable
companies, and jobs will simply come to you.

I'm not going to engage in the debate about what certifications _should_ be in
the industry, but I'm happy to show which option is most advantageous for your
particular needs right now:

* Certifications mostly do not teach you anything that you, as a competent web developer, cannot learn from the same five textbooks tptacek, others and myself recommend in these threads.

* Certifications cost money.

* Certifications optimize for companies and roles that disproportionately do not pay highly.

* Many certifications require upkeep.

Let's contrast with bug bounties:

* Bug bounties grow your real-world, hands on experience.

* Bug bounties do not cost you anything (in fact, you can get paid!).

* Bug bounties cover a much more diverse and up to date set of security flaws than certifications.

* Bug bounties optimize for companies and roles that will respect you more highly, pay you far better and aggressively try to hire you after you find more than, let's say, two serious vulnerabilities in recognizable companies.

* Bug bounty recognitions do not expire.

~~~
djcapelis
This is the best hot take I've seen on how to deal with certifications in
security so far. This is an excellent suggestion.

------
ef743b3baa573
so, How does one get into security? (to be specific like Incident response and
analyzing intrusions)

------
jcoffland
There is a lot of self promotion and all my friends agree with me type logic
in this article.

------
geekamongus
OSCP for the win.

