
The SOC2 Starting Seven - lvh
https://latacora.singles/2020/03/12/the-soc-starting.html
======
ejcx
Very good post. This ticks all the boxes on the fundamentals when spinning up
a security program.

SOC2 Type2 is really where you want to be, but it takes time. Navigating
compliance for startups is pretty challenging and I see so many not having a
clue how to navigate sales without certs but it's super doable, and getting
these things finished get you pretty far along towards Soc2 type1, and shows a
lot of goodwill to share these practices even _before_ you have any certs

------
antoncohen
I'd like more detail on the advantages of AWS AssumeRole, and what attacks it
is designed to protect against. It must be more than CloudTrail logs, because
normal API usage gets logged to CloudTrail.

One thing I'd like to highlight is that SOC2 is about having controls (of your
choosing), and documenting and proving you are following those controls. I
think it is important to pick controls you actually want to follow, and that
those controls aren't so specific that you get fenced in to a bad process.

I've worked at a few companies that have gone through SOC2, one of them
(Dropbox) did it very well. I think the average engineer didn't need to know
about SOC2, or what all the controls were, because the controls were well
thought out, and automation took care of the proof. At another company every
engineer was constantly saying "you have to do that because of SOC2", because
the controls intruded on the way engineers worked.

"SOC2 requires that two people approve every PR". Hmm, no the accountants that
created SOC2 don't know what PRs are. "Only members of the team called 'QA'
are allowed to transition Jira tickets from state A to state B, SOC2 requires
it". No, I'm pretty user SOC2 doesn't specify issue tracker workflows. "You
need to name your git branched XYZ-abc-foo-bar, because of SOC2". You're
telling me that accountants are specifying source control conventions?

~~~
time0ut
As mentioned, AssumeRole is a critical part of a multi-account strategy, which
is considered an AWS best practice. In addition to that, it is an integral
part of an SSO strategy. Ideally, instead of a pile of IAM users with separate
credentials, user identities should be federated from your identity provider
(like Auth0, Ping, AzureAD, whatever). There are a number of setups, but they
all boil down to your users trading their corporate identity for temporary AWS
credentials via something like an AssumeRoleWithSAML call. This is only
slightly more complicated to set up than a good IAM user/group architecture,
but has the benefit of not needing to deal with IAM users and multiple sets of
identities for your users. You can also map particular roles back to something
like AD groups to control who can assume what.

~~~
antoncohen
Thanks, that was a very clear explanation. I know AssumeRole is needed for
SSO, and I understand the benefits of SSO. The way the article phrased it it
sounded like AssumeRole has some very specific benefit by itself.

And if the answer is "short lived credential", then I'd like to understand how
short lived credentials that require a long lived credentials to get them are
better than long lived credentials, _if_ both can be just as easily revoked.

------
teilo
Honestly, this misses a huge point and does more harm than good. Having some
of these things in place is fine of course, but it don't really help you much,
nor is most of it required.

SOC2 does not require: SSO, MDM, or almost anything specifically (And what's
with the JAMF pro recommendation? MDM is more than iPhones). What it does
require are controls. It does not specify what those controls are. You need to
create them yourself, and they need to be sufficient to meet the requirements
of the Trust Services Criteria. And to have controls, you need to know what
you are controlling and why.

This brings me to my point: To succeed in SOC2 you actually have to have a
plan. You have to have written, reviewed, implemented, and audited policies.
THAT is the single biggest hurdle for most companies.

You also need to know how to do a risk assessment on your organization. This
is not terribly hard, but it is essential to SOC2.

And finally, you need buy-in from the governing bodies of your organization.

None of these things I mentioned are products. You can't buy them. They take
work. A hell of a lot of work. I will guarantee that most companies doing this
for the first time do not have half of the processes in place that SOC2
requires. I doubt that 20% have internal audit procedures, for example, or
sufficient end-user training.

And one more thing that will take nearly anyone who had done security by
surprise if they don't know it's there. The COSO Principles. These were added
to SOC2 in the wake of the Lehman Bros/Enron scandal. They extend SOC2 to the
entire governance structure of your organization, and include controls on
financial reporting, hiring practices, employee review processes, etc. This is
a major PITA, because it lies far outside the bailiwick of most CISOs or
equivalent, and, again, it requires extensive cooperation from the rest of
senior management.

As you can probably tell, I've done this before.

~~~
tptacek
We've watched our clients get SOC2 certified and been directly involved in the
IRL and audit interview processes, and circulated this among peers that have
also been SOC2 certified, and we're pretty comfortable with the advice we're
offering. So:

As we've said repeatedly, SOC2 doesn't "require" much of anything. It
certainly doesn't call SSO or MDM out by name. What it instead involves are
abstract control points that are addressed with evidence. As the top of the
post says, the point is to engage in security engineering practices that
generate evidence useful in SOC2 engagements.

So, you can certainly freestyle your application and infra access controls,
and make up policy responses to the IRL questions on the fly. You can also sit
around and worry about which of the "COSO Principles" you're covered on or not
covered on.

Or, you can do some basic sensible security engineering things that are almost
certain to cover the majority of the evidence requests your auditors generate,
in most cases without even needing to be aware of the AICPA guidance line
items.

You make your own process recommendation here: do a "hell of a lot of work" to
have "written and audited policies" and a "risk assessment" and become aware
of the "COSO Principles". I have no trouble believing that approach works,
because after seeing and being involved in a bunch of SOC2 audits and talking
to people who did SOC2 audits, I believe _everything_ works.

More importantly, we see startups led around by the nose by insane
questionnaires (and a SOC2 IRL is certainly that). We've talked to firms that
have been told by their SOC2 auditors to install IDS systems, WAFs, and Mac AV
software. We are alarmed by the prospect of a startup without in-house
security or compliance expertise reading what's out there about SOC2 ---
_especially_ the AICPA documentation, or example IRLs --- and then just going
and _doing_ that stuff. They'd be wasting a tremendous amount of time without
realizing that smart startups skip a lot of that work and deploy none of the
junk and get SOC2'd just fine.

So, if you're going to do something, do something sensible.

What I think people should be especially careful about is taking advice from
people who have just done SOC2 at a single company or with a single auditor.
_They are all different_. There are processes where you'll end up explaining
what a URL is. There are processes where you'll end up explaining why you have
VPC flow logs disabled. The value we're offering here is that we've seen it
done repeatedly, with multiple auditors.

~~~
teilo
Look, I agree with pretty much all of what you say. I would never suggest any
company, whether a startup or an established enterprise (which is my case) go
through the SOC2 process unguided, attempting to interpret the ACIPA docs on
their own. That is a recipe for disaster. The TSC is almost inscrutable on its
own. The AICPA cannot write in plain english to save their life.

That said, I don't understand this statement: "You can also sit around and
worry about which of the "COSO Principles" you're covered on or not covered
on."

In the end, to pass a SOC2 audit, you have to have evidence for each of the
applicable TSC points, including the COSO Principles, which have little or
nothing to do with information security but are all about general business
governance. It's not a question of "sitting around wondering." It's
understanding what the COSO Principles actually are after, so that you can
assemble the appropriate evidence. And that means working with HR and finance.

You make it sound like the COSO Principles are irrelevant. Believe me, if I
could skip them I would. If I could skip a risk assessment, I would. If I
could skip all the "written and audited policies" I would. But you can't skip
them. And in the case of a risk assessment, you shouldn't skip them. I mean,
that's the whole point in the end: knowing your risks, and having the controls
in place to mitigate them.

So yes, of course you need to be sensible. But you also have to do the work,
and the work is hard. It is by far the last favorite part of my job.

But my point is: It's not about the product category boxes you have ticked.
Security products are important. They make compliance much easer. No question.
But for SOC2 to have real value to an organization (and I think there is real
value that can be gained from it), it is _far_ more about process, and it is
process that is the most difficult thing to implement and religiously maintain
and document.

~~~
tptacek
This has simply not been my experience of how SOC2 processes have actually
worked at real startups. The experience I've seen, repeatedly, in variations
but always on this one theme, is of a collection of mostly boilerplate
policies being exchanged for a turbocharged vendorsec questionnaire, followed
by a series of interviews. The line items on the questionnaire are answered in
any ways that is convenient for the auditee and credible for the auditor.

At no point is anyone on the engineering side made familiar with TSC points or
COSO; the runic accountancy is driven be the auditor. I would, in particular,
_not recommend_ that startups thinking about pursuing SOC2 spend a lot of time
familiarizing themselves with the TSC or (god forbid) the COSO Framework,
because it simply isn't going to matter.

My general belief is that SOC2 is of no intrinsic value to organizations, and
going into it with a mind towards letting your engineering practice be _guided
by_ SOC2 rather than _dictating_ SOC2 is a recipe for heartbreak. Among
security engineers (and managers) at tech startups I've talked to, I don't
seem to be far out of the mainstream. SOC2 is like most university degrees:
it's a demonstration of seriousness, and nothing more.

If you think this blog post is about "buying the right products", you've
entirely missed its point.

------
wglb
>Any time anything security relevant incident happens, you could make a
private Slack channel named for the incident.

This might be a good start, but it makes me a little uncomfortable. In a true
incident, there is a metric ton of stuff that I would not want to be
discoverable. I like to set aside an area with locked-down access to keep the
truly toxic material found during a serious incident.

And for startups or other small companies reading this--the things mentioned
here should not be a big deal or take enormous time to do. The earlier you do
what is recommended in the post, the happier everyone will be.

I'm reminded of my Dad's saying: Tires are like insurance: When you need it,
it is too late.

[edit typo]

~~~
tptacek
A couple people told me the same thing, and I think it's a valid concern _as
security process_. On the other hand: Slack-channel IR has worked well for
clients of ours _as a compliance step_ , and having Slack-channel IR is better
than having no IR from a security posture perspective.

~~~
superq
That's a good point. Even just using Github or Gitlab issues to track
incidents with approvals is better than nothing.

------
abeld
Any suggestions for MDM for Linux laptops? Most device management solutions I
have seen are for either Windows or Mac, but is there one that is accepted by
auditors and is not utter garbage UX wise for the user on Linux, where the
user is most likely going to want and have full admin access to their own
laptop? (I.e. since the user in question is a developer, I feel a strong
aversion to not trusting them to do system administration on their own
laptop.)

~~~
diafygi
We've been looking into Osquery. It's open source and appears to be very
lightweight. Has anyone else used it?

[https://www.osquery.io/](https://www.osquery.io/)

~~~
tptacek
osquery is great and, while it does not tick precisely the same checkboxes as
Jamf or Fleetsmith, it'll almost certainly suffice for evidence generation for
compliance (osquery is in fact much better at producing "evidence" than most
MDM tools are).

~~~
zercurity
Just a shameless plug we've written thousands of queries to automate many
compliance checks against major frameworks such as: NCSC-CE, CIS, SOC and ISO.
[https://www.zercurity.com/product/compliance/](https://www.zercurity.com/product/compliance/)
It'll even collate all the evidence for you and help your employees improve
their own cyber security posture based on your corporate policies.

------
saagarjha
> Another bonus point: standardize everyone on Chrome.

:(

~~~
Fnoord
The next sentence was worse:

> We don’t have to grumble about why; we’ll just observe that this is what
> almost every security team does anyways.

------
superq
I've been through the process before and just went through it again, and the
first thing listed (Single Sign-On) is not required at all. I'm not even sure
it's mentioned directly in the AICPA criteria.

I'm also not sure what SSO really buys you, except for just moving the
criteria target to another SOC-2 report. ;)

Also, MDM is not really critical at all, as long as you have a
solid/documented way of managing accounts on web and mobile apps.

I do like the rest of the article except for the list itself, however. SOC-2
is all about documentation and processes. Justifying your security choices is
less important than carefully and fully documenting them, and the processes
that you have in place.

~~~
tptacek
SOC2 doesn't really "require" much of anything. The question we're answering
is: "which security engineering projects can you undertake that will make a
SOC2 audit go more smoothly". Because SSO directly addresses a lot of access
control control points, and generates a lot of evidence for those control
points, _and_ because it's generally a very good practice to have anyways,
it'd be the first bit of security engineering we'd recommend to ratchet up
your maturity.

What I can tell you is that the IRLs I've seen, both from big 4 auditors and
from SOC2 boutiques, have dozens of questions that are directly answered by a
sane SSO configuration.

We've watched clients clear SOC2 audits without any coherent SSO. It's clearly
doable.

This is all made more tricky by the fact that different SOC2 auditors have
different requirements. They're in one sense or another _all_ negotiable, in
that you can always switch auditors if they're unreasonable, and we've
definitely collected several horror stories about aborted SOC2 processes that
were resumed with a different vendor. And, to layer more complexity on that,
different bigCo customers will have opinions about who did your SOC2. It's all
a mess.

~~~
superq
Right, agreed 100%. In terms of security, the value of centralizing
credentials is debatable.

~~~
tptacek
The other bit of subtext in this post is that you often have multiple ways of
addressing control points auditors bring up, and we'd like to steer startups
towards the _good_ ways of doing that. So if an SSO keeps you from having an
insane password rotation policy, or an MDM (also not "required") keeps you
from installing Mac AV software, it's super worth it to us.

~~~
superq
That makes sense to me -- thanks for clarifying!

------
zeveb
A lot of really good stuff in here. I have a few questions & quibbles though.

> You’re probably all Macbooks.

What if we're not? _Are_ there any good MDM solutions for Linux? I would
prefer not to force macOS on folks who don't want it.

> Another bonus point: standardize everyone on Chrome.

Why not stick with Firefox? It has the advantage of at least _trying_ to keep
user information secure.

There's a lot exciting in here, though. I would love to transition to SSH
certificates issues by some fancy OAuth2 service. I _imagine_ one could use a
named pipe to request them on-demand, so one could just issue 'ssh foo' and
everything would Just Work.

------
tzs
The article and the comments here contain a lot of TLAs and the like that many
will not be familiar with. Anyone want to expand some of these to save people
a bunch of Googling/DDGing/Binging?

    
    
      AD AES AICPA CISO
      COSO CSO DEP FDE
      FIM IDS IR JAMF
      LDAP MDM OIDC SOC2
      SSO S/N WAF
    

I don't recall ever seeing a comment thread with that many such things.

~~~
julianj
Sorry for any mistakes, mobile.

AD active directory

AES advanced encryption standard

AICPA American Institute of Certified Public Accountants

CISO chief information security officer

COSO risk management framework

CSO many things, probably chief security officer in this context

DEP data execution prevention

FDE full disk encryption

FIM file integrity monitoring

IDS intrusion detection system

IR incident response

JAMF apple device management

LDAP lightweight directory access protocol

MDM mobile device management

OIDC open id connect

SOC2 audit standard

SSO single sign on

S/N serial number

WAF web application firewall

~~~
tptacek
Different DEP here. :) We're talking about the Apple Device Enrollment API.

~~~
julianj
Thanks, I hadn't yet made it through the full thread for context.

------
tialaramex
Typo: In the list of "things you don't do" it lists Web Application Firewalls
and I thought there's an amusing asterisk for the claim that they "don't work
at all"†

But sadly there isn't - that asterisk is supposed to be the next bullet point
for Endpoint Protection and AV.

† So many missed opportunities. These things are so awful.

~~~
lvh
Thanks! Just fixed that.

~~~
pvg
Also 'log munging' and scene from _Love Actually_ instead of _Say Anything_

~~~
lvh
Wait, I can't find Love Actually and Say Anything in the post. Are you talking
about a draft?

~~~
pvg
_taped to the boom box playing Peter Gabriel you hold aloft outside their
offices_

That's _Say Anything_. It's a movie from the previous century. The scene from
_Love Actually_ you can actually recognize from gifs and recent SNL parodies.

------
jgalt212
It's amazing how many of these recommendations are cloud based. If you have a
cloud based box with bad security, anyone can get to it.

If you have a box you own on your own premises, you can have crummy security
on that box and still be in good shape.

I like belt and suspenders. 2FA and physical security.

------
csmattryder
> Sign up for or set up a centralized logging service. It doesn’t matter
> which. Pipe all your logs to it.

Is there one recommended by HN for multi-platform use?

I'm in the process of preparing for ISO27001 certification and as the topic
has arisen, I'd love to get a few opinions if possible.

~~~
abarringer
There are a lot of amazing fancy services available... for a fee. We log
billions of lines a year and use GrayLog because the other services are cost
prohibitive.

~~~
tptacek
Which is totally fine. Auditors aren't going to care what your log service is.
They might, on the other hand, ask you to show evidence of, like, a log line
when someone accesses the admin console. You want that to be easy to generate
and, just as importantly, you want _the process_ of obtaining that log line to
be _easy to describe_ : "go to this one place, type in this one query, save
the result". In a perfect world, you want the answers to _lots_ of auditor
questions to be the same, perhaps modulo the "one query" you type in to
generate the result.

This isn't because the auditor is going to care whether you have to log into
15 different hosts to grep for 20 different log lines; they have no idea what
"grep" even is. It's because it's a lot of work to document 15 different
processes coherently.

This idea is the basic lens through which people should be reading our
recommendations here.

------
mushufasa
To be clear, when you talk about using SSO with all your applications, are you
talking about A) all of the applications your team uses e.g. SaaS like GDocs
or AirTable or whatever or B) the application you are creating? e.g. in lieu
of your own database?

Or maybe you mean for both?

~~~
tptacek
Both. As much as possible. You'd like as many of the answers to questions
about "how do you manage access to this application, know who has access to
it, and ensure that people who shouldn't have access to it don't" be
"screenshots and logs from your SSO system".

~~~
abraae
I'd be interested in any opinions as to whether keycloak is an acceptable
solution for SSO wrt SOC2. (Hopefully so as we are well committed to it).

~~~
kasey_junk
SOC2 doesn’t judge which SSO you use (or even if you use SSO).

If you write up your processes with your SSO provider and you follow your
processes any SSO (or none!) works.

------
tialaramex
I guess this is appropriately sceptical about audit at least.

I think people both over and under-estimate audit (in different ways), but
certainly if I have to pick it's the over-estimation of the value of audit
which hurts and so the scepticism in this piece is warranted.

~~~
lvh
Maybe! I get the impression there are definitely plenty of people have bought
SOC2s the way they also buy pentests: "I'm spending $X0,000 today so I can
close $X00,000 in sales tomorrow". Which of course, we as security people
would say is spending a lot of money and getting nothing, and the business
person says is a necessary expense and something to keep in mind when re-
evaluating margins :) (But yes lots of companies also mistake SOC2 for a
security audit of course!)

------
jesseendahl
This is a fantastic article overall. As someone who has now gone through SOC 2
twice (first at Dropbox as an IC, now at Fleetsmith as cofounder/CSO, the only
part I disagree with is the advice re: MDM.

If you want to make SOC 2 easy, you should choose us (fleetsmith.com) over the
competition.

There are some specific things SOC 2 cares about with regard to laptops. The
ones that can take effort are:

\- User to device inventory attribution ("give me a list of all your Macs and
which employees they belong to")

\- FileVault enforcement, with automatic (and secure) escrow of the recovery
keys

\- Enforcement of some kind of AntiVirus (either the native stuff built in to
macOS, or deployment of a third party solution like MalwareBytes)

Each of those 3 items are very time-intensive efforts with competing products.
With Fleetsmith, each of them are quite literally (not marketing hype) a few
clicks.

We're also the leader in terms of security. If you're interested in the
intersection of MDM and security research, check out the following:

"Hacking a Brand New Mac Remotely, Right Out of the Box" — Article summarizing
research presented at Black Hat 2018.

[https://www.wired.com/story/mac-remote-hack-wifi-
enterprise/](https://www.wired.com/story/mac-remote-hack-wifi-enterprise/)

\--

A Deep Dive into macOS MDM (and how it can be compromised) — 61 page
whitepaper related to Black Hat research.

[https://i.blackhat.com/us-18/Thu-August-9/us-18-Endahl-A-
Dee...](https://i.blackhat.com/us-18/Thu-August-9/us-18-Endahl-A-Deep-Dive-
Into-macOS-MDM-And-How-It-Can-Be-Compromised-wp.pdf)

\--

On DEP, MDM, device identity, and authentication — Blog post on the security
of the device enrollment process. Includes Fleetsmith’s secure device
enrollment implementation, in hopes that other vendors in the space will adopt
it.

[https://medium.com/fleetsmith/on-dep-mdm-device-identity-
and...](https://medium.com/fleetsmith/on-dep-mdm-device-identity-and-
authentication-82c9b78cb9c2)

\--

An Enterprise Security Roadmap for macOS — Detailed blog post on improvements
that Apple should make to macOS and MDM, and why. Includes a detailed list of
over 50+ bug reports/feature requests.

[https://blog.fleetsmith.com/macos-enterprise-security-
roadma...](https://blog.fleetsmith.com/macos-enterprise-security-roadmap/)

If anyone is interested in chatting about any of this stuff (the product or
the security side) I'm on Twitter at @jesseendahl and on the MacAdmins Slack
(macadmins.org) at @jesse.

~~~
lvh
I have no problem with Fleetsmith and sure Jamf ain't perfect but... just to
restate your point briefly: you're claiming "show inventory, turn on FDE,
don't fuck up XProtect" are "very time-intensive efforts" with Jamf Pro?

~~~
jesseendahl
Showing a list of devices is very different than accurate user <-> device
attribution. Doing that in competing solutions to Fleetsmith requires at
least: (1) having an LDAP server (2) configuring server details, including TLS
etc. (3) LDAP attribute mapping in the MDM solution, and then (4) manually
assigning each device to a user one a time -- a process that usually involves
repeating the following procedure for each device:

1\. Copy S/N to clipboard

2\. Tab over to inventory spreadsheet

3\. Command-F and pray your IT helpdesk recorded which employee the device
belongs to

4\. Tab back over to the MDM solution, select the employee from the dropdown
menu.

This becomes extremely time intensive if you have hundreds or thousands of
devices.

As for FDE--enforcing it is easy. Doing that while also _reliability_ and
_securely_ escrowing its recovery key is a much different story.

On the reliability side--not every solution out there is equally reliable when
it comes to ensuring that the recovery keys actually make it into the MDM.

On the security side, I believe we're the only vendor that does CA pinning
across the entire management lifecycle. Those stages are covered in great
detail in the previously linked BlackHat whitepaper.

On the server side, I believe we're the only vendor to automatically encrypt
sensitive fields (such as FileVault keys) before the hit the database, with
per-customer AES keys (key generation and key management handled by Hashicorp
Vault's transit backend).

Last but not least, there's the matter of generating evidence for auditors to
prove you're doing the things you claim to do. We automatically surface the
status of these things in the product, instead of requiring people to input
boolean logic and generate custom reports.

Overall, Fleetsmith is just much more "turnkey" when it comes to checking the
boxes for SOC 2 than competing MDM solutions.

~~~
lvh
I'm guessing we have a different opinion on what "competing solutions" or
"accurate user <-> device attribution" or "setting up an LDAP server" mean.
I'm sure there's a version of this story where somebody picks Intune (it
manages everything!) and then decides to run their own AD install
(inexplicably) where that description is accurate, but we've seen a pile of
people deploy Jamf Pro/Connect to GSuite/Okta and what you're describing does
not match my experience.

With GSuite in particular, nobody set up any LDAP. It's an OIDC app, you do
not run Connect Verify or Connect Sync. There's LDAP going on when you're
authing against Azure, but if you're in that situation AD seems like what you
want?

I read your description as suggesting that if you pick anything other than
your product, there's necessarily an AD DS or slapd in your future, and I hope
we can agree that's definitely not the case. In the most common case for our
audience (startups) it's not even any LDAP at all.

Is it fewer clicks in Fleetsmith? Maybe? Probably? And you have to know
whatever the hell a "PreStage Enrollment" is which is not as easy as it could
be. But I think you're making it sound a lot more hairy than it is,
particularly for a deployment with "hundreds or thousands of devices". The
hard problem facing that IT team is not finding someone who is unafraid of the
words "Preference Domain", come on.

(To reiterate my position: I am not saying Fleetsmith is bad, I'm saying that
I think your post makes it sound like anything besides Fleetsmith is a world
of pain, and I accept that you probably extra-believe that as someone who
founded Fleetsmith (not in the sense of "you're lying" but in the sense of you
don't bet the farm on trying to fix things you don't think are broken), but, I
think a) whatever Fleetsmith is fine and yes it's probably better at doing the
limited set of things you should be doing but b) compared to our experience
this is a pretty wide exaggeration of how bad anything else is like.)

------
deedubaya
> SOC2 is about documentation, not reality

Yup. If you’re a fixer, knowing this beforehand is going to save you a lot of
pain.

------
statictype
What VPN tools support SSO? What's a recommended soution?

------
user5994461
>>> Web Application Firewalls: Most SOC2’d firms don’t use them, and most WAFs
don’t work at all.

Most startups are on Cloudflare and Cloudflare does WAF out of the box.

