

Log Files Do Not Improve Security - lmacvittie
http://devcentral.f5.com/weblogs/macvittie/archive/2009/09/09/log-files-do-not-improve-security.aspx

======
acdha
It doesn't seem like the author has much experience with security, which is
what you'd expect from a technical marketing manager - it sounds reasonable
but the real goal is convincing you to buy Ronco spray-on-security appliances.

Log files have never been intended as a security measure in the same way as a
firewall, but it's naive to claim that they don't improve security. The
concept the author is missing is that security is a process, not a software
feature. Log files address several key parts of the security process:

1\. Damage containment: if someone does compromise a system, logs are your way
to catch that as early as possible - the difference between, say, someone at
your bank getting malware and their authorization credentials being stolen.
Identifying failures can be done close to real-time so it's realistic to be
able to do things like quarantine desktops which suddenly acting like botnet
nodes.

2\. Verifying normal functionality: things like security updates being
installed _and_ services restarted, whether your admins are following correct
policy, etc. This stuff matters a lot in any large-scale environment and while
you can get a lot using a security scanner, logging is faster, safer and
easier in many situations.

3\. Identify anomalous behaviour: the classic example is adding firewall rules
to drop traffic from hosts which attempt noisy attacks but this can also apply
to things like banks notifying their customers that their browser is outdated,
showing signs of being compromised, etc. Having actual data makes many
security decisions a LOT easier.

------
colbinator
Monitoring logs may not BE security in the classical sense, but it should be a
security enabler. Things like:

* why am I seeing ingress/egress traffic to an IP I thought was firewalled? * why am I seeing triggers to IDS signatures on hosts I thought didn't communicate in/out to the internet? * why am I seeing web traffic bypassing a content filter/proxy I thought everyone had to use? * why am I seeing router/switch VLAN/routing errors that could expose more of my network? * if I continue to buy all of those technologies, how do I monitor them to know that they are functioning/doing their job/reporting any exposure? * how do I know if I've been exploited/exposed?

If you use it right, it isn't just a post-attack/exposure response and
investigation system, it can be an early warning system, and an aid in your
normal incident response techniques. If you do get exposed (which
unfortunately does happen to most organizations - whether it's a simple policy
violation or a worm infection or worse), it can help with containment (who is
still infected?), patient zero identification (when/where did it start?), and
mitigation itself in some cases (though that's not always practical or
possible).

For a lot of people/teams, knowing really is half the battle. A lot more are
strapped for resources and using a tool to extend the IT team does enable
better security in that it enables them to focus their time somewhere other
than monitoring.

~~~
lmacvittie
"Monitoring" was not part of the equation and that's part of the problem. The
implication in the cited article was that logs IN AND OF THEMSELVES result in
better security. That's simply not true. Monitoring logs is better, but
analysis and even reactive tools that make use of those logs is better. That
improves security, but log files don't..

Absolutely agree that logs are an enabler of security. Either someone needs to
look at them (analysis) or they need to be leveraged via third-party tools (if
you're looking to do more real-time analysis or catch a breach in progress).

And very good point on the investigation aspect - I was thinking more external
facing (breach) but there's also good for internal facing
(infection/containment).

~~~
colbinator
Yes, ma'am, we are on the same page. :)

------
shykes
I disagree. Log files greatly improve security _if you read them_. When you
spot problems early, you fix them early. Would you rather spot a burglar as he
lockpicks your door, or as he's jumping out the window with all the jewelry?

Use logcheck to filter out benign messages; have _everything else_ emailed to
you in real time.

~~~
pasbesoin
A while back, someone (here, I think) cited an article / blog entry wherein
the author describes his log reading process as first filtering out all the...
I'll use the word "typical" entries. He identified patterns for these and lets
a script strip these out. As he continues to identify those types of entries,
he adds patterns for them to the script.

He then focuses on what remains. At least as described, it sounded like an
effective way to focus attention on the needles in the haystack.

Note that this doesn't mean shrinking your logs. It means an automated process
for extracting an interesting subset. You still have the complete logs for
other work. I'd also regularly audit the extract process both for sufficiency
and as a reminder that its output is not the only thing you should be paying
attention to.

It's not my idea, and I'm probably not doing it justice. Maybe someone else
recalls the article / blog entry and has a link. It's also not something I
need to do; perhaps if I did, I'd see shortcomings in the methodology.

~~~
shykes

       > [...] first filtering out all the...
       > I'll use the word "typical" entries.
       > He identified patterns for these and
       > lets a script strip these out. As he continues
       > to identify those types of entries, he adds
       > patterns for them to the script.
       > He then focuses on what remains.
    

That's exactly what logcheck does. <http://logcheck.org>

------
bartw
They can help in prosecuting offenders, and in that way be part of a deterrent
that improves security. If security is measured in a way such as (average leak
size/mean time between leaks)

------
peoplerock
As a novice at security issues, I find the discussion _sounding_ helpful, but
would appreciate HN discussion.

The general issue regarding security and blogs seems obvious: Shouldn't logs
be considered one element in security _analysis_ ? - not something I see
discussed in my short time here. Anyway, it gives me pause to think, "Where
have I implemented log-analysis in my (or my cron's) routine... and planned
for follow-up?"

~~~
jwilliams
All boils down to the fairly generic statement in the last line:

 _Logging is an integral part of organizational security policies and best
practices and well it should be. But don’t make the mistake of thinking that
logging access to records is the same as securing them._

------
rman666
Here's a weak argument to say that log files can improve security:

IF

* security means reducing risks to the confidentiality, integrity, and availability of information;

* risk is a function of likelihood and impact;

* the longer an attacker has unauthorized access to information, the bigger the impact; and

* if log files are reviewed frequently

THEN

* unauthorized access can be detected;

* corrective actions can be taken;

* unauthorized access can be stopped or limited;

* thus reducing the impact;

* thus reducing the risk; and

* thus improving security.

But, in reality, the corrective action reduced the risk, not the detective
control (logs and reviews), so this is a weak argument.

------
lazyant
The title is trollish. As explained in the first sentence of the article, logs
don't provide "real-time security", this is very different than "they do not
improve security". Sure they do.

One pretty standard way to classify security counter-measures is in the three
areas: Protection, Detection and Response/Recovery. Logs play a very important
part in the detection area.

------
tsally
Log files have _never_ been about improving security.

