
Clojure web security is worse than you think - r4um
https://hackworth.be/2014/03/26/clojure-web-security-is-worse-than-you-think/
======
ddellacosta
The talk he links to
([https://www.youtube.com/watch?v=CBL59w7fXw4](https://www.youtube.com/watch?v=CBL59w7fXw4))
is really fantastic. Kudos to Aaron Bedra for issuing a well-considered and
practical wake-up call to the Clojure community.

I support a Clojure/ClojureScript web app in my day-to-day work. I've also
written an OAuth2 workflow for friend ([https://github.com/ddellacosta/friend-
oauth2](https://github.com/ddellacosta/friend-oauth2) ...pull requests
welcome), one library Aaron Bedra spends a lot of time talking about. The
story really is exactly how he describes--Clojure web security could be
better. I suppose you could argue that, at least, the defaults are generally
not bad...but that's because there aren't defaults in many cases.

I think the main issue is, ironically, the same thing that makes Clojure
development so great: you end up composing a bunch of small libraries
together, each of which do one thing well. But as Aaron points out, there are
things that aren't taken care of and you can't assume they are done for you,
and a lot of security is something that you don't technically _need_ to get a
web app working...

I wouldn't want to give up the approach of composing a bunch of libraries to
build concise, easy-to-reason-about web apps in Clojure--I think it helps with
security in and of itself--but the community could really benefit from having
more libraries which nail these things down and help developers build web apps
more safely, with less work.

That said, I don't think it's all bad: you _can_ build high quality, secure
web apps in Clojure, it just requires you to really pay attention to security.
And we should be doing that anyways, right?

~~~
freshhawk
I watched that talk yesterday and it _was_ great, his list of issues and
concerns are very justified but I wasn't excited about the proposed solutions:
"just pick one templating language", "everyone use a (heavy, highly
opinionated) framework", etc.

They struck me as both unrealistic considering the clojure community and a
little too much "find the silver bullet" for my tastes. That said, I
understand why a security focused person would want this and if my role was to
be concerned about these things I would probably push in that direction as
well. When taking a larger view however, this is a poor tradeoff IMO.

"I wouldn't want to give up the approach of composing a bunch of libraries to
build concise, easy-to-reason-about web apps in Clojure--I think it helps with
security in and of itself--but the community could really benefit from having
more libraries which nail these things down and help developers build web apps
more safely, with less work."

You nailed it, well said. I want a checklist and good libraries, not a
framework that promises to handle things for you but quietly leaves a big hole
open in that one part where you had to work around the framework to solve your
problem (there is always at least one isn't there?).

You just can't avoid understanding these common attacks and how to mitigate
them, at least at a surface level. You have to at least know where to apply
the appropriate techniques, the implementation of that technique is left up to
a well documented library written by experts.

"That said, I don't think it's all bad: you can build high quality, secure web
apps in Clojure, it just requires you to really pay attention to security. And
we should be doing that anyways, right?"

Not only _should_ you be doing that, but if you aren't designing with security
in mind then you are doing it wrong IMHO, "secure by default" framework or
not. Either the people configuring the framework or the people writing the
layers at the edges need to understand web security.

All that said, I understand the push in this direction. If you have a large
team of junior devs then you want to limit their power. It's just not an
approach that fits very well with Clojure. It's like Rich Hickey's attitude
towards unbounded queues: there is no support for them, because you _must_ be
aware of this issue and hiding it _causes_ bugs. There, now I've invoked Rich
Hickey to justify my anti-framework bias ... so I must be right.

~~~
stcredzero
_" just pick one templating language"_

He has a point. Is one really that much better than another? Also, shouldn't
it be possible to come up with a single library to do HTML escaping that has
the suitably paranoid defaults, which is then used by all the templating
languages? Is one templating language's code for that going to be so much
revolutionarily better than another's?

 _" everyone use a (heavy, highly opinionated) framework", etc._

My understanding is that he pointed that out as a solution other environments
have taken (Rails) but that there was also value in the composable way of
doing things in Clojure. His actual position, I understood as, "there must be
_some_ way of unifying this for Clojure."

~~~
freshhawk
In regards to templates: yes - there are drastically different ways to do
templating. Enlive style templates are plain HTML files and clojure
transformation functions, hiccup style are clojure data structures that mimic
html structure and render to html and there are the placeholder template
languages involving intermixing a crippled language and HTML (Django
templates, Ruby's ERB, etc).

It would be business suicide to demand all your web designers learn clojure
edn syntax in order to write out HTML in hiccup. They have work to do and they
are experts with their existing tools, including quite complex IDEs.

If the html is being generated programmatically by clojure devs then hiccup
type formats are extremely efficient. String building techniques suck worse
for HTML than they do for SQL.

By far the most common types of template languages are the placeholder type.
People love them. I hate them, moustache/ctemplate style templates are an
excellent attempt to remove logic from them but why not just go all the way
towards an enlive style if you recognize this benefit? I'm not the right
person to defend them but the general argument is that designers can learn the
crippled DSL easily and they can implement the simple parts of the template
logic themselves. Moustache style templates are also useful if you are
templating something like plain text that doesn't have structure.

And yes, I absolutely think that the single library to do HTML escaping with
extremely paranoid defaults is the right solution. But a security person is
going to be nervous that developers will not use it every place they should
and unless I'm remembering incorrectly the request was for this to be
integrated into the templating library to remove that error path. I'm not even
sure how that could work properly outside of the placeholder paradigm (it has
been done well in Haskell using types to track escaped/unescaped segments if
memory serves).

In regards to frameworks: I think you are right about this position, I just
really very much disagree that there "must be some way of unifying this in
clojure" outside of an opinionated framework. The problem is the lack of
ability to enforce a specific dataflow / data transformations that can be
audited for security and generally applied. Enforcing a specific dataflow is
the definition of a framework IMHO.

I fully agree with his push for more and better web security related libraries
(I'm tired of doing these things myself but don't have the security expertise
to create general solutions that anyone should trust) but some of it struck me
as "if we just limit the expressive power of clojure for web development we
can mitigate a large swath of security issues". He's absolutely right, but I
have found that tradeoff to be a very poor one to choose outside of the
simplest of CRUD apps.

~~~
dllthomas
_" it has been done well in Haskell using types to track escaped/unescaped
segments if memory serves"_

Right, that's done in Hamlet (which was developed for Yesod, but is its own
library).

[http://hackage.haskell.org/package/hamlet](http://hackage.haskell.org/package/hamlet)

------
moondowner
The title should be: Don't assume security is happening automatically under
the hood.

Even if using a web framework which supports security out of the box, don't
take it for granted either. For example: that's a common error developers make
when using Spring Security. It's powerful and full-featured, and developers
get security and everything in a breeze, but when doing things on the fly
misconfiguration happens and vulnerabilities are out there in the open.

------
St-Clock
I believe this warning could apply to any young ecosystem or micro-framework
(e.g., bottle). Could web applications written in Go, or to a lesser extent,
node.js, suffer from the same shortcomings?

After working so much in Django, I've been interested in smaller, leaner
solutions, but then, as my feature list keeps growing, I find I miss all the
bells and whistles provided by larger frameworks. I did not think of the
security implication before reading this article.

~~~
route66
I would want to turn the question around and (genuinely, not rhetorically)
ask: which ecosystems right now _do_ have a good security baseline? Am I wrong
when I guess Java/.Net/Ruby and possibly more recent PHP frameworks?

~~~
MAGZine
Symfony on PHP has a great security baseline--though it's pretty complicated.

------
john2x
I felt some relief after watching the presentation. I thought I was just a
whiny newbie who was spoiled by Python/AppEngine (although I guess the spoiled
part is true). Turns out it is a real issue with the Clojure ecosystem.

Another talk from the conference I'm glad was presented is "How Clojure
Works"[1]. Towards the end, he shows the negative effects on tooling caused by
Clojure's dynamic + hosted nature. Clojure's stacktraces are a nightmare. And
the painful startup time kind of negates the excitement a newcomer might have
towards REPL-oriented development.

Hopefully it won't be too long until things improve.

[1]:
[https://www.youtube.com/watch?v=8NUI07y1SlQ&list=PLZdCLR02gr...](https://www.youtube.com/watch?v=8NUI07y1SlQ&list=PLZdCLR02grLp__wRg5OTavVj4wefg69hM)

~~~
leephillips
"the painful startup time kind of negates the excitement a newcomer might have
towards REPL-oriented development."

But you just need to start up once and then work all day in the REPL. Unless
you mean that the JVM startup means that Clojure is not a good choice for
writing utility scripts and such.

EDIT: As pointed out to me below, and in the excellent talk linked above, it's
much more the Clojure bootstrapping time, which almost dwarfs the JVM startup
time.

~~~
omaranto
The startup time that makes Clojure unsuitable for utility scripts and such is
really Clojure's, not the JVM's! JVM start up time is actually quite small
these days. See for example,
[http://nicholaskariniemi.github.io/2014/03/19/solving-
clojur...](http://nicholaskariniemi.github.io/2014/03/19/solving-clojure-boot-
time.html)

~~~
stcredzero
Heh. Back in the day, Smalltalk had a reputation for being too slow to write
command-line apps and utility scripts. There were even bits of wonderful tech
developed that could produce Smalltalk images as small as 45k!

But even more surprising: You could take the default VisualWorks, tell it to
skip the herald sound and the splash screen and shut a few other things off,
and it would load its 12MB default image faster than Perl 5 could start up
with a 768k memory footprint.

------
eudox
Web development is very particular because of security. You can write, say, a
console newsread with any programming language that has an XML parser and a
curses binding. But if you want to write a web application that will actually
see production, you need a web framework that has been sufficiently battle
tested and audited by security-competent people.

Which is a shame, because I'd really like to be able to use Common Lisp for
web development. But as much as I love the language, you have to be
reasonable.

~~~
userbinator
> you need a web framework that has been sufficiently battle tested and
> audited by security-competent people

Or alternatively, you just need to be careful with keeping things simple and
straightforward. IMHO frameworks only hide problems somewhere else beneath
layers of complexity. Assuming a fixed bug-rate, the more code there is, the
more bugs that will be present (and whether they get discovered is another
issue.)

~~~
adestefan
The argument is that making things _too_ simple is what kills you. The
defaults are not secure and you need to take steps to make sure you are
secure. This is one area where frameworks are huge plus. Good ones ship with
security conscious defaults.

------
whyme
I've spent the last 3 years developing in Clojure and found I was quite happy
not having an all encompassing framework. It forced me to learn all these
issues and understand them well enough to be able to customize each and align
them with a products design.

Sure it's more work, but in my mind, it's worth it.

~~~
phillmv
> It forced me to learn all these issues and understand them well enough to be
> able to customize each and align them with a products design.

This is a standard developer response - I recall when Sinatra was a fresh kid
on the block, and the same arguments people used against Rails - but just
because something is more fun doesn't mean you're going to get all these
details right.

The attack surface in modern web apps can be such a large search space that
the majority of people will never get it right. The point behind frameworks is
that you can engage in efficiencies in scale when it comes to sensible
defaults; even the most well seasoned and security paranoid developer can
drown in the cornucopia of details that all have to be done _right_.

~~~
whyme
> This is a standard developer response

vs. a what? I've worn many hats in my day, the least of which has been a
developer hat.

There's equal risk in relying on a code base incorporating features with
underlying concepts that you or your team does not understand.

I'm all for having a framework, where you can review the code and be
comfortable with what you're relying on, but in the absence of having a good
one, well then build one. The benefits Clojure brings to the table more than
make up for the work in either case.

There's just no excuse for not knowing the concepts and not knowing if your
product needs them.

~~~
phillmv
>The benefits Clojure brings to the table more than make up for the work in
either case.

No one's saying "stop using clojure". The argument here is "build a clojure
framework lots of people use".

>There's equal risk in relying on a code base incorporating features with
underlying concepts that you or your team does not understand.

You do this all the time. I mean, I understand on a high level what my
computer does but I can't describe to you the exact tree balancing algorithm
my filesystem uses, how my CPU performs cache invalidation and branch
prediction or even how exactly the JVM works.

Computers are too complex to _not_ rely on underlying concepts you or your
team doesn't understand.

The point is, within just the context of a web developer developing a web
application _there is too much stuff to know to always get it right_. We need
to reduce the number of things people have to think about, or else we'll
always miss something.

~~~
whyme
> The argument here is "build a clojure framework lots of people use.

The argument I am addressing is that people are deploying products with poor
security because a great, all encompassing, framework does not exist. There's
no excuse for doing that.

This is not about having to understand cryptography, it's about knowing when
and where you need to use it and making sure it's included when you do. The
issues outlined in the video are trivial.

~~~
phillmv
The excuse is precisely that there isn't a great, all encompassing framework.

Setting up a scenario where everyone has to get everything right all of the
time is setting everyone up for a lot of heartbreak.

Having bad password management and opening everyone up to CSRF and XSS is not
"trivial". They're gotchas, and looking out for gotchas is a waste of
everyone's time.

~~~
whyme
When I make mistakes, which I often do, I don't go around thinking oh, darn,
they should have known that for me. That goes for using a bad framework or
lack thereof.

As I said, I'm all for having a great framework, but I would not excuse the
mistakes being made here, and I'm simply saying that I believe that my
products and my knowledge level benefited by not having one.

> Having bad password management and opening everyone up to CSRF and XSS is
> not "trivial".

I'm not suggesting everyone should need to be aware of these issues, I'm
saying someone on your team should be. And if you're a team of one, be
prepared to make mistakes (which will not be limited to the issues outlined
here).

~~~
phillmv
>When I make mistakes, which I often do, I don't go around thinking oh, darn,
they should have known that for me.

Well, you should! These are mind numbing details that are best taken care of
by other people. I am too concerned with building a good product to want to
spend too much time going over mind numbing details.

The argument from "I become a more well rounded developer" is valid, but these
should be things you learn once and then stop worrying about forever. There's
a lot of magic in Rails, but it all goes away given enough time. Better
abstractions are our bread and butter and we'll get nothing done if we
continue to shy away from them.

~~~
whyme
As I said, I'm all for having a great framework.

------
manishsharan
Web Application is one of the places where Clojure's java compatibility falls
short. I have a ton of time invested in Spring + ACEGI security and it just
works. I have been exploring on how to develop servlets, which can use servlet
session, in Clojure and they all look very ugly compared to normal Compojure
based apps. I am coming to the opinion that maybe its not ready for J2EE web
apps , which is a shame as J2EE has a very rich ecosystem.

------
homakov
All non-mainstreem web frameworks are very immature in terms of web security.
No CSRF/XSS/SQLi projections built in, don't even say about security headers.
If you use an unpopular framework you should do all protections yourself.

~~~
mantrax3
SQL injection is not a problem of web frameworks, because _SQL should never be
a part of the web layer_ of an application in the first place.

~~~
joevandyk
Virtually all web frameworks encourage SQL. So it _is_ a problem. Making a
private api to access the database can add a lot of needless complexity.

~~~
mantrax3
That private API is called a service layer, and without it, your code devolves
into copy/paste spaghetti.

We live in an age when having a complementary iOS/Android app for your site is
not the exception anymore. Without a service layer you'll have coupled your
services with your web pages. Connecting your server with your apps will be
quite painful. You don't want that.

Done right, your service layers keeps your code simple, secure, and easy to
debug, even if it's web-only. I'd argue that anything else is just due to lack
of experience.

A service layer also makes the work on a project more parallelizable in terms
of number of developers that can work together on it, each focusing on their
part of the puzzle, without breaking each other's code all the time.

~~~
tensor
But then your service layer has potential SQLi issues. Moving the problem
doesn't make it go away. Many of the clojure SQL libraries solve this by using
prepared statements by default. This is not a hard problem.

~~~
mantrax3
If it's "not a hard problem" \- and I agree it isn't, and it's solved by
Clojure's SQL libraries, why should the _web_ framework deal with it in any
way?

And moving the problem actually helps, when the original place was the wrong
place to solve the problem.

Security is one problem when you write queries all over the place. The other
two problems are data integrity, and maintenance.

In a service layer you have exactly three concerns:

1) Validate _abstract_ input (and permissions of the caller).

2) Perform the transaction.

3) Return _abstract_ output (and/or errors).

No routers, no controllers, no views, no templates, no CSRF, no XSS, no HTML
escaping, no GET, no POST, no nothing.

Just input, transaction, output. Pure data. Pure business logic.

And suddenly things that seemed hard to get right, or things you had to repeat
all over the place in your code, get done simply, and just once. And all your
web code calls the service layer.

All your iOS and Android app code - also calls the same service layer.

And they're both secure because you need to get the service secure just once -
then all interfaces (web, mobile, desktop, API) use the same service.

And when you _don 't_ isolate the service, you'll be running SQL all over the
place, and use every framework's souped up solutions to avoid SQLi.

Which one do you feel is better?

~~~
lukeschlather
I've seen SQLi blindly passed from frontend code to a backend service API and
then executed. The indirection did help mask the issue so there was no
suggestion the attack succeeded or way that it could actually be exploited,
but it's perfectly possible to what you describe without actually solving the
sanitization problem.

SOA is great for scalability, but not really that much of a boon for security
(especially since you now have a problem of authentication between your
frontend service and your backend service which is often ignored since it's
behind the firewall and nobody is going to be snooping.) (Other than the NSA,
GHCQ, and China.)

~~~
mantrax3
You can always shoot yourself in the foot even with a service layer.

But in a service layer you need to get it right _once_. And therefore if you
don't, you have to fix it just _once_.

And coupling your business logic with your frontend layer suggests spaghetti
code and violates DRY, because you typically have many frontends, but one app
state.

Security is about focus. If the service coders can focus on the service being
secure & fast, frontend guys can focus on the frontend being usable.

Otherwise you're asking everyone to think about everything, and human
attention span, memory and skill sets are limited, and this _does_ affect
security.

------
danneu
When I think of other developers like me that are building projects solo on
top of Express/Sinatra/Compojure, the most helpful resource would be a simple
checklist.

For example, the CSRF section could briefly explain what cross-site request
forgery is, `<img
src="[http://example.com/logout">`](http://example.com/logout">`), and
recommend some approaches and good practices.

Code examples could be provided for various languages/frameworks each in its
own tab. The Clojure tab could demonstrate
[https://github.com/weavejester/ring-anti-
forgery](https://github.com/weavejester/ring-anti-forgery) and a `anti-
forgery-form-to` macro that wraps `hiccup.form/form-to` and inserts
`(ring.util.anti-forgery/anti-forgery-field)` into the form.

The XSS section could remind us to ensure that our templates are escaping user
input. The Clojure tab could chart the safe vs unsafe templating libs. (Hiccup
is unsafe)

The Sessions section could remind us to sign our cookies so that someone can't
just change a cookie from {:user-id 42} to {:user-id 1}. And that our never-
expiring authentication cookie means that anyone with that cookie can log in
as that user forever. It could explain an overview of the main solutions and
their drawbacks like how to back sessions by a database or set an expiration
within the signed cookie.

Just like convos about crypto, discussions about security too frequently turn
into some abstinence-only pamphlet.

~~~
stcredzero
_Just like convos about crypto, discussions about security too frequently turn
into some abstinence-only pamphlet._

As opposed to a "Purity Test" as a checklist?

------
borski
This presentation echoes a lot of what we've seen. For what it's worth, we
scan a lot of Clojure apps and find a lot of the types of vulnerabilities
Aaron/John are talking about. Feel free to run a free scan at
[https://www.tinfoilsecurity.com](https://www.tinfoilsecurity.com) if you want
to try it out. :) Would love feedback.

------
jmnicolas
So what are the secure by default web frameworks apart Django and Rails ?

~~~
e12e
For something fairly modern, you might want to have a look at web2py. It goes
to quite great lengths to balance sane-out-of-the-box and developer
productivity:

[http://www.web2py.com/](http://www.web2py.com/)

------
pc86
FTA:

> _and session cookies don 't default to http-only flags (allowing XSS attacks
> to steal them)._

Can someone explain that to me?

~~~
dminor
Setting this flag on a cookie means it can't be accessed from JavaScript, so a
malicious script won't be able to determine your session (for instance).

------
stcredzero
Perhaps a better title would be "Complete beginner web apps in Clojure less
secure than complete beginner web apps in other environments." (EDIT: Isn't
security a perfect example of a place where (good tech + good community) >
(miracle tech + bad community)?)

This brings to mind an incident, about which I'll say: While I do find that
the community is friendly and a source of satisfying discussion in person, the
Clojure community online is notably less beginner friendly than other language
communities. (tl;dr - Attempts to "help" newbs that are really opportunities
to be a language weenie considered harmful.)

I find some parallels with the Smalltalk community, which had a reputation for
an "ivory tower" attitude and a "Not Invented Here" snobbishness. When I was a
beginner in Smalltalk, a coworker of mine had written a nice but arbitrary
little DSL specification language using Array literals and symbols. So I made
some sort of comment about the syntax she'd created, and which prompted a
_knee-jerk reaction about how Smalltalk had a small non-ALGOL syntax_ \-- as
if I were totally unfamiliar with Smalltalk syntax.

I've had experiences like this asking about the arbitrary function definition-
like syntax of Compojure web application routes. Those look somewhat like
function definitions, but they're really macros. I had tried for some time to
Google the answer, but no one had ever posted about the exact code to do what
I wanted. (Which wasn't at all abstruse. Something like getting access to the
entire request map while modifying the returned session.) Well, it turned out
that all I needed to know was to not refer to the request as

    
    
        [request]
    

but as:

    
    
        request
    

But instead, I got people hounding me on IRC, saying "It's all made out of
functions" \- with what struck me as an almost cult-like zeal -
[https://www.youtube.com/watch?v=ZYTkSFRmKzM#t=5](https://www.youtube.com/watch?v=ZYTkSFRmKzM#t=5)
One individual snidely comments he'd put me on "ignore" and others kept
telling me to go read the source code, and how wonderful that it was all made
out of functions, and how wonderful it would be once I finally understood.
Well, I told them that I did understand that, and that "everything being
functions" had little direct bearing on what I was asking about. I am reminded
of how Smalltalkers got a chip on their shoulder from the circumstance of
Smalltalk's significant differences from mainstream. (Me included, I am
afraid.)

Just what kind of community treats with newcomers by assuming that the
newcomers are lying and then telling them to go read Lisp macros? Also, this
is only the most easily related situation I've encountered while interacting
with the Clojure community. There have been others.

No non-algol patterned language has ever gained more than a niche acceptance.
If Clojure doesn't want to commit long term mindshare suicide, some attention
needs to be paid to avoiding inadvertent hostility to newcomers. (This
includes unintentionally being a bad security-defaults minefield.)
Smalltalkers had the same "read the source" knee-jerk. In the end, it didn't
work out so well. Care needs to be taken to understand exactly what the newbs
are asking, because they know the least about how to formulate the question.
Also, if you find yourself running a simplistic "pattern match" head-daemon
eagerly looking for opportunities to bedazzle the newb with your wonderful,
superior, and different way of doing things, just stop. You are giving your
community a "weenie" reputation and your mental energies are better used
elsewhere.

~~~
reitzensteinm
For what it's worth, I regularly browse the IRC logs to see if anything
interesting has come up, and the only time I've ever seen any amount of
conflict was when you were involved - and not the incident you're describing.

They seem generally pretty helpful to newbies, from my own selfish perspective
to a fault, since I have to skip past the dozens of "how do I add to arrays"
with clojurebot responses.

It might be in the way you're asking.

~~~
stcredzero
_They seem generally pretty helpful to newbies, from my own selfish
perspective to a fault, since I have to skip past the dozens of "how do I add
to arrays" with clojurebot responses._

Lots of them seem to be stuck in "quick pattern match" mode, so in the other
occasions they were "pattern matching" on my asking about a much simpler
scenario than I was actually asking about -- then taking my trying to inform
them their assumptions were wrong to conclude something like "he doesn't
understand threads."

 _It might be in the way you 're asking._

I say as much in my comment above -- I don't know enough about how to ask the
questions in the most recognizable way. However, there seems to be an
eagerness to throw everyone who doesn't otherwise match in the "total newb"
bin, and once that happens, an inability to parse _facts_ presented that
should indicate that their initial mental models were wrong. ("You can't do
that with thousands of threads."/"its dozens of threads" \-- (EDIT: and
actually, it can be as few threads as one wants.))

I posit this: There are other people like me who have asked non-typical
questions, were treated in this way, then just turned their backs on Clojure.
I just happen to have a very strong allergy to counterfactuals and make noise
in this situation.

------
mantrax3
Many things said in that video are kinda off.

He mixes password hashes and MAC (message authentication codes), saying we
should HMAC our password hashes. There's little to gain from that:

\- Password hashes exist to hide passwords. The attack is to discover the
original password by brute force, dictionary, rainbow tables etc. The
protection against this is irreducibly slow hash algorithms and unique
_public_ salt per hash.

\- MAC exists to authenticate message origin. Unlike passwords, the message is
often public. The attack is therefore _not to discover the message_ , but to
manipulate the message, yet make it appear from the same origin as the
original message. The protection against this is the HMAC algorithm with a
_secret_ key that only the original message author has.

HMAC-ing your password hash won't stop, or even significantly slow down any of
the attacks performed against password hashes, so it feels cargo cultish to
claim it's just magically more secure by using it. It's even more WTF-y to
claim a password hash library _should_ offer HMAC, or it's insecure.

Once a system is compromised, and the attacker doesn't care to recover the
original password, it's far more trivial to just replace the hash or just
directly read the data protected by the login directly in the DB, rather than
play out a MAC attack on a pass hash. It makes no sense.

HTML escaping. You either escape for HTML or you don't. There's no sane way to
escape _some of it_. His idea that we should be able to escape some tags, but
skip (or "strip") others is a huge vector of attack as HTML is notoriously
filled with edge cases that your _selective escaping /stripping engine_ won't
have. It's _not trivial_ to parse what's a tag and what isn't a tag to a
browser, trust me. It only looks trivial if you're ignorant.

If you want to allow some tags, and disable others (say, allow bold/italic on
forums, but nothing else) you should go through a strict DSL allowing only
those types of formatting, which are then _converted_ to HTML. Just like
HackerNews does it - I type star-word-star and I get "word" in italic. That's
a DSL. That DSL _could be made_ to look like HTML, but it won't be HTML. It's
fully parsed and rebuilt from the DSL syntax tree. So it can be made safe.
Selective HTML escaping without those crucial steps is almost impossible to
make safe by comparison.

His rant about "there are too many templating engines - we should stop" \-
yeah, good luck saying _we should stop_ and this working out.

Likewise about databases. "Let's just stop". B.S. The thing he's missing is
that different DB engines have different strengths and weaknesses. That's one
major reason there are so many, and why many get used in the same project.
PgSQL isn't SQLite isn't MongoDB isn't Redis.

By enforcing one type of database for multiple separate project components,
you don't improve security, you just cripple architecture and performance.

~~~
emidln
I'm pretty sure the intention behind something like this:

    
    
        (def a-password (-> pass
                            (hmac server-stored-key)
                            (scrypt-kdf work-factor))
    

is to prevent all information necessary to calculate the stored hash being on
the same system where the hash is stored. By HMAC'ing and not storing that key
in the database (but on the app server), you now make the attacker pull off
two attacks.

You could replace HMAC with encrypt with AES (or scrypt for that matter) to
get the same practical effect.

Edit: clarification

~~~
mantrax3
The problem of lookup table attacks is solved by the public salt I already
mentioned. HMAC protects against attacks like length extension (google it),
which aren't applicable for discovering a password from a hash.

Encryption also does nothing here, because it's reversible.

Your encryption key is somewhere on your server, so whatever you encrypted
(either the plain text password, or the hash) can be decrypted once, and then
if it's a hash you just attack the bare hash as usual (brute force, dictionary
etc.).

Instead of encryption you can just store part of the salt outside the DB, but
with a good hash this is usually not needed and just pointless obfuscation.

~~~
emidln
Consider this scenario (which is extremely common):

Public access to a SQL database. Input isn't properly sanitized while building
some complicated query. $badnick executes a SQL injection. Luckily, the
database user from the public site only has SELECT privileges (yay SOA or
something). This is an extremely common enterprise situation.

The attacker can dump data (like password hashes), but in an HMAC'd password
setup, cannot actually gain access to modify or login as a user via brute-
forcing. They cannot pivot the user account to access to another system. The
attacker must find a different hole to gain access to the app/login server.

~~~
mantrax3
Well first of all your "common scenario" terrifies me, and betrays an app
developed by incompetent devs who wouldn't even know what a "hash" is, because
in any sane situation:

1) The SQL database is not public (any more than it's common to have public
anonymous FTP to your server's app deployment). It listens on a specific
interface, specific port, from specific IPs with a specific user and password
(which is _not_ "root"/"").

2) Input isn't properly _sanitized_? Hello? If that's "common" then fix _that_
first. It's not 1998 anymore, no one has an excuse for that level of
ignorance. First of all "sanitize" is an incorrect term for what needs to be
done. You either escape an SQL literal, or you pass it as a parameter to a
prepared statement. You do this to all data, _all_ DB layers provide escaping,
and it's absolutely trivial to do. And for prepared statements, there is _no_
way to do it wrong, because it's not you who does it. Data and query in
prepared statements are separated at the protocol itself and injection is
impossible, you can't possibly mess it up. All of this is "using databases
101". Heck, your app will break even with innocent data containing, say, a
quote, if you fail those.

3) I mentioned twice already, there's public salt for hashing the password,
and there's irreducibly slow hash to be used to protect against a weak
password (like Blowfish). This is what breaks brute-force attacks, not HMAC.

4) Let's assume your exact scenario is indeed terrifyingly common. Still, HMAC
does nothing at all here, that keeping part of the salt off-the-DB won't do.
HMAC doesn't mean "when you have a key outside the DB". You don't need HMAC to
keep part of the salt outside the DB. Just... _keep part of the salt outside
the DB_ , without HMAC.

But if you feel the need to obfuscate your code like this, it means you're
working in hell and some people don't deserve their jobs, and better fix
_that_ , before you start thinking about compensating for it at the wrong
place of the app.

~~~
emidln
I should clarify that when I say "public access to", I mean specifically that
a public person can access an application talking to the SQL database.

I.e. User <-> App <-> Database

Input is simply not properly sanitized in the real world because mistakes
happen. This isn't by design. This is a source of remote execution. The remote
execution only yields read access since the app only needs read access (and
thus the fewest privileges possible are read access). The attacker can read
password hashes, but cannot update them.

As I mentioned before, HMAC is just an example of keeping all information
necessary to login to the site out of the stored hash. Any non-constant value
per password will do (feel free to maintain a lookup table of unique lists to
active passwords in your app if you feel you must, I'll be HMAC'ing the user's
input password against a constant secret key before sending to KDF). The idea
is that an attacker who can read your database (but not the application code)
cannot gain access as a user without first also compromising the application
code.

~~~
mantrax3
Once again there's no way to make a mistake with a prepared statement. The
only way to do it is using poor practices like using half baked "sanitizing"
functions which no competent developer will use.

And once again, you don't need _HMAC_ to store salt outside the hash. You just
don't. HMAC doesn't dictate where you store your salt. You're bundling these
two things together as if they're inseparable, but they're two completely
separate things.

~~~
frio
Security's something you do in-depth. You might build a perfect app, which
correctly sanitizes your DB inputs (and you're right, it's easy these days).
That won't necessarily stop $developer making a little PHP interface long
after you leave, which exposes your DB again.

I'm not up to the play with HMAC so can't confidently comment on that, but the
point being made is that _there is lots of SQL injection out there_. Keeping
the salt out of the DB seems like a not-insane way to help mitigate a
compromise.

~~~
mantrax3
> "That won't necessarily stop $developer making a little PHP interface long
> after you leave, which exposes your DB again."

Let me ask, does this hypothetical company have someone in charge of
architecture, or everyone just codes randomly entirely on their own with no
rhyme or reason until it works? Because that's what you're describing. Anarchy
& chaos, with no one in charge.

The answer to the above hypothetical situation is that the PHP developer will
code to the service layer, not to the database. In fact, that precise setup is
very common in the projects I do: services (say in Java) consumed by delivery
mechanisms (say PHP web sites).

Multiple independent apps reaching into the database directly is not only
recipe for a security disaster, but a recipe for data integrity and
maintenance disaster.

Here I explained service layers, so I won't repeat myself:
[https://news.ycombinator.com/item?id=7476725](https://news.ycombinator.com/item?id=7476725)

