
CloudFlare vs Incapsula vs ModSecurity - Comparative penetration testing [pdf] - dsl
http://zeroscience.mk/files/wafreport2013.pdf
======
bluetidepro
I tweeted CloudFlare asking for a statement about this, as a few of us
commenters have asked for (and wanted), and this was the response from their
co-founder...

> " _@bluetidepro @CloudFlare flawed testing strategy that assumes a last-
> generation, rules-based WAF. We've always worked differently._ "
> <https://twitter.com/eastdakota/status/307604650436210688>

> " _@bluetidepro @CloudFlare that said, we'll have a significant release soon
> that allows the people who want a rules based WAF to have it._ "
> <https://twitter.com/eastdakota/status/307604791054434304>

> " _@bluetidepro @CloudFlare but it's kinda silly because it's certainly not
> the future. Feels like taking a step backward._ "
> <https://twitter.com/eastdakota/status/307604919605661698>

(My tweet: _Hope @CloudFlare can make a statement about
this:<http://zeroscience.mk/files/wafreport2013.pdf> \- I'm pretty annoyed as
a paying customer._)
<https://twitter.com/bluetidepro/status/307603272645111808>

~~~
druiid
Wait, what? That is, I'm sorry... but a stupid response on their part. A SQL
injection attempt (successful or not) is an SQL injection attempt. If it makes
it to the server how is anything the 'future' or 'working differently', it
still made it to the server and the service isn't doing the job one is paying
for.

This response is so bad that I just did a literal _facepalm_. Needless to say
we will be getting in touch with them as well...

------
antr
Page 18:

 _"Though CloudFlare is presented as, besides other things, a very proficient
web application firewall, we concluded that that’s just a marketing sales
point and nothing more. ... CloudFlare does NOT protect from web attacks!"_

I'd really like to hear from Cloudfare on this statement.

~~~
druiid
As a Cloudflare user whom has been protected by these kinds of attacks in the
past... I too would like to know what the eff is going on here.

They supposedly (according to tech support in the past) USE mod_security... so
what did they break?

~~~
driverdan
You were protected from DDoS attacks or security attacks?

~~~
druiid
Both. We were victims of up to something like 21GBps DDoS attacks (sorry to
any other clients on that segment of the Cloudflare network... _whistles and
walks away_ :P).

We've also seen directed attempts from toolkits and the like for SQL injection
attempts, etc.

------
corry
Anyone else tried testing the security claims of CF? This particular test
looks pretty bad but it's just 1 case...

~~~
JimWestergren
Here is another test (October 2012) with similar findings:
[http://tonyonsecurity.com/2012/11/13/protecting-your-
website...](http://tonyonsecurity.com/2012/11/13/protecting-your-website-
cloudflare-or-incapsula/)

SQL Injection: Incapsula blocked all 30 attacks and CloudFlare blocked 1

XSS: Incapsula blocked all 12 attacks and CloudFlare blocked 0

RFI: Incapsula blocked 0 and CloudFlare blocked 0

I really like CloudFlare and hope they can improve this.

------
JimWestergren
I am getting a "Invalid Http Request Header" when I try to reach incapsula.com
with Firefox 19. Perhaps because I am currently located in Bolivia. A bad
false positive and they lost a potential client.

------
drakaal
CloudFlare does about as much for protecting you from web attacks as a condom
does in protecting you from a rabid pitt bull.

In fact it might even make things worse, since CF can be tricked in to
refreshing its cache of pages pretty easily, making it easier to create a DoS.

Also with very little work you can trick CF in to thinking a site is under
attack from Google's IP range which will trigger it to block Google, seriously
hurting your search rankins.

------
newman314
Waiting to see if eastdakota or jgrahamc will chime in...

------
david_shaw
I think it's worth mentioning that CloudFlare, while advertising its WAF as a
"pro" feature[1], gives it a one-paragraph blurb halfway down its feature
page. Don't get me wrong--I'm _not_ saying that it's okay to fudge features--
but CloudFlare is not primarily marketed as a WAF.

Among other security features, CloudFlare is designed to load balance and
protect sites from DDoS attacks, which it does well (though, like any service,
not perfectly). Additionally, it looks like they assign visitors "reputation
scores" that can have different effects based on the status of the
application. They also seem to use heuristics to "learn" about new attacks,
which while not the _fastest_ way to block attacks can produce some alright
results.[2]

I'd be interested to see whether the Zero Science Lab study used CloudFlare's
feature to allow them to manage mod_security rules or not, and whether that
would have made a difference. The PDF does state:

 _> Remember, we are using their Business Plan which should be an enterprise
WAF solution for your company._

... but it's unclear which features they enabled.

Last point: I've done a _lot_ of web application security assessments, and
while, yes, WAFs can help an organization defend against XSS, SQLi, LFI/RFI,
CSRF, etc., they are really just bandages over the much more serious issue of
_insecure code_. Developers have a tough job, sure, but there is _no_ WAF that
will protect you against all attacks. Maybe _' OR 1=1;--_ won't work, but
there are a _lot_ of ways--even emerging attack vectors--that security
software won't protect against.

That said, though, if you're going to rely heavily on a WAF, going with a
"sub-feature" like those presented in CloudFlare is definitely not the way to
do it.

\--

As I finished up this comment, I realized that bluetidepro posted some
comments from CloudFlare about the report:

 _> @CloudFlare flawed testing strategy that assumes a last-generation, rules-
based WAF. We've always worked differently._ [3]

This statement in particular furthers their own marketing; they're trying to
create security via heuristic rules, rather than a static ruleset. Really, a
combination would probably be best.

\--

Lastly: Although this comment might read as a defense of CloudFlare, I'd like
to note that I have no affiliation with the organization whatsoever; it just
seemed like a very one-sided argument.

[1] <https://www.cloudflare.com/features-security>

[2] [http://blog.cloudflare.com/thats-freaking-awesome-
cloudflare...](http://blog.cloudflare.com/thats-freaking-awesome-cloudflare-
automatical)

[3] <https://twitter.com/eastdakota/status/307604650436210688>

~~~
dsl
> .. but it's unclear which features they enabled.

they turned everything on/set to "high". It's mentioned in the report with
screenshots of the UI.

