

Sleepy Puppy: XSS detection tool - pm24601
http://techblog.netflix.com/2015/08/announcing-sleepy-puppy-cross-site.html

======
WA

        With delayed XSS testing, security engineers inject an XSS payload on one application that may get reflected back in a separate application with a different origin.
    

No need to go beyond application scope. Most tools even fail to detect XSS if
it is reflected not in the immediate response, but in a later response. They
inject the XSS payload and check if the immediate HTTP response contains that
very string. If not, _there 's no XSS._

Maybe Sleepy Puppy is a solution. You could now define some standard click-
paths through your app with Selenium IDE and inject various payloads into
every parameter. I didn't take a deeper look in how Sleepy Puppy works, but
automated workflow execution + payload injection + delayed XSS detection is
what matters in modern complex web apps.

------
altano
Has anyone demonstrated XSS attacks that work even when CSP is properly
configured? Seems like all discussion of XSS now-a-days should be preceded
with a note that you should go turn on CSP before worrying about any other
form of prevention.

~~~
david_shaw
From what I've seen, no -- but that depends what you mean by "properly
configured."

If you look at challenges like this (
[https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-
Minicha...](https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-
Minichallenge-3:-%22Sh*t,-it%27s-CSP!%22) ), you'll see that CSP _appears_ to
be properly configured, but there are actually ways to bypass. It's problems
like this that I've seen "in the wild" most of the time.

The other side of the argument is that not many people are building web
applications with CSP built-in from the beginning. If you don't start
development with CSP in mind, it can become very cumbersome to put CSP into
place (outside of report-only mode). As a result, a lot of people are
currently using CSP as just report-only analytics on potential violations.

------
billyhoffman
Sleepy Puppy doesn't really address the primary problem I've found when trying
to automate the detection of non-reflected XSS issues: Mapping the input
points and outputs outputs

The first thing you need to do when testing for XSS is you need to find where
a user-supplied input is returned back out to a user. In reflected XSS, its
the page you sent the request to. With "Stored" or "Delayed" XSS, it's
somewhere else in the app or even a different app entirely.

Building this mapping when input and output points are in different areas, and
doing that automatically has 2 challenges:

\- Clobbering your input

\- Finding the output point.

Clobbering input is something automated tools are terrible at. Let's say you
are auditing a "save my settings" functionality in an e-com store, but the
address values you save aren't actually displayed until later in the checkout
process (i.e. you are dealing with a stored XSS). Let's say the vulnerable
input is the state name (devs often forget to validate input when the input is
a dropdown, because they forget people can make that input be anything they
want).

You submit an attack string for the address, and valid input for state and
zip. You don't see it reflected. You then do valid input for address and zip,
but state has an attack string. Nice, but that doesn't appear until to later
in the app. As an attacker, you don't know this, so you see its not a
reflected XSS, and you do a 3rd attempt and submit valid input for address and
state, and attack the zip. You have now clobbered your attack string that you
submitted against the State with a valid, normal input for the state. When you
get deeper into the app and see the state returned in the checkout process, it
wouldn't be rendering your attack probe. That got clobbered by the valid
input.

Unless you inject an attack token, and then sweep the entire site for that
token, you won't find most stored/delayed XSS. Probe address field, crawl rest
of site looking for your probe. Probe State field, crawl rest of site looking
for your probe. This is super time consuming to do.

Automated tools suck at this. Humans are a better, because they understand
context. I know that an address associated with an account is a likely stored
XSS vector, and I have a gut feeling about where in the app the address will
be rendered. Knowing context helps you also prune the number of inputs you
should test for stored XSS. Unless I find an admin or analytics area in a
site, I'm not going to bother doing this work to test if an HTTP header like
User-Agent or Referer [sic] is a stored XSS vector as it is highly unlikely
those inputs are reflected out somewhere else in the site.

If you have an output in another app its super hard. Sleepy Puppy tries to get
around this by having the XSS payload "phone home" and pull an external JS
file for a site they control to register where the output point is. While this
is the best attempt I've seen, it has short comings. CSP defeats this. Also,
it means the vulnerable input has to allow far more more characters to do the
attack. spaces, quotes, parens etc cannot get filtered or encoded for the
various Sleepy Puppy payloads to work.

This is a hard problem to solve. I'm glad NetFlix open sourced this.

~~~
pm24601
I agree with you about Sleepy Puppy not being a universal solution - however
this is true for almost every tool.

The big wins with SP:

* automated

* narrows what kind of XSS can be present.

Security seems to be like quantum mechanics: there is always an uncertainty
principle that prevents perfection.

