Hacker News new | past | comments | ask | show | jobs | submit login
Stripe CTF Writeup (ioactive.com)
126 points by santaragolabs on Aug 30, 2012 | hide | past | favorite | 89 comments



First ctf here too. Though some people are claiming people were posting answers to irc, I didn't see that happen once. Though, I didn't start lurking until I got stuck on level 5.

What I did see was an awesome group of folks willing to help nudge others in the right direction. Many folks would stick around in prior level channels once they solved the level and offer pointers as to what to look for and where to look for it. Awesome.

I convinced myself it wasn't "cheating" because a) I'm a total noob, and b) sometimes pentesting happens with a team where people bounce ideas off one another.

I spent probably 20 hours trying to finish level 8, and even though I didn't end up getting it, this was a great experience. I'm fully inspired to get better at python and actually ctff next time.

Also, again, awesome how cool the people in irc were.


I spent a lot of time on IRC in #level8, but people were good enough to not just post the answer. Somebody let me bounce my ideas off of them, and they helped me just step back and think clearly, which was all I needed to get it.


Most of the time, folks on IRC made this sound way harder than it actually was (my mistake!). That threw me off as I started premature optimizations with threads, pipelining and multiple async webhooks. Spent a lot of time there, and eventually I couldn't get it working on the production machine. Then I just wrote the 'naive' solution and that just worked fine.


I saw the full solution (not the code, of course, but the idea) posted in #level8 several times when I was hanging out there on Friday. There was also another channel called #level8spoilers or something that posted the full solution with even more frequency.


I received an explanation of the level 8 vulnerability via private chat, after hanging around in IRC for ~10 hours. By that point, I knew about (and easily implemented) SSH, and had inferred several other useful tidbits, but didn't quite understand the port-counting vulnerability. Partly, as I had a mental block against "timing attacks" based on the instructions ... this was not technically a timing attack, but was certainly very timing dependent.

I still wrote the code myself.


I jumped in the irc on level8 to see if people were hitting the same issues as me. It struck me that people were conducting themselves very well.

Awesome data for Stripe to hire against - they have a list of very technically competent people and a log of how they conduct themselves in a public forum.


Agreed, the IRC channels were fantastic while I was still working through the levels, but when I joined much closer to the end (the last few hours before it ended), there seemed to be a fair amount of people just asking for code to run, and a few people obliging. It was really kind of disappointing after working all weekend on level 8, to see someone complaining that they only had two hours to get someone else's script working and needed help.


Heh... admittedly I didn't spend much time in irc in the waning hours. Was trying desperately to get my code to work. But yeah, that's kind of a bummer.


For levels 4, 5, 6, the Sinatra configuration used client-side cookies for session and the session secret is exposed in the error page. Crashing the app and injecting custom values into the session cookie was how I got through all 3 of them. I did feel like I'm missing the point of the challenge somehow.


Nice! I don't think this is how you were supposed to do it, but you could submit this as a bug to the Stripe team, or even suggest it as a new level for the next CTF.


They know and they already fixed it.


This is what I did for level 5 without realizing it wasn't the intended solution. I actually feel like it was a more difficult and relevant to real world attacks than the actual solution.


So between Sinatra's default error message giving away sensitive information and the suggested "params[..]" approach gets ALL input variables, regardless POST or GET.

What other gotchas should be know about Sinatra development?


Basically every framework has a development mode that gives very descriptive error messages, and you're always supposed to disable it in production.


Can you elaborate on injecting session cookies? If the secret was in the error page why did you need to do anything besides crash the app?


The "secret" exposed in the exception page wasn't the actual flag, it was the secret used to sign session cookies. Once you had it you could modify your session cookie (typically to pose as a different user) and re-sign the cookie with the secret.


The secret is used to create a valid cookie with the required values in the session e.g. User = karma_fountain. After setting the cookie, I can use the browser to transfer karma to my account, since the browser will be logged on as karma_fountain.


First time I took part in one of these challenges and I enjoyed it a lot. I made it only up to level 3 and seeing comments of others at higher levels saying it was easy, wasn't very good for my self-confidence :-) However, I can only assume they had some prior experience in this field and / or had an IQ of 150 (http://abiusx.com/stripe-ctf-2-web-challenges/).

After having spent more than a day on level 3 I decided to look on-line for solutions and came across the aforementioned website. I was happy to see my attempt was pretty close but looking at the rest of the levels it was obvious I was out of my depth and gave up (it was already Tuesday anyway).

These write-ups are nice (although abiusx' ought to have been posted only after the challenge was finished) to see how others have solved the levels in different ways.

I can't wait for the next challenge from Stripe and will definitely check out some others on-line. It's like puzzles for grown men.


I got stuck on level 3 too, unfortunately after reading this I had the answer the entire time! I had been trying an SQL injection but couldn't get it to work, so eventually I assumed maybe format() does something weird (I've never used Python before) and that is blocking the SQL injection and there must be another solution. I must have been malformed SQL or typoing, hah.


Haha, same here. I tried so many things including trying to exploit .format (not much Python experience here either) and finally decided that it must be the SQL injection. Didn't solve it though, but got close according to the write-ups.


For future reference: When you run into a problem where you think that syntax errors are the cause, try to replicate the database on your own computer.

In this case, the database schema and the code to access the database is known. The errors that are thrown when trying to exploit it locally will help you find a solution :)


For me I didn't have much time, but got stuck on the wrong track on one level, so I gave up and went back to work ;) I think some of them were pretty difficult puzzles, so I don't think we should feel bad about our abilities.


Great, I've been waiting for this. Now I can finally get past level 2 -- I didn't realize a non-existent file would work, even though I did figure you could overwrite variables due to the get. I considered overwriting filename with "index.php" and figuring out what attempt would have to be to pass -- but that was too much work for a simple challenge when I had enough other stuff to do ;-)

At least I identified the "correct" vulnerability.


I spent a great deal of time using index.php and README as filename parameters, and using the contents of those files as the password attempt. This attack worked in my local environment, but not on the live ctf. It took a while to realize that `file_get_contents('nonexistent') === ''`, raising no exceptions.


I used README and just manually stripped out all the whitespace. It worked fine. But a non-existent file is way more elegant I suppose.


file_get_contents('nonexistent') actually returns false, but calling trim() on that coerced it into an empty string.


I actually uploaded a txt file to dropbox and then passed that in and the contents of the file as the two parameters. It worked for me, although that method seemed to stop working later on.


I didn't notice it either, so I requested the root URL and the response from there as the attempt.


Relying on php to open an url? Or do you mean filename="/" ? What did your attempt end up being? (Apparently they've taken down the ctf-servers, so playtime is over).


Yes. You can do $x = file_get_contents("http://news.ycombinator.com) and it will work (well, in that case it wouldn't as the machine had locked down network access, but you get the idea).


Yes, I am aware php allows one to open urls via the file*-procedures (unless it has been disabled on the server -- which is generally a good idea for production deployment). I meant -- what did you end up using for the challenge (url/$filename and $attempt)?


filename=file_that_doesnt_exist&attempt=


Yes, for non_existent_file I got it -- I was actually asking gee what (s)he ended up with, using "root url".


I used the server 1 root URL (https://level01-2.stripe-ctf.com/) and the response from there as the attempt.


I can't check it right now but I do wonder whether /dev/null as a filename would have worked.

Does somebody know?


I guess it would've. I tried to think of stuff that lives under /etc mv -- didn't think of /dev/null. However the "empty filename fall trough" is better as it also works under a chroot without such files available.


Agreed, and I used it but I was really surprised that it worked!


You could also pass it nothing, e.g. ?filename=&attempt=


Yes, I used /dev/null.


I spent a while trying to do a path traversal exploit to set password.txt readable, trying various encodings along the way. I head-desked when I realized I could just upload a PHP script.


I learned another interesting way to create strings in JavaScript without using a quote character:

    var s = /string you want/.source;


Another way is to use the technique described at http://patriciopalladino.com/blog/2012/08/09/non-alphanumeri...

It transforms javascript into a sequence of ()[]{}!+


Wow, it seems like there is no limit to what you can accomplish by abusing type coercion.


unfortunately i ended up with String.fromCharCode, would of made the final code a lot more readable! https://gist.github.com/3527252


I figured out how to insert strings with quotes on level 6 - if you use a param list like username[]={string"with'quotes'"}, it bypasses the safety check but still gets coerced to a string by the ORM. Unfortunately, I wasn't clever enough to actually do anything with that...


This works well if for some reason they block fromCharCode:

    eval(unescape(/your escaped code goes here/.source))
Use Javascript's escape() to generate the code


I used DOM api to avoid quote character: document.links[0].href


As various people are commenting on #8: modern server operating systems do not do this behavior "by default". The hints from Stripe were all "run it and it will be obvious", but while the exploit worked fine on my laptop, I went to the trouble of duplicating the setup they had as exactly as I could, setting up the same version of Ubuntu they were using in the same EC2 region (which then implies the same version of Twisted: 10.0.0, despite their being adamant about 11.1.0, which I figured might even be the problem), and it didn't actually do that: reasonable server systems are configured, out of the box, to randomize that side channel. I personally believe that this is part of why there was such a cliff when everyone hit level #8 with regards to difficulty in solving.


So, as far as we could tell, there is actually no way to turn off sequential source port assignment. We reproduced the vulnerability in a number of environments in the wild (including several shared hosting providers and a Mac OSX system), and dug into the relevant kernel code. The only kernel I know that has TCP source port randomization as a feature is grsec: http://grsecurity.net/confighelp.php/. Without patching, your kernel should be vulnerable -- I'd be very curious to hear if you're seeing something different.

Incidentally, there's a relatively recent RFC on TCP source port randomization: http://tools.ietf.org/html/draft-ietf-tsvwg-port-randomizati.... There's a decent amount of mailing list traffic on implementing this for Linux, but it doesn't appear anything has been mainlined here yet.


I can confirm that it was definitely doing sequential assignment on my Ubuntu 12.04 virtual machine. As soon as I figure that out I was a bit surprised that the OS would let that slip through.


Ok, wow, that's really irritating: what I had been staring at was the output from subsequent connections to different hosts (such as between the backends, or between the backends and the webhooks)...

    # ./password_db_launcher 012345678901 127.0.0.1:3232 2>&1 | grep Received
    [127.0.0.1:34651:1] Received payload: '{"password": "012345678901", "webhooks": []}'
    [127.0.0.1:39664:1] Received payload: '{"password_chunk": "012"}'
    [127.0.0.1:34991:1] Received payload: '{"password_chunk": "345"}'
    [127.0.0.1:42667:1] Received payload: '{"password_chunk": "678"}'
    [127.0.0.1:33434:1] Received payload: '{"password_chunk": "901"}'
...but it turns out that to a single host source port assignment is still both sequential and mediated by the same global sequential counter...

    [127.0.0.1:35502:23] Received payload: '{"password": "112345678901", "webhooks": []}'
    [127.0.0.1:38011:23] Received payload: '{"password_chunk": "112"}'
    [127.0.0.1:35504:24] Received payload: '{"password": "112345678901", "webhooks": []}'
    [127.0.0.1:38013:24] Received payload: '{"password_chunk": "112"}'
The algorithm for determining the ephemeral source port apparently has this behavior because it involves 1) entropy that is generated once at boot, 2) an md5sum of that entropy with the destination/source address and destination port, and 3) a hashtable insertion with linear probing.

As #1 is at boot, it doesn't affect the result in a way that really matters, but as #2 is based on other properties of the connection, we get behavior that looks random between backends and webhooks, but is in fact deterministic to the webhook.

What bugs me, however, is that hashtable: it is treated as a hashtable with open addressing and linear probing; the hash value that it starts with is the aforementioned md5sum: it really shouldn't end up allocating a sequential port to outgoing connections across hosts (it should only do so per host).

Except, I would assume to make it more efficient (as otherwise the hash that is going into it is going to keep returning the same thing if you are making lots of connections to one place), there is a "hint", and the hint is a static integer that is incremented for each port skipped.

So, you could seriously delete that one line of code from the kernel, then, and "solve" this at some minor performance expense. Alternatively, you could add the high bits of port_offset instead of 1 and get a quite reasonable middle ground.

Regardless, there: that's the story of why I've been operating under the assumption that Linux has not had this problem for a very long time, and exactly why I was wrong (down to the individual line of code that screwed me: inet_hashtables.c, line 521).

That said, it only somewhat changes what I was saying: I am curious if the level was designed and thought about on a device that had truly sequential port numbering. It definitely is incredibly obvious once you run it on, say, a Mac OS X laptop, that the port numbers are a side channel.

However, if you run it on Linux, you are going to have to sift through the sea of port numbers and notice that the connections to individual servers is incrementing even though, for the most part, the port numbers you are seeing look mostly like garbage.

(By the way: except for the massive difficulty cliff at level 8, this contest was amazingly well put together. I believe I said something similar already about the previous one, but I'll also say it again: that one was also amazing. I'm really curious if you have some kind of need to hire an entire army of security people, given the effort you are putting into this ;P.)


I wrote this level on my Ubuntu laptop. I was inspired by the idea of TCP idle scans (http://nmap.org/book/idlescan.html), and spent a lot of time trying to find other ways to do connection counting before I realized that this worked. One thing I really like about it is that in contrast to other levels, this one isn't so much a bug in the code, but instead the interaction between your code and something three abstraction layers down the stack. But it turns out that a lot of real-world security is that. Ultimately, we ended up with a pretty good distribution of solvers (the dropoff between each level is basically linear), which I think is a good sign for our difficulty curve.


(After a few people solved it, there was a community of people on #level8 helping others solve it: that's how I finally got helped to the answer by a very kind and patient user I finally found; the result being that the dropped will correlate more with continued interest than difficulty. So, you can't look at the final distribution: you have to look at the time taken before each level was first solved, which allows you to slightly control for interest by staring at the few most-excited people; and level8 stood for 14 hours while all the other levels were beaten by some of the faster hackers, like vos, in a couple hours.)


An important aspect to #8 was the 250ms lock, that allowed you to send multiple requests while others were locked out. First I was trying to send the requests from my local machine, which made it nearly impossible to get usable numbers because of the delay in the network.

Before I started using level2 machine to both send and receive, I was planning to bind on every port of the level2 server to stop/slow others thus raising my own chances of hit. Luckily it didn't come to that.

Even before the port number attack, I was trying to find a pattern in the request object pointer values in subsequent backtraces. This gave me the port idea.

Also spent a few hours trying to exploit the Twisted HTTP client by malformed HTTP redirects (taking the over- specified Twisted version in the README as a clue).


Yeah, the attack barely work without the lock, as you would have to rely on pure luck on getting the port delta small enough.

Because of the lock, the first request you make after you get the lock was likely garbage, and then you have 250ms to get as many requests as you can. Performance here was key, since you couldn't easily do simultaneous requests so you ere limited by the number of round trips you could do in that time.

Some people managed to do simultaneous requests, but you had to do more book keeping since the delta on the first responding webhook contained all the information (eg, for a failed first chunk on N requests, you would expect a difference of 2*N). You would also have to synchronize with the lock, which isn't easy.

Due to load on the servers, SSL negotiation was extremely slow, so using a Keep-Alive connection was required for decent performance.

My final Python implementation managed to eliminate about 4 candidate chunks/sec even under heavy load. I managed to finish #90.


I thought about the possibility of somebody hogging all the local ports in the level 2 server but I read somewhere that Stripe was resetting unused ports.


Finished as 30th, when we still had to actually work to get the answers (shortly after that everyone just started posting the answers in the IRC channels). Level 8 was a really nice challenge. After completing the challenge I managed to optimize my level 8 solution a lot, so much that it managed to finish in under 3 minutes by only doing roughly 2400 API calls (2000 for the bruteforcing itself, 400 to compensate for other players).

https://gist.github.com/38c0430b5084f8442858 for those who are interested. There aren't many comments in there though


I finished as #235 - not as early on - but it was still before the weekend rush and I had already spent 2 days figuring out the solution.

Add your level8 solution to https://docs.google.com/spreadsheet/viewform?formkey=dHBYSjJ... - it's a registry of level8 solvers for comparison (Mine are under 'IntruderAlert').


Can you link to the actual spreadsheet? I can't read most of the urls, or correlate comments to programs.


Nice code. I don't get what I was doing wrong. I was pinging two requests in quick succession and taking the port difference between them. I found that worked fine on local and occasionally worked on the live machines but most of the time the port differences were huge. Every now and again the diff would be 2.


This was difficult to accomplish when server usage was high. There were a couple of ways to deal with this.

For the first chunk, a diff of 2 meant the first chunk server returned false, as the change in port #s meant no other network traffic had occurred between the primary server's request and reporting false from the chunk server. The chunk servers were called synchronously, so if the first chunk was correct and the second chunk server returned false, the difference would (ideally) be three. It's easy to see how a high network load would cause a lot of jitter here.

Some in IRC suggested the best way to reduce jitter was to improve the processing speed of the successive curl calls by using a faster language, such as C. My initial script was in Ruby, which felt qualitatively slower than Python, so I was stymied by jitter too. (I started out rejecting 2 in 10, and it got progressively worse.) I simply checked back every one in a while. By Monday night, the server was either empty or Stripe had increased processor resources, so 9/10 of my calls were effective.


Someone else was getting in requests between yours causing the port numbers to increase.

You had to accommodate for it in your script, if you received a number higher than the expected diff then you needed to keep sending the same request till you where sure they hit concurrently.

Basically the time to crack got longer the more people who were trying to crack it, it became an optimisation race in the end about who could get the requests in faster/closer together

I ended up finishing #65 due to not wanting to rewrite my slow python script. It took upwards of an hour to get a single chunk because of the load


I was running my script at about the same time as you then (multithreaded python with url2lib) - around 70 people had finished. I was obviously too impatient.


while diff > 2: tryAgain()


I was a bit bummed to see so much blatant information leakage after a while. :( Thankfully I finished before that happened. That just ruins the fun.


I only made it to level 5, but I see that the later levels would have stopped me in my tracks anyway! I'm curious to see other solutions, especially for level 8. Was brute-forcing with a script really the only way?


Here's my solution for level 8: https://gist.github.com/ceb360baacb794f39686

It's a bit messy, as my original solution didn't work on the live server and I had to rethink it. My second plan was to reduce it to a handful of potential chunks I could brute force. Once I got it running, though, I realized it was eliminating about half of the chunks on each pass, which left me with the correct one in 7-13 iterations.


I took a similar approach. When I found a guess that returned a result that didn't make sense (due to the jitter/business of the server) I kept track of it. At the end, I looped through each of those weird results and hammered them until I got something that seemed stable.

I actually had no problems with chunks 1 and 2. Half way through chunk three the server started acting up so I aborted my script and added the modifications that I mentioned above. I tweaked the server and client so that they had a hard coded value for chunks 1 and 2.


I only got to level 5 too, I would have had no hope of getting to 8 anyway. It was a positively challenging experience nevertheless :)


Yes it was. But you could optimize the brute forcing so much that you needed only ~2400 calls to the API to crack a 12 digit password.


Could you explain how to get the request count so low, please? I fired off 1 request per possible chunk => 4000+


From the code it was clear the password was 12 numeric digits.

Chunks of the password (of equal size, see the code) were stored in multiple chunk servers. From crashing the program or reading the code, it could be determined there were four chunk servers, so each hosted three characters of the password.

Using a port-counting vulnerability (sending multiple identical requests, and rejecting guesses where the difference in response port #s was too low: 2 for first chunk, 3 for second chunk, etc.) it was possible to brute force each chunk server in sequence.

Thus, the problem space was 4*10^3 rather than 10^12.


https://gist.github.com/38c0430b5084f8442858

You only need 1 call per possible number for each chunk, yes. But since the numbers are random between 0 to 999, that averages to 500 requests per chunk. Additionally, you don't have to do any kind of port checking for the last chunk, so that saves on any overhead you might have for the port checks. Based on this I'd say there's (3x 500 requests + 20% overhead) + 500 requests for the final block. That's 2300 requests on average.


I thought you needed 2 per possible chunk? One for the base port and another to get the port difference? Though maybe that's where I was going wrong...?


You don't have to get the base port on each try, you can just re-use the previous result. I achieved this by simply storing them on the global scope and only keeping the "last" and "before last" port numbers.

https://gist.github.com/38c0430b5084f8442858 for my entire implementation.


Ah, of course - I did read through your code (obviously not carefully enough). How do you get down to 2400 requests though?

Edit: saw the answers above - it's 2400 average case, not worst case.


It's not really brute force, but an educated analysis of the results coming back. I think my script was pretty elegant.


For those like me who enjoyed level8 so much and want to see how their strategy compares to others', go to http://level8-scores.danopia.net/

These two awesome guys (early capturer) rewrote level8 server, set up a level2-like server, and created an IRC bot that lets you run your code (after requesting SSH access to the level2-bonus server) against other in a speedrun on a random flag (same flag for every participant to a round).

Best capture time and lowest requests count wins. It's quite a lot of fun, check it out.


Very similar to the methods I used :)

On level 5 I used a textfile on the compromised level 2 server instead of the 'cleaner' method shown here.

On level 6 I used some more JS:

}];</script><script type=text/html id=payload>$.get(/user-hfbnljhhim/user_info).done(function(data) { var pwd = escape($(data).find(table tr td:last).text()); $(#title).val(pwd); $(#content).val(pwd); $(form).unbind(submit); $(form).trigger(submit); } )</script><script type=text/javascript>$(function() {eval(String.fromCharCode(118,97,114,32,112,97,121,108,111,97,100,32,61,32,39,35,112,97,121,108,111,97,100,39)); eval($(payload).text().replace(/[*]/g, String.fromCharCode(39))); var post_data = [{}];});</script><script> var t = [{

It's funny to see how similar the python script is in level 8 with what I wrote, would be cool to see more writeups on this one with different solutions :)


Here's my level 6 https://gist.github.com/3526645

At first I was trying to use jQuery selectors to get only the table cell with the password in it but in the end I found it much easier to just post the entire page and worry about it later.

Of course there are endless ways to do it! I had much fun with the CTF this time around since I actually knew what I was doing.


Arrg! So close https://stripe-ctf.com/progress/fduran if only I had started earlier.

For the last level I solved it locally but in the server I didn't get the '3' port difference for the first password chunk (at lest in the first rounds) so I made it more complicated by getting as candidates the small port differences and running those again and culling again, got the 1st chunk of my flag before running out of time... oh well.

Stripe people made an excellent job, I really enjoyed the challenge. They were blogging than the set up took only a couple of man-weeks, given the success of this CTF I think there may be a start-up idea here: to use similar challenges for programming and/or security training/job interviews.

I'll take someone who solved by himself/herself this ctf over a CISSP in most situations any time (I'm a CISSP myself).


I got to level 6 before I quit. I'm certain I could complete it but I could not justify spending any more time on it.

In contrast to their previous ctf, this time the contest itself seemed less technically robust. The site didn't work in a browser without javascript support (elinks). On level 6, the "social network" app did not work with the (quite old) version of Chrome I have; Opera could post a single message after loading the page, but after that it would silently fail to post any further messages.

I still have the shirt from the previous ctf. Dealing with disassembly was a lot more enjoyable than trying to learn jquery.


This was the first time I seriously tried to do a CtF and it was surprisingly fun. I spend almost the entire weekend working on it and was surprised by my eventual success despite needing some help. I'm pretty sure I would have given up on level 7 if I hadn't seen the paper on the Flickr API. This insight will actually make my code better.

Thanks to the Stripe Team for the challenge and the T-shirt :)


I got through level 6 on my own. Level 7 requires an understanding of crypto, which to me isn't a web vulnerability per se. I eventually broke it with the hint that the vulnerability was a hash length extension attack, but I never would have thought of that on my own. Level 8 seemed really involved, and I threw in the towel. :)


This was a lot of fun while it lasted. We should do it more often. Are you guys up for a community driven CTF on a regular basis? It could be a lot of fun. We could create a pot where everybody can contribute towards the prize.


[deleted]


Before scrolling down in the first paragraph of the article it already says: "wanted to share with you a detailed write-up of the levels, why they're vulnerable, and how to exploit them.". I don't think it's necessary; if you don't want to read it because you're still playing you just don't continue reading the article.

Edit: And the CTF has ended, so no harm done.


The CTF has ended.


My bad, indeed, the bookmark is still viciously looking at me every day, so I was under the impression I'd still might finish it this weekend.

So spoiler alert suggestion revoked...


The last task was really obscure, must have taken a while to figure that one out




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: