Solid writeup. From someone who does/did a lot of this professionally:
1. Android typically is easier for this kind of work (you don't even need a rooted/jailbroken device, and it's all Java/smali),
2. That said, instead of installing an entire framework like Xposed that hooks the process to bypass certificate pinning, you can usually just decompile the APK and nop out all the function calls in the smali related to checking if the certificate is correct, then recompile/resign it for your device (again, easier on Android than iOS),
3. Request signing is increasingly implemented on APIs with any sort of business value, but you can almost always bypass it within an hour by searching through the application for functions related to things like "HMAC", figuring out exactly which request inputs are put into the algorithm in which order, and seeing where/how the secret key is stored (or loaded, as it were),
4. There is no true way to protect an API on a mobile app. You can only make it more or less difficult to secure. The best you can do is a frequently rotated secret key stored in shared libraries with weird parameters attached to the signing algorithm. To make up for this savvy companies typically reduce the cover time required (i.e. change the secret key very frequently by updating the app weekly or biweekly) or by using using a secret key with several parts generated from components in .so files, which are significantly more tedious to reverse.
I am also doing a lot of reverse enginering work for fun and profit. I agree with most parts, except android.
While reversing Android apps, usually they are obfuscated ( I think it is included in the standard build process ), but when you are start to be familier with objective C (or swift) binaries there are really straight forward, with a nice disassembler and script, in my experience much faster.
Request signing, certificate pinning etc are not actually target at reverse engineers, more like to snoopers, replay attackers etc.
Also I agree with 4 totally, there is no point there, you should do your security on server side, instead of client side. (Except games I guess, which they can ban players etc, making it a cat and mouse game there can work pretty good actually)
Much harder targets, like obfuscated c++, or VMs (bytecode interpreters etc) usually helps with #4, but still then, it just slows down. (This slow down combined with regular changes in protection is basically targeted to make reverse engineers to give up)
On iOS you have to go through the extra step of getting the binary onto a jail broken phone to strip the drm in order to re-sign it. Once you've got a way tamper with the binary and run it, most platforms are pretty straightforward to work with - it's really just down to which byte code or assembler variants you're most comfortable working with. Obfuscated code isn't a barrier. Obfuscated control flow is more irritating, but I don't think the standard Android tools offer that?
yeah but this step is usually done for you, you can always google ‘appname’ ipa and reach latest version’s IPA.
Obfuscated control flow is like hell especially in c++ (ex: pokemon etc) but as I said I am comparing average android vs IOS. Once you have rooted platform you have much wider tool set of course.
> That said, instead of installing an entire framework like Xposed that hooks the process to bypass certificate pinning, you can usually just decompile the APK
I remember reading something years back about Java decompiling, and I believe it said that all Java code is decompileable except for inner classes and nested try-catch. Assuming my memory and the source are correct (which might not be the case), why hasn't it become standard practice for developers who don't want their app reverse-engineered to just put every class inside a wrapper class? I can imagine there could even be tools for doing this at compile time so that you wouldn't need to manually deal with the indirection when writing the code.
The standard way of frustrating decompilers for Android applications is with heavy obfuscation using ProGuard [1] or DexGuard. [2] IMO, DexGuard is a real pain in the ass to reverse around. If you aren't dealing with heavy obfuscation, decompiling APKs is trivial using jadx. [3]
In general I have found that most smaller app developers don't obfuscate at all, and often you can find hardcoded keys/secrets in these smaller applications.
Modern Java decompilers (fernflower, cfr, procyon) are very flexible in what they can handle. They're best used in combination, though, as they all tend to choke on different things.
One can easily reverse inner classes. They have a unique naming pattern that tells you exactly where they live: OuterClass$InnerClass. Anonymous inner classes just get numbers instead of names after the $, so you can even tell normal inner classes from anonymous ones and decompile them appropriately.
As for nested try-catch... I don't see how that would be an issue unless the compiler somehow merges them into a single block. Which, as long as it's semantically equivalent, doesn't matter at all.
I mean, Java compiles down to a byte-code, which can be decompiled. If that were a limitation of decompilers at a time, it's likely not now. There's no way to "hide" instructions.
You can hide them in the sense that it's very difficult to find the 100 instructions and correct state machine out of the millions of instructions and possible states.
I tried doing 2 recently, found the file with CertificatePinning in the name, found the part returning 1 and 0 based on whether the check passed and patched it to return 1 in both cases. It still didn't work and after a few hours, I gave up, grepped through the decompiled files for the website name, got an endpoint which turned out to work perfectly in a web browser and which led me to the ultimate endpoint I needed.
I could probably have gone deeper and found more functions to patch, but I don't know smali so everything was guessing and matching up to the java decompiled source (and the original source which seems to have been Moxie's?)
(>$100 billion company making the app with >1 million users.)
From my experience, for certificate pinning, fastest option is to search for existing certificate/fingerprint, and try to replace it with charles’/mitmproxie’s
Edit: oh another trick usually works is to change transport to HTTP from HTTPS, just changing endpoint to something you control and changing it to http (with little hex editing the endpoint), and reverse proxy with mitmproxy/charles to the target, you can speed up process.
I don't know certificate formats and I imagine it would require conversion between hex and something - maybe you can write a blog post with an example?
ooh I like the second idea - so since it's an HTTP the cert pinning code won't be triggered at all, and all you need is to grep https to find endpoints and change?
I do this occasionally, and recently found our local real estate app that typically hides sales prices in the form of a range actually returns the actual value through their mobile API. For #2, you don't even need to recompile it, by the time you decompile the APK, you can usually just read the endpoints and parameters/data and reimplement in your language of choice.
It is the same with app keys, you have to gate it behind an intermediary/signing server and implement any impersonation/abuse detection there.
https://casper.io/ is an example of doing this (for SnapChat) - they used to take registrations for their own API. Not sure how it's working out for them these days.
Any chances you would be willing to share some of the online resources you use for this? I have recently been getting into mobile security development professionally.
I don't use online resources. I mostly used a combination of dex2jar and apktool. Some custom coding to automate the process. When I'm doing this on iOS I use Hopper.
Could you pontificate a bit on four? Because of the identity problem, or is there something about a mobile app that is fundamentally less secure than say, a web browser? I'm genuinely ignorant, seems like it would be good to know.
The "protect" which 4 is referring to is fundamentally like any other DRM: you're trying to give someone access to the content, but also deny it at the same time. In the case of an API, you've given someone an app which knows how to use it, which they can execute on their own computer and control the inputs of, and inspect the output.
If you don't feel like RE'ing the API, you can always just supply the inputs to the app yourself from somewhere else. ("All problems in computer science can be solved by another level of indirection", as the saying goes.)
Actually it is opposite, mobile platform is more secure.
But...
Security depends on ‘sense of security’, when people think platform is more secure, they tend to ignore/skip a lot of parts on security. Developer tend to skip edge cases (such as pizza API in this thread). When they are developing on secure platforms, they tend to skip more.
For example if you are developing for not jailbroken platform, you trust platform DRM (mostly consoles), and skip a lot of parts, you put the certificate pinning, and call it a day. When platform is broken, you are totally exposed.
But when you are developing for web, you are exposed from the beginning, you dont have that sense of security anymore, so you try to fix all edge cases.
Sure. The only way to sign requests is through something both parties can verify. The client you're using must have access to the shared secret key used in (e.g.) the HMAC process. While you can obfuscate the secret key to extents that would make a reverse engineer's life miserable (for a case study in that, see the Facebook app), you fundamentally cannot prevent the request signing process from being reversed with enough effort.
It's a very simple principle: the relevant data must necessarily be exposed, even if only in memory, at some point. Like any other DRM, it's imperfect.
Yeah, no worries I understand what you mean. I reversed the Starbucks app myself a few months ago to see if I could find interesting data. There's less low hanging fruit to use for reversing large companies' mobile apps these days.
I did this with the Papa John's webapp a while back (which was waaaay simpler btw). They limited duplicate toppings to (I think) 3 of the same, but "duplicate_item" was just a numerical property on the (e.g.) "bacon" object. Turns out you could just add multiple "bacon" members to the toppings array to exceed the limit, and they didn't charge for duplicates, so I ordered a pizza with like 50 bacons.
It definitely didn't have 50x worth of bacon, but it did have more than 3x, maybe 5x-6x. The receipt was hilariously long though.
Whenever you do this sort of thing, you should check whether they do negative number checking on the client side or in the API itself. You might be able to get 1 pizza with bacon and (1) pizza without, all for the price of the bacon.
I work for a company with a similar sort of online ordering API, and a while back some enterprising person discovered that:
1. Our API had a hidden "tip" feature; we hadn't exposed it in the storefront yet, mostly because the developer in charge of it had left with it halfway finished, and nobody had picked it back up yet
2. But the API itself was perfectly functional. If you manually submitted an order to the API with a tip set, it would be reflected in the total, would bill your credit card, etc.
3. Except (I'm sure you saw where this is going...) the now-departed dev hadn't added any validation yet. So if you ordered $40 worth of pizza and added a negative $39 tip...boom, nearly free pizza.
We learned a few valuable lessons for the cost of buying a college student a couple of pizzas. :)
We actually debated offering them a job interview when they graduated. If they'd contacted us first, we might have. :) Instead we just fixed the bug, called it a day, and spent a few days reviewing the rest of the API for weak validation.
Good advice; I'm not sure I tried that. The goal wasn't to steal toppings or get free pizza though; it was an impromptu response to my desire for lots of bacon and encountering the duplicate limit, and then a busy signal on the phone. Once I figured out it didn't charge me for more, I just wanted to see what would happen. I figured they'd think it was a system glitch and call when they received the order.
The employee will ignore the negative sign in front of the number and make you that pizza anyway. The calculation of the total is where it might give you free pizza.
Yeah that's what I figured. The wording of the post made it sound like it had something to do with the bacon though. You could just do this for any arbitrary pizza.
The bacon is there so you pay something, a free pizza would be weird even to a minimal wage employee (not that they're dumb, but they probably don't care).
Though that's assuming the people making online orders even bother looking at the amount paid. I don't necessarily see why they would (or why they wouldn't assume you had merely cashed in some sort of promo or balance).
I wonder how did the people on the other end react.
One evening I ordered a pizza on-line to the office. I was too lazy to go out to the nearest grocery store (even if I knew which will be open at those hours) to buy a ketchup bottle, so in the form I added ~16x sachets of ketchup (figured it'll last me a while). I don't think it was more than 5 minutes after submitting the form that my phone rang, with the pizza guy on the other end asking if I really, really want 16 bags of ketchup?
The dominos ordering app in India had a terrible flow a while back. Once the products are added to the cart and proceeded to checkout , the flow was as follows
1. First a payment collection flow is initiated from the browser (asking Credit Card details, pin etc)
2. The payment confirmation comes to the browser
3. The browser then places the order(the pizza information) to another api end point, marking the payment part as 'Paid'
The thing is, one could add as many pizzas to the cart in a different tab, while the original tab proceeds to payment. The end result is, you get to pay only for pizzas that were initially in the cart, but could get any number of pizzas. For literally Rs100 one could order thousands of rupees worth pizza.
I discovered it accidentally and did report to them. Neither did they acknowledge nor did they send me a free pizza :(
They later fixed this, by not allowing to load the cart in a different tab. But there is a high chance that there could be another hack even now. Since I had wowed not to eat junk food anymore, there was not much incentive for me to spend any more time on it.
Surely you can do just point 3 if you know how the API works. Atrocious design.
There used to be some "secure pendrive" which worked similar way: an app asks you for password, checks if it matches hash stored on the drive and optionally commands the drive to unlock itself. What could possibly go wrong? ;)
I have to take issue with the "Starbucks app is great" line, though. I think I've had more problems with it (on iOS) than any other app. It's the only app that (for a period of many months if not a year) was regularly unable to find my location. Even if I opened up Maps or some other location enabled app and found my location before launching Starbucks, it would just bomb out.
Overall the app seems to have tons of 'issues'. It's been better the past few months though. And it beats the hell out of standing in a 10 minute line. I honestly wouldn't stop at Starbucks anymore if it wasn't for the app.
Agreed. I'd also add that the UI for the app needs work. Ordering coffee for a group of people becomes a huge hassle on my phone because their search is not precise, and customizing drinks is a bit cumbersome.
I spent like 20 minutes trying to order 5 drinks the other day. It was also annoying because when I left the app, all my selected items disappeared.
I would not. Ordering a large group of people drinks from Starbucks is not in any way the primary use case for the application. For it's primary use cases (paying at the store, rewards redemption/collection, ordering yourself drinks ahead) it works perfectly.
Additionally, if you think about the feature set that would be required to facilitate bulk orders then you will quickly find that it would be quite large feature set and require it's own flow. The app does has new-ish features that allow you to re-order prior orders and save drinks, which would help for these bulk orders, so they are making progress. If you are consistently ordering drinks for a large group of your co-workers then you can save their drink preferences and boom.
Of course it's not the primary use case and the app guarantees it never will be. I can say from my experience that sending a gopher to get everyone coffee was a twice daily occurrence at our family business in NYC. Shave 5 minutes off that time and we would go to Starbucks instead of the deli.
Select the sandwich, tap on Sandwich Options, tap the google for No Cheese. You've been able to customize orders for as long as I've been using it to order stuff.
I have also had issues, except on Android. Admittedly, I am using a rather old smartphone now, a Note II, which my carrier has only allowed to update to Android 4.4.2.
Whenever I try to order something through the starbucks app, I have to try multiple times because at some point in the ordering process the message, "unfortunately starbucks has stopped," or something like that will pop up and the app will close. Usually no progress is saved. Often it will crash right as it is processing the payment, and when I open it back up I have to go to the order history to check to see if it went through. Usually if I make it that far before it crashes then my order will have been submitted successfully.
I would love to be able to skip the UI and just submit my order an alternate way, but maybe the answer is to just stop being so cheap and to get a new phone.
Here's a little story about how my gold card was disabled;
I was ordering coffee and my balance wasn't quite sufficient. I reloaded it via PayPal, but the balance didn't update so I tried again (it did error out). The app crashed, I reopened it and found it had added both transactions to my account. Open dispute on PayPal for just one of the transactions, got a refund but my card was completely locked out - $0 balance, couldn't reload it (the dispute was after I tried contacting support). So now I don't go to Starbucks.
Agreed. The app has definitely sent a different order than was on screen before; until recently it refused to work with Apple Pay unless your Apple Pay address matched character-for-character what your bank had on file (if one used # and one used APT, or one used VA and one used Virginia, it would fail), and we've lost the ability to customize orders as much as we used to (e.g. you can no longer order a flat white with chai unless it's saved as a previous order).
If an open API existed yes there would be more integrations. Of course you would have to hire engineers to perform upkeep. Eventually if the ordering API isn't profitable you get a bunch of sunk costs and have to reassign people. Its not just "make this open" and POOF. Also your access could be revoked by unofficially using the API and or they could just change it at any time.
Why would their API need to be anything more than what they offer to their app? Why not something like "hey, you can use our stuff, but you've only got access to the same set of APIs we use in our app, and we may change it as we please.
Oh btw, here are the auto-gen'd docs, hope those are enough"
That seems like it'd be both developer friendly and no more maintenance than they're already putting into it. The alternative is basically things like this, where someone figures it out anyway, and we have to wait and see if they play it cool, or take it badly and make the author take it down.
All you have to do is to say: "this is our internal API, we're releasing endpoint documentation for curious developers, but be warned, that the API may, and will, change without any notice".
I.e. when you're releasing something in the open, you don't have to take social responsibilities if you don't want to. It's not a binary choice between "release a fully supported product" and "don't release at all".
Why not? The real cost would be documenting it, but a lot of devs would be willing to accept essentially being on a mailing list for breaking changes a few weeks in advance. It's not like Starbucks can roll out breaking changes to their own API instantly.
It would only really be used by enthusiasts and techies who are used to this sort of thing.
You can't data mine users as much.
You can't easily advertise specials or high margin items.
Not that i think those are legitimate, but if a 3rd party app became popular and easily accessible, the company would lose the control over their perception.
But then your users start using the Starbucks ordering Slack channel, cool! But their credit card is hacked. Or the slack channel breaks. Or a stalking ex-boyfriend intercepts the order and delivers the latte himself. Starbucks would likely be blamed for these problems.
Remember the Snapchat leak from a few years ago?[1] I'd bet almost everyone who heard about the leak thought that Snapchat got hacked, and many then thought that their personal photos are not safe on Snapchat. But the reality is different. Users logged into a site called "Snapsave", many probably thinking it was part of Snapchat. It was Snapsave that used the private Snapchat API and got hacked... not Snapchat.
It also expands the total attack surface of a system, which can lead to security problems. If you read some of the public disclosures from various bug bounty programs, neglected APIs have led to some serious vulnerabilities.
"Underprotected APIs" is actually number 10 on the OWASP Top 10 for 2017.
The attack surface is the same whether the API is officially open/documented or not --- if you think that data received through a web API is somehow more trustworthy/less demanding of validation just because you haven't documented it and only release an app which uses it, you are doing it wrong.
Conceptually, it's just a listening server on the public Internet, and will be subject to arbitrary data anyone willing to connect to it can send.
I've always wondered the legality of doing things like this. Have there been cases where someone was taken to court (in the US) for reversing an API from a mobile app or website? Assuming no malicious action or intent of course. Could the CFAA be used in this case, even if the intent was just to understand the API for personal use?
With so many IoT devices out there relying on 3rd party web services that may or may not be around a year from now, I expect that the right to inspect and understand these APIs will become more and more important. Not to mention wanting the ability to build interactions between devices where the manufacturer may not have interest (IFTTT, etc).
Maybe a bit off-topic, but APIs always make me wonder a bit when they can be reverse-engineered or.. for lack of a better word, misused.
I know of one website (site A) that sells items for sports and uses an API of a sports website (site B) to provide current statistics and other information.
Thing is, that sports website's API is now deprecated for public use, there's no way to request a token, and from what I can tell it's only to be used for commercial purposes/paid for by companies.
But, I can easily find the API token being used on site A, dig through the private/deprecated docs for the API of site B, and use any of their endpoints and data for pretty much whatever I want.
At least, this was the case roughly 4-6 months ago and I haven't looked into it since; perhaps they've changed it since then.
But I wonder how this works. Wouldn't it be a misuse of their API and something they wouldn't want allowed? Usually sports statistics APIs are fairly expensive, and the fact that some random person like me can get access easily for free seems unfair to site B, especially when they don't want normal people using their API anymore.
I was looking at statistics for prices of sold houses in a European country a few years back, and started digging around in the javascript of a local newspaper that every month listed the local sold properties. Turned out in the API they were using from the central statistics provider, they just had "days=30" and "municipalities=[list]". They weren't even caching it, every user visiting the page was another (identical) request to the API provider. And sure enough, sending a request with "days=3650","municipalities=[{all of them}] gave me 10 years of data with every property sale in the country.
I wonder why they went such great lengths to prevent unauthorized clients (which also is a thing that’s fundamentally impossible. All you can do is making it harder for attackers). What would be so bad about third party apps ordering coffee?
Generally, it’s a good idea to be protective, but between cert pinning and that encrypted device fingerprint and the time based signature, this adds a lot of points of possible failure to an app you want to be as available as possible to as many people as possible.
What information this API has access to is so precious to warrant all of this?
I'm thinking they probably don't want unauthorized clients just to lower the support threshold. Some percentage of people will hold them responsible for broken third-party clients.
Opening up an API like this is ripe for abuse, so taking care makes sense. Bad actors translate directly to lost money.
A real method of securing APIs would be a godsend, but in current tech it's just not possible. This is the one place where mediocre security-by-obscurity is your only choice =(
lol. Reminds me of high school. Constructing a green roof garden. We have the genius (see stupid) idea to go order 20 large waters from McDonalds. They just tell us to drive forward and park. Sure enough, not long later, they bring us 20 large waters!
An interaction that cost McDonald's under $2 and is something you won't forget and will actually tell others about. Sounds like they got their money's worth.
> A real method of securing APIs would be a godsend, but in current tech it's just not possible. This is the one place where mediocre security-by-obscurity is your only choice
It's magically possible when tying it to a payment processor, though.
Say a third party app orders the wrong coffee, or orders too much stuff, etc. There's a real business risk because they have to shell out physical capital to make coffee.
I might be slightly off because thinking about it the app probably requires up-front payment. But either way, fully open APIs can cause problems if they're more meant for internal processes
One funny (ab)use case that may not be immediately obvious: any public API that accepts payment for goods via credit card will sooner or later attempt to be used by carders to test purchased/stolen card lists for validity.
You can get pretty far with a secure element. Server challenges with signed payload of nonce + current timestamp, and client verifies it and then re-signs the payload. For something like a third-party coffee-ordering app, the economics of the usual attacks don't scale (either physically tearing apart the chip or exploiting a vulnerability in the first-party app).
I want to write a similar write-up for a company which basically does everything over HTTP with their own half-baked hardcoded AES key in app for sending credit card info. and that their confirmation checkup is stupid (for SMS) and can be bypassed.
The problem is that their site TOS forbids reverse engineering, and I am afraid their lawyers will go after me instead of fixing the security issues (even if I just contact them), any tips for me ?
Besides telling me to use tor (which I know in general what it is), is there a guide I can use ? (i.e. how to send / recv anonymous email on tor from a non tor address ?) or something ???
Once you're in there you can just use the Tor browser to create an email account at any email provider (make sure you give them fake details though) and send your email.
Just formally ask the company, if they don't want you to they will tell you. Could be worth including a pitch of sorts ('Hey, Ive noticed a few problems using your site...') but MAKE SURE you don't incriminate yourself. Do NOT show evidence of you 'reverse engineering' anything.
Do not do anything without gaining permission, as there's a very good chance you're going to run into their legal team.
I don't want to contact them in a way that they can trace me, since I'm pretty sure that they will try to covert up / threat me (I would hope that someone there already knows this is an issue....).
> HTTP with their own half-baked hardcoded AES key in app for sending credit card info.
You mean, no TLS, JavaScript crypto and credit card info?
Are there no authorities to report that kind of garbage? No certification they are supposed to pass? I would research that angle first before hacking. It doesn't take any reversing to realize everyone can pwn you on a public WiFi.
I like getting out of the house in the morning. Starbucks is a ~4 minute walk away. I get to chit chat with folks, take the dogs out with me and get a feel for the weather that day. If I made my coffee at home, I would be on my laptop and totally forget about making it.
The way to solve this is just for companies to give out their API in a public manner. You'll almost never be able to secure it from scrapers.
We experienced this at Honeywell, when I first started here we were blocking users that scraped the app instead of giving them public access and teach them how to use it correctly.
This is super interesting, especially the part about trying to reverse engineer their "security" measure. I did something like this a few months back for Paylocity, a time logging system that has a terrible web interface. After trying to talk with their sales people about them potentially offering an API, I was told "no API, just this mobile app." Turns out the mobile app is just an Ionic app with all of the resources (all, including tests and even test environment logins and stuff) baked in. Super easy to grab their API out of that (literally called /mobileapi/), but then the trouble was figuring out how they generated their security token header, which was also a little dance of timestamps and faked device info.
The best part was when I contacted them afterwards and warned them about the extra pieces of info they had baked in, their response was basically "yeah, we're aware that there's more in there than there should be, but it's not a priority." Oh well, they just have all of my personal information.
I'd argue that time spent actually securing it against abuse would yield more actual value than time spent obfuscating, but this is almost a religious argument that has been carrying on since before I was born.
CharlesProxy is a must-have whether you're a dev or reverse-engineering.
JADX is the best for reversing Android because it produces readable Android code which is fantastic for current Android developers who expect things to be organized like Android Studio.
1. Android typically is easier for this kind of work (you don't even need a rooted/jailbroken device, and it's all Java/smali),
2. That said, instead of installing an entire framework like Xposed that hooks the process to bypass certificate pinning, you can usually just decompile the APK and nop out all the function calls in the smali related to checking if the certificate is correct, then recompile/resign it for your device (again, easier on Android than iOS),
3. Request signing is increasingly implemented on APIs with any sort of business value, but you can almost always bypass it within an hour by searching through the application for functions related to things like "HMAC", figuring out exactly which request inputs are put into the algorithm in which order, and seeing where/how the secret key is stored (or loaded, as it were),
4. There is no true way to protect an API on a mobile app. You can only make it more or less difficult to secure. The best you can do is a frequently rotated secret key stored in shared libraries with weird parameters attached to the signing algorithm. To make up for this savvy companies typically reduce the cover time required (i.e. change the secret key very frequently by updating the app weekly or biweekly) or by using using a secret key with several parts generated from components in .so files, which are significantly more tedious to reverse.