
Reverse-engineering the Kayak app with mitmproxy - shbhrsaha
http://www.shubhro.com/2014/12/18/reverse-engineering-kayak-mitmproxy/
======
101914
Note that mitmproxy, which requires a Python install, is not necessary to
monitor what is being sent out from your computing device.

The same results can be achieved using only socat and the openssl binary.

While I understand the terminology is popular, I would not call this "reverse-
engineering"; to me this is simply viewing your own traffic.

I believe users have a right to see the traffic they (or the apps they use)
are sending, and for security reasons alone they should monitor what is being
sent. https plus third party CA usage complicates such transparency, making
proxying techniques necessary.

I wish more users would view their own traffic.

Keep up the good work.

~~~
userbinator
I've been MITM'ing my connections ever since I discovered the Proxomitron (
[http://en.wikipedia.org/wiki/Proxomitron](http://en.wikipedia.org/wiki/Proxomitron)
) - it lets me customise webpages and filter out the crap _before_ it gets to
any browser, and works in all the browsers on the system; even the ones
embedded in apps. It has a logging feature to show traffic too.

It would be great if a proxy could be run directly on the mobile device, as
then it could be used anywhere along with the apps its monitoring.

As you said HTTPS is a bit of a pain since it's actively designed to resist
such "attacks", but as long as there's the option to specify your own CAs it
will work. Certificate "unpinning" is still a manual process, however...

~~~
dmix
How would you use a proxy on a mobile phone? Just curious. I'm not familiar
with Proxomitron, so I assume adding filters like privoxy or captures?

People have done this in ways by hacking the built-in VPN system I believe.

~~~
guiambros
You just point to your Proxy server under Settings / WiFi. Alternatively you
may want to install your proxy SSL CA on your mobile device, so you can see
inside HTTPS connections (if supported).

Charles Proxy has some easy-to-use pointers [1][2].

[1] [http://www.charlesproxy.com/documentation/faqs/ssl-
connectio...](http://www.charlesproxy.com/documentation/faqs/ssl-connections-
from-within-iphone-applications/)

[2] [http://www.charlesproxy.com/documentation/faqs/using-
charles...](http://www.charlesproxy.com/documentation/faqs/using-charles-from-
an-iphone/)

------
chrisan
Another great mitm proxy is Charles Proxy
[http://www.charlesproxy.com/](http://www.charlesproxy.com/).

It has a really nice UI for looking at JSON responses such as the Kayak.
Sometimes a collapsible tree is invaluable in looking through a response.

The easy filtering and formatting is primarily why I like it so much. Here is
how it handles SSL for various ways
[http://www.charlesproxy.com/documentation/using-
charles/ssl-...](http://www.charlesproxy.com/documentation/using-charles/ssl-
certificates/)

Here is a screen shot of my iPhone Kayak app request for comparison
[http://imgur.com/gvKB6fr](http://imgur.com/gvKB6fr)

~~~
nnd
How is it better than mitmproxy? Also, it's not free/open-source.

~~~
brad0
In this case you get what you pay for. You get filtering. Local and remote
mapping. SSL decryption. Request and response detail views. Breakpoints. The
list goes on. Seriously, it's the best $50 I spent as part of my developer
toolkit.

~~~
scr4ve
Charles is definitely good, but to be fair, you get all of that (except
mappings) in mitmproxy as well. :-)

------
ketralnis
I don't know about Kayak's economics, but at least at Hipmunk we pay our data
providers per search and it's really quite expensive. If they aren't offering
an API anymore, it's probably because it was too pricey to operate.

You could easily cost a travel search company thousands of dollars very very
quickly using an API they don't want you to use. I don't know if it's illegal
or not, but it's certainly immoral.

~~~
edward
I'm happy to pay to use a flight search API. Kayak and Hipmunk could provide
an API and pass on the cost to the API consumers.

~~~
kremi
The problem with any paid API is that the API consumer needs to make money
somehow so that he can offer the price of the service.

How will the API consumer make money? I don't think Ads will generate enough
revenue -- especially not on mobile.

Kayak will get some revenue from the bookings resulting from the searches, and
would need to circle back some of this revenue back to the API consumer. This
is technically complicated because it requires a lot of tracking
infrastructure.

In the end, a service that would make sense is a lot more complicated than
simply having an endpoint and saying it's X dollars per Y calls.

~~~
edward
I don't need to make money. Access to a flight API would be useful for
planning my own trips.

------
dthakur
> From that folder, get the mitmproxy-ca-cert.pem file onto your mobile device
> by emailing it to yourself, for example. Then follow certificate
> installation steps for iOS or Android.

You can just go to [http://mitm.it](http://mitm.it) on the device. It's a
'magic domain' for the proxied host. See
[http://mitm.it/doc/certinstall/webapp.html](http://mitm.it/doc/certinstall/webapp.html)

~~~
nnd
I've tried this website on iPhone 6, and it redirects to the standard version
of the website instead.

------
vertak
Does anyone know the possible legal repercussions open-sourcing a web
service's API when the company doesn't explicitly grant permission? This is
really neat, but could also raise the ire of a service that doesn't offer an
API for a reason.

~~~
declan
There are two obvious issues, as I see it. #1 is whether someone's liable for
poking around inside the Kayak app in the way the author of the post did. #2
is whether it's legal to "open-source" the API by disclosing this information,
once obtained.

My quick take is that #1 likely violates Kayak's terms of use for the app, its
web site, or both. The TOU for the website has a host of prohibitions,
including don't "bypass or circumvent other measures employed to prevent or
limit access to Our Website," no deep-linking (such as a link to the API
endpoint), and as a catchall also prohibits all inappropriate-as-defined-by-
Kayak activity. I haven't read the app TOU, but it would be standard practice
to prohibit reverse engineering of the app itself.

So we don't really need to reach point #2, which raises some 1A issues the
first doesn't. (Note I'm excluding DMCA and CFAA issues because Kayak isn't
our sue-happy friends at the MPAA or RIAA.)

As a practical matter, though, the author of the linked post, Shubhro Saha,
appears to be an undergrad, so is probably judgment-proof and not the most
likely target of litigation.

~~~
moyix
Just to add to this: the relevant case law seems to be Bowers v. Baystate
Technologies [1], which held that EULA clauses prohibiting reverse engineering
are enforceable. I personally think this is a terrible decision, since it also
prevents a lot of good uses of reverse engineering (see some examples, like
fighting censorship and diagnosing software vulnerabilities, in a recent paper
of mine [2]).

[1]
[http://en.wikipedia.org/wiki/Bowers_v._Baystate_Technologies](http://en.wikipedia.org/wiki/Bowers_v._Baystate_Technologies)
[2]
[https://mice.cs.columbia.edu/getTechreport.php?techreportID=...](https://mice.cs.columbia.edu/getTechreport.php?techreportID=1588&disposition=inline&format=pdf)

------
xrjn
Great writeup, there's definitely potential to do something cool, especially
if it's possible to get around any tracking and the following price
manipulation. I tried installing your demo client, however on running it I got
the following error:

    
    
      root@kayak:~/kayak-mobile-client# python client.py 
      Departure airport code: LBG
      Destination airport code: HAM
      Departure date (MM/DD/YY): 12/26/14
      Traceback (most recent call last):
        File "client.py", line 56, in <module>
          searchid = json.loads(r.text)["searchid"]
      KeyError: 'searchid'

~~~
shbhrsaha
Appreciate your kind words.

Have you set up your UUID and HASH according to the post? You'll actually need
to run the app through mitmproxy yourself to get your unique credentials.
Then, just drop those into the variables at the top of the script.

I just added some error handling to the script to make this a bit more
obvious!

------
ianlevesque
A useful technique for sure. The only technique I know to slow this down is to
use certificate pinning, but it's probably pointless. Some of your users are
probably extremely motivated (like ours [1]) and it's obvious to them that
what they are doing is unsupported.

1\. [http://difm.eu/dox/](http://difm.eu/dox/)

~~~
nnd
Certificate pinning would help to prevent sniffing traffic with tools like
mitmproxy, but once you decompile the binary, you can disable certificate
validation.

I'm curious, in your case did you consider preventing using your API by third-
party clients?

------
javiercr
Interesting! We've used a similar technique to reverse-engineering mobile apps
from different banks in order to create a Ruby gem to fetch bank data (balance
and transactions).

[https://github.com/ismaGNU/bankScrap](https://github.com/ismaGNU/bankScrap)

------
nnd
I wouldn't call this reverse-engineering.

How are UUID and HASH are generated? Are they unique to every installation?

