Hacker News new | comments | ask | show | jobs | submit login
Reverse Engineering WhatsApp Web (github.com)
350 points by wjh_ 10 months ago | hide | past | web | favorite | 119 comments



I'm very hopeful this reverse engineering effort will enable the creation of a tool to export my conversations (WhatsApp can do email export, which let's be real, doesn't cut it for most cases).

A point to those that support migrating to alternatives such as Signal. Signal is good, but far from great for a single reason: you need a phone number. This is very bad in necsec and reliability terms, my case:

Reliability: like more and more people, I travel all the time between countries and live out of Airbnbs. Hence my pre-paid phone numbers changes very regularly. If I lose my phone, I lose the phone number, I also lose my Whatsapp/Signal key associated with my phone number.

Netsec: A phone number is associated with your physical identity, you might not care, but more and more people do care about this stuff. Yes there are ways around that, but nothing straightforward and actually practical.

I'm patiently, but eagerly, looking forward to status.im .


This may sound like an advert, but it's not - I used "Backuptrans Android WhatsApp to iPhone Transfer for Mac" last year when I moved from Android to iPhone, and surprisingly it actually worked!

I could export all my Android (Nougat) messages and media, and restore it onto my iPhone (iOS 11).

It was a bit dodgy though - it asked me to install an old (custom?) APK first to export my messages, and the iPhone restore process looked like an actual iOS system restore..


I wonder if it's easier to manipulate a backup than to inject data into an app actually on the phone -- make a backup, replace the app's data, then restore the backup. You (presumably) have root on the Mac, so finding any needed encryption keys is probably easier, and adding data to an app on the phone doesn't sound easy unless the app/phone provides an interface for it.


> and the iPhone restore process looked like an actual iOS system restore..

As in it 'hijacked' the UI? I guess its possible that it's using the backup-restore mechanism to get data onto the iPhone.

I wonder if it still works now in iOS 11.


which platform? On iOS “export chat” exports a .zip file which you can send to any app that accepts it, or to a comp using airdrop. Works pretty well for exporting all photos and videos from a chat.


Sounds like you could get a stable VoIP number and just use that. One time cost and has its uses for cheap long distance calling.


Most people expect you to have 1 mobile phone number, not separate ones for calling and messaging.


I don't see the problem. Once you've registered with your messaging service, like WhatsApp, you're sending messages to people not numbers.


Then if someone wants to call you, they won't be sure which number to use.


Why not use wire then?

iMessage works with email addresses too.


I second wire. Love it.


If you want a good alternative try xmpp. It is basically email but for chat. Completely free, open source, distributed. And, you can have as many as you want with no need for a phone number.


Just use telegram. Open source, native clients for every platform. No phone number necessary.


Open source client, as far as i know the server is still closed source.

I suggest using Riot, preferrably self-hosted.


Not even that. They only dump the source code over the fence once in a while. Try to find the source code for the latest release of any one of the Telegram mobile clients. Good luck!


"No phone number necessary" is only true if Google Voice is available in your country. A phone number is necessary to create a Telegram account.


> "No phone number necessary" is only true if Google Voice is available in your country. A phone number is necessary to create a Telegram account.

To add to that I can use Signal just fine with Google Voice. So if both Telegram and Signal require a Google voice number, might as well go with Signal.


I would like to use Signal, but I am forced to use Telegram for the same reason. (I have also to say that Telegram mac client is pretty awesome).

It makes no sense to create a "secure" chat app, and then to force your users to use cellphones, which is the most unsafe technology I can imagine... Why this cellphone fetish?


Cellphones are far more safe than your computer - especially iPhones. All apps are run in a sandboxed environment and are vetted before being released. Further, the secure enclave is far better at protecting secrets than anything on a typical laptop/desktop machine.


Nonsense. Some adroid virus, like Triada, is a nightmare compared to nearly anything even on Windows.


but:

- you are tracked everywhere - you don't control the software for real - you have almost no control on connectivity - it's super difficult to kill a process

etc.


no, this is wrong. secure enclave or not. radio chips have direct memory access. phones are only as secure as providers want them to be. Computers actually do what I tell them to --I don't need hacks to "root" them or inspect their behavior.


iPhones are secure, maybe they contain backdoors from Apple and we don't know, but Android are not secure at all, especially because most Android vendors usually don't update the OS to the latest security patches, so the majority of the Android phones out there are full of unpatched security vulnerabilities.

Also there are not good free software mobile operating systems, sure there is LineageOS and other ROMs that still require some proprietary parts, mainly the firmware of the device and binary blobs in the kernel, for one person concerned about privacy that is a problem, because proprietary software means backdoors, and it's useless to use a fantastic free software secure communication app if we can't trust the OS where we run it.


You're talking about Android not being secure because it uses proprietary blobs, but saying that iphones are secure because both the hardware and software is proprietary?

Apple's reality distortion field in full effect...

Neither Android nor iPhone can be considered secure.


> No phone number necessary.

How did you sign in without a phone number? It's not possible on https://web.telegram.org/#/login.


I see a lot of "just use X instead."

Unfortunately, unlike the old chat protocols, switching to any other platform means convincing your contacts to use a new platform. They like you and all, but that means they also have to use a special app just to talk with you now.


It astounds me that something as simple as talking to people is in some ways objectively more difficult now than it was in 1990.

What the hell are we doing?


Wow that's impressive. But I would imagine WhatsApp/Facebook can just change their protocol at any time since it is easy to redeploy a new version of the WhatsApp Web client, thus breaking any 3p clients built on the original protocol. That would require yet another reverse engineering effort that can take a while. And by the time its reverse engineered again, they can yet again change the protocol. So the only reliable way to create 3p clients would be if WhatsApp itself publicly publishes its protocol.


They have a closed beta program for an "Enterprise" version that is supposed to have an API. Seems like a good way to monetize.


On really? In that case they definitely would not want to allow this project to cannibalize their revenue. On the other hand, it makes it harder for them to change their APIs then as it impacts clients not directly under their control.


they still have to support older clients. so it is not that easy to just change the protocol.


No, because it is a website.


They also have native apps on most mobile platforms (including Windows Phone, Blackberry and even Nokia Series 40). It isn't as easy to update all of them as one would think.


But those are the apps themselves. This is about the WhatsApp Web protocol which allows you to use WhatsApp from a browser window by pairing it with a live session on your phone. Since the API endpoint are their own servers and the client is literally just a website, they can update whenever they want however much they want.


One of the things that has repeatedly pushed me off WhatsApp has been their very aggressive stance on this: I've several times stopped using it because "you need to update your app or this will all stop working in 7 days" and I've not been in the mood for jiggling the apps round on my tiny phone to be able to perform that update: the 7 days has passed, and I'm off WhatsApp again until such time as I "need" it.


You can always screenscrape.


> it is easy to redeploy a new version of the WhatsApp Web client

You can choose your version when you send the requests


I've wanted to create my own WhatsApp client that's and actual native desktop application. If this reverse-engineering process continues, it might now be possible. It's irritating when people say "Signal/WhatsApp has a desktop app!" because technically, they're right, but I have enough web browsers on my system already, thank you very much


> my own WhatsApp client that's and actual native desktop application.

What would it do differently?


look at yowsup


Curious to know why they chose to require python in addition to node, wouldn't node with npmjs/yarn be sufficient and require less setup? Does python/pip provide any benefits here?


Hi, I'm sigalor, the original creator of the project. Actually, now I also regret using Python in addition to NodeJS. When I started all of this back in November, I originally chose Python, because it's, well, "quick and dirty". Especially trimming arrays and working with byte strings requires a lot more code in JS than it does in Python. I even wrote a reimplementation of the decryption routines in JavaScript, but it's not working entirely (the HMAC authentication of received messages fails, though login works).


Funny, I had the exact opposite question.


Wouldn't it be better to simply start convincing your friends to use something more open, where you have choice of the client? It feels like solving the problem from the wrong angle…


Have you tried that?

Ends up with some friends throwing you a bone and downloading / registering for a new service. Some of them remember to keep it open. Some use it. The rest of their friends don't. But a few of them love you enough to use a special app just for you because you seem to care. <3


Actualy I almost don't use WhatsApp. Saddly I settled on Telegram (sigh) but it's the only remotlely sane solution to having desktop client that doens't require mobile phone being constantly on. I would love to have XMPP more popular, but while I love the protocol it had quite some shortcomings (carbons only getting popular recently for example)


Better? Yes. Easier? No.

Besides, what's more open, as usable and secure?


Signal. It is at least as secure as Whats App by design, has pratically the same interface and also a Chrome-based desktop app that works untethered from the phone app.


Out of curiosity: I’ve noticed a long-term sceptical attitude to telegram in HN audience and have seen multiple arguments against it. Something like that their crypto can’t be trusted, that it’s not time-proven. Don’t you know any good source with some sort of domain expert explanation, why shouldn’t it be used or trusted? No intention to start any flame against Signal, only curiosity regarding telegram flaws. Personal point of view is also appreciated.


> Don’t you know any good source with some sort of domain expert explanation, why shouldn’t it be used or trusted?

People like tptacek have talked here at length about why Telegram is not trustworthy, you can see a history of his comments with a simple search: https://hn.algolia.com/?query=tptacek%20telegram&sort=byPopu.... Moxie Marlinspike has also pointed out a bunch of problems with Telegram, and even if you don't consider him a trustworthy source because he runs a competing service, the technical reasoning behind his opinions is sound.

If you want a personal POV, here are three reasons why Telegram is a bad idea:

1) The large number of unsound technical decisions. See Thomas and Moxie's many comments for details, or the "Security" section on its Wikipedia page.

2) Within days of launching, they had a critical security vulnerability: https://news.ycombinator.com/item?id=6948742. Frankly, this alone should have discredited them forever, especially considering how much boasting they were doing beforehand, but people are stupid.

3) They have a consistent pattern of responding to criticism not with technical defenses, but with ad hominem attacks and conspiracy theories ("You're paid by the US Government!")


Some years ago all you needed for Whatsapp was the phone number and MAC address to login and view all messages. Nobody gives a shit about this today. Should have discredited WA forever too.


It's not even end to end encrypted by default. That's the main reason why you shouldn't use it.


This right here, how can such a basic step to protect your users be skipped?


Same way as in email, banking etc?

Also I find it puzzling that so many people here keeps on recommending WhatsApp over Telegram after all the lies from WhatsApps owner.

Edit: While I have no way to verify this, AFAIK both Telegram and Gmail stores data and keys in ways that makes them hard to access by everyone except for the user.

Telegram in particular say they do this by storing data and keys in different datacenters in different jurisdictions.

Add to this that WhatsApp has had their fair share of issues as well before they started working with Moxie.


They rolled their own crypto. Just Google "telegram security" and you'll find explanations of why that's a red flag.


Someone has to roll new crypto, otherwise we're stuck. That said, I know about the potential issues with Telegram's encryption.


You don't roll a new crypto and use it the day after, it must be tested for vulnerabilities, reviewed by expert cryptoanalysts.

It can take years, much like a car has to be crash tested, a new crypto algorithm must go through a certain process to be considered good enough.


AFAIK WhatsApps crypto isn't too old either but that doesn't seem to prevent HNs resident cryptospecialists from recommending it.

That said: I belive them when they say that WhatsApps crypto is stronger.

On the other hand I would expect them to leave a little note somewhere about WhatsApp being a data collection tool for Facebook that also still happens to works as an instant messaging platform.


Any flaws in Telegrams crypto that lead to decryption?


None discovered yet, just like my new design, this cardboard box on wheels I'd like to offer you as a replacement for your car.


Car comparisons aren't viable.


It's Russian. You know, like: hand over the encryption keys or you, or someone you love will disappear.


Telegram is actually fighting the Russian government over the encryption keys, saying that it is impossible to hand them over (I assume this is for e2e encrypted secret chats). The consequence of this action is that they'll likely get banned (i.e. removed from app store). How much of this is a farce remains to be seen, since the whole nation, from casual users to small businesses to government employees use the app daily.

As a Russian, I do appreciate the fear that the "russki" brand instills in your soul, but I think you are rightly being downvoted for jumping to conclusions simply based on nationality.


I too appreciate nationalist sentiment, but the parent clearly stated: 'personal opinions also appreciated'.

But to be clear: I have a lot of distrust against governments regarding mass surveillance. But I distrust some governments more than others. And Russia is relatively high on that list for me. I think a healthy dose of distrust would be fitting for Russians citizens too.


Signal has most of the problems that has WhatsApp: mainly the fact that it's dependent on a smartphone, yes you have a Chrome-based application that is the same as WhatsApp web, it's only a remote interface that connects to your phone.

Telegram in my opinion is far better, it's completely cloud based, you can use it from whatever device you want, it has real desktop apps, you can send files, you have bots, channels, large groups, usernames, you name it.

I don't get why using Signal, yes it's free software, also Telegram it is (ok, the server is proprietary but even if you have the source how can you be sure that what they release is what is running on the servers ? If you don't run your own server the source are useless), but I don't see other advantages, so why bother with a third messaging app ? I use WhatsApp for the large user base, and I use Telegram for the advanced features if I need them.


No, Signal Desktop is NOT the same as WhatsApp web. You still need your phone for the initial setup (same as Telegram), but after that, the desktop app is untethered until de-authentication.

Also, it has a much more praised security and cryptography than Telegram, is always encrypted (Telegram is only encrypted in secret chats) and has a much more secure codebase, with more open development (Telegram sometimes takes weeks to release source code), reproducible builds and a more transparent history.

I do use Telegram (mainly for group chats), but I treat everything posted in it as I would treat a public forum like HN.


Are you certain Signal is dependent on a phone? The new Signal desktop client I use seemed to send messages fine over in-flight wifi.


Signal relies on GCM, and needs a smartphone app. I'm failing to see the win.


Not anymore since 3.30.0. Also CopperheadOS maintained a fork on their fdroid repo without the hard Google Play store dependency.

https://copperhead.co/android/docs/usage_guide#signal


The more I dig into Signal, the more complicated everything with it is.

Just go XMPP with OMEMO, so no hard smartphone dependency, no electron app monsters. Thankfully XMPP doesn't have a problem with 3rd party and federation.

https://omemo.top/


My friends switched from whatsapp to signal for day. We switched back after a matter of hours. We couldn't find a way to quote messages to reply to them, which is critical for group conversions where multiple threads are happening at once. Also there was no ability to @user to direct a message at a specific person. As for the app, there was unnecessary whitespace between messages--even using smallest font size. Usability vs unauditable security is tough calculus but I think technical people weigh security more then their less technical friends.


I wish signal had a webapp.


Not very likely any time soon. There was a long debate about it and they decided that putting your trust entirely in the CA system and Signal's servers every time to not serve a malicious client that couldn't be validated by the user wasn't acceptable.


Why can't I self host a web app.

I just don't want any more applications open.

I'm CA's will be bypassed soon. There's a lot of brain attention on this.


It has a desktop app for MacOS, Linux and Windows. https://github.com/signalapp/Signal-Desktop


It's as "native" and "desktop" as... well... ekhm: "Signal Desktop is an Electron application that links with your Signal Android or Signal iOS app."


It's also exactly as open as WhatsApp, but not as usable.


Signal is open source both client and server. To my knowledge the same isn't true for WhatsApp.


It's open source, but Moxie has said he doesn't want federation. I don't think he'd be okay with someone writing a third-party client, for example.


He doesn't, but only because of the maintenance burden that would bring: https://github.com/LibreSignal/LibreSignal/issues/37#issueco...

I'm sure if a third-party client would contribute to support the maintenance (both financially and in terms of the time and effort investment) he might be open to that, but obviously that's not going to happen.


Sure, burden. Like say, Pidgin is a burden to [x network]. Not allowing 3rd party is bad, it always is. It takes away choice.


Federation is a completely different story from a third-party client.


Open Source, relies only on Twilio, AWS, GMC, and Apple Push. https://github.com/signalapp/Signal-Server/blob/master/confi...



After playing around with federated XMPP (on my own server): XMPP with OMEMO is brilliant. No battery drain (Conversations and Astrachat tested on android), multi-client e2e encryption, even voice/video is possible. And, being federated, I finally own my identity.


What's the benefit of end-to-end encryption if you don't have anyone on the other end?


Nobody can read your secrets if you don't have anyone to send them to.


I've underestimated XMPP myself, there's a surprising amount of providers out there.


[Riot](https://riot.im), native mobile clients for all platforms plus web client.

It's self hostable or you can just login on their Matrix server.

Ah, and no phone number needed.


I have high hopes for Riot, but last I checked there was no end to end encryption (or maybe it was a proposal). Is it farther along now?


e2e encryption landed a while back and works pretty well (although technically is still in beta). the UX has some warts which we're working on :)


Impressive work.

Obviously, WhatsApp/Facebook would want to avoid a bunch of third party apps connecting to their service. How long until they make changes to make this more difficult/impossible?


If you have a web API it's impossible to secure it, especially when you have many platforms that access it ( web / mobile ect ... )


You can change it often enough to be really annoying though. And Whatsapp bans users of third-party clients it can detect. At least the users I know stopped trying these things after a while.


They "secure" it through legal force. Many startups are shut down by legal threats based on the CFAA, which effectively makes it illegal to talk to a server after you've been informed that you aren't allowed to do so (ordinarily, this information is conveyed through the Terms of Service -- a C&D is typical but not strictly required).


Let's hope it does not take them much, because I don't like spam.


There are much easier methods to spam, and they're very good at dealing with them.

This is only useful for real users who want to write custom applications that connect to their phones.


FWIW Repos like these that reverse engineer a proprietary API that post stuff on GitHub are usually taken down with a DMCA enforcement. The same thing happened multiple times when folks reverse engineered and documented the Snapchat API. https://news.ycombinator.com/item?id=6083812


yowsup has been online for years.


From the GitHub readme:

> An UI that is not that technical, but rather starts to emulate the actual WhatsApp Web UI.

No, no, no. This trend of 'Phone UI' chat interfaces on desktop/laptop screens needs to stop. If you are going to all this effort to reverse engineer the protocol, at least make your front end customisable or at the very least IRCish in style.


I'm missing why this needs both a python and a node backend.


Hi, please refer to my answer to a similar comment: https://news.ycombinator.com/item?id=16796791


when i've done similar stuff (with java and node), i've used node because nodejitsu can proxy websockets


Many of the vendors we partner with (tourism industry) live in countries where the main communication channel is WhatsApp. There's a lot of communication we want to automate in the near future—eagerly looking forward to seeing this progress!


Were you inspired by this repository https://github.com/mukulhase/WebWhatsAPI?

Do you need a phone running to use this project?


I'm wondering how they actually reverse engineered WhatsApp in the first place. Is there a specific type of software that does this or was it just built from scratch using already available information?


Hi, I'm sigalor, the original creator of the project. The reverse engineering was almost entirely done using the Chrome debugging tools. That is, pretty-printing the JS source files, setting breakpoints and stepping through the code for hours. When I started, all of this was incredibly difficult, but the longer you do it, the more you get used to it. Additionally, the debugging tools also provide you with looking at what is sent through websockets, which makes it rather easy to see which JSON data is sent (e.g. for login).


That must've taken forever. Do you have any plans to reverse engineer other apps? I know people like you are in short supply and high demand.


It certainly did, but after all it was just a fun spare time project. I guess there would be a lot of interesting software to reverse engineer; I am always open to suggestions that are able to extend my knowledge. And well, if you mean it in context of a job... I don't have any experience regarding the job market yet, but that also sounds quite striking :)


I don't know much about the job market, I was talking about the internet in general. Too many applications are locked black boxes and reverse engineering them basically keeps them alive after their demise. However, not a lot of people actually put in the effort to reverse engineer this stuff, so keep up the good work for this stuff!


A pidgin plugin would be nice. Oh there seems to be one already - https://github.com/davidgfnet/whatsapp-purple/


There's also a Python library: https://github.com/tgalal/yowsup


Don't even bother with yowsup, you will be banned after wasting a lot of time setting it up


Yeah, but it hasn't been maintained in a long while. Unless someone steps up to maintain it, not sure it'll be usable for long.


Let's bring some more misery to Zuckerberg


Why did they not write the backend server for Whatsapp web in Erlang, which the original Whatsapp was mostly written in?


Well, I don't know Erlang (yet) and AFAIK, Erlang is rather focused on fail tolerance, high availability etc., which wasn't really a concern when I started the project. Python and NodeJS are quite good for quickly trying out ideas though.


Noob question: Is traffic going through the websocket-servers properly end-to-end encrypted? That's what always held me back about using Whatsapp Web


Hi, that's definitely not a noob question. I'm already having plans on investigating this, but this topic is even more difficult than WhatsApp Web itself (see https://github.com/sigalor/whatsapp-web-reveng/issues/10#iss... ). Thus, at this time, I am not able to give you a definite answer, though, to put it informally, "it looks good" (at least on the surface).




Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: