Hacker News new | past | comments | ask | show | jobs | submit login
App suddenly crashing on startup due to FBSDKRestrictiveDataFilterManager.m (github.com/facebook)
596 points by reubensutton on July 10, 2020 | hide | past | favorite | 432 comments

Waze, a google owned app for the past 4 years, crashes because of the FB app.

WhatsApp, a Facebook owned app, will not give you read access to your own chat database - it’s encrypted. But they have an arrangement with Google where it’s will back up unencrypted to your Google account (though, Google won’t let you retrieve it ... only the WhatsApp app can)

Everything about this duopoly, their cooperation and lack of competition stinks to high heaven.

Also, you can't migrate WhatsApp between iPhone and Android. There's a utility called BackupTrans, which follows an extremely convoluted process including a step where they install an old version of WhatsApp. It seemed to work but it trips some wire and WhatsApp put me in an hour-long timeout state, twice. Since I very much depend on WhatsApp, I dare not use it anymore.

The platform lock-in because of this is immense.

I've helped a lot of people to set up new phones and saving WhatsApp chat history is always low on their list of priorities. Keeping their contacts, their photos and all the saved logins for websites and apps are the top 3.

Wiping the WhatsApp chat history on the old phone before giving it to a new owner is also much more important. It often contains highly embarrassing or even incriminating details.

wouldn't it be advisable to do a factory reset on an old phone before giving it to someone else? There is a lot more personal info on there than whatsapp

Maybe so, but it doesn't benefit either platform.

It may not _favor_ either platform but I'd say they both benefit.

In ecology, it's rare to species compete to extinction or exclusion from a niche. It's much more common to observe segmentation. It's a lower energy state for birds to mate at different times, eat leaves from different parts of the same tree, etc., dividing the resources of their niche among themselves, than to force each other out of the niche completely.

The situation is analogous. I'm sure they'd destroy each other if they could, but it's obviously not happening right now. It's much easier to say, you can have your current users, we'll have ours, and we'll focus our energy on capturing emerging markets. Which is why you see them racing to provide free internet to developing countries and such.

BackupTrans also caused my iPhone to bootloop, requiring a factory reset.

You can still download the history as a text file though - I did that.

Not in Germany: https://faq.whatsapp.com/android/chats/how-to-save-your-chat...

Don't ask me what's leading WhatsApp's legal department to believe that this feature doesn't comply with German privacy laws. It's quite puzzling. (I would've rather expected a feature like this to be mandated by EU laws.)

As of now, there's no non-hacky way of exporting chats for German users. (Hacky ways include: dubious third-party software, copying each message individually, taking screenshots.)

Most probably not privacy related, but the result of Blackberry suing Facebook over some patent stuff in Germany. (German source: https://t3n.de/news/whatsapp-kein-chat-export-mehr-1238327/)

Thanks for the pointer! Of course it's software patent trolling, ugh.

Can you obtain that same information via a GPDR request for all of the information they have about you?

Messages and associated media aren't permanently stored on WhatsApp's servers, so sadly this wouldn't help much with chat export.

Time for a road trip?

Isn't that just for one chat? Can you download all combined?

Which direction? I and my sis did both directions quite painlessly (includign images etc) a year back.

From iPhone to Android. The software works. But shortly afterwards, WhatsApp threw up some kind of "timeout" screen twice, where I couldn't use the app for a hour. I don't need that uncertainty.

The android/iPhone thing isn’t on purpose, it’s an artifact of the organizational structure at WhatsApp

WhatsApp isn't made by a single indie developer, and wasn't just released by that single indie developer a month ago. If it can't sync between Apple Phones and Android, then WhatsApp has intentionally chosen not to fix that and instead spend many millions of dollars on other stuff. It is on purpose.

Never used WhatsApp, but is it related to Apple policies? I just ran into something reminiscent of this with Microsoft's Authenticator app - runs on both platforms, supports backup and restore, but apparently on Apple devices it's only able to use iCloud for backup rather than an associated Microsoft account.

The net effect is that if you're switching you have to redo 2fa for all accounts where you haven't separately backed up your TOTP codes.

What makes you sure it's not on purpose? I don't disagree, I'm just curious about your statement.

Why would WhatsApp want to make it hard to stay on their platform?

Thats a different question but 1) they know and 2) they can fix it fast therefore 3) it is on purpose

I'm pretty sure that's the same agreement that every developer has with Google just by writing an android app and using Google APIs.

You can store data on Google Drive that isn't accessible from Google Drive and only that app can read or write it.

> I'm pretty sure that's the same agreement that every developer has with Google just by writing an android app and using Google APIs.

But why would WhatsApp store it unencrypted on Google Drive, and encrypted (without giving you any access to the key) on your own SD card or local backup? It makes no sense.

Except ... Google and Facebook have agreed that WhatsApp stores the database unencrypted, and in return Google doesn't count it against the user's quota. Everybody wins! Facebook get to have free backup for users, Google gets to see all user messages and metadata. And the user gets rid of this pesky privacy thing!

[0] https://faq.whatsapp.com/android/chats/about-google-drive-ba... - "WhatsApp backups no longer count against your Google Drive storage quota." and "Media and messages you back up aren't protected by WhatsApp end-to-end encryption while in Google Drive."

"But why would WhatsApp store it unencrypted on Google Drive, and encrypted (without giving you any access to the key) on your own SD card or local backup? It makes no sense."

The design is pretty clearly to support the bog standard backup use case of "something happened to my device, I got a new one, I want my old data".

It would be nice if they added an option to also encrypt backups, although I think it would be non trivial to design where the key material would live. (phone enclave? Multiple devices? adding/removing devices? etc) Not to mention just having the option might cause more harm than good if people turn it on and regret it.

I think their design philosophy here is likely "let's err on the side of smooth UX and let the 'end-to-end encryption is srs bzns' loonies go to signal"

As mentioned in my previous post:

They already support encrypted backups. In fact, they don't support unencrypted locally.

> The design is pretty clearly to support the bog standard backup use case of "something happened to my device, I got a new one, I want my old data".

They had, in fact, perfected the local encrypted seamless "something went wrong" workflow - they keys are delivered from the server (stored or generated, I don't know) once you've proved you can receive texts on that phone number. Without that, the local backup is useless.

And IIRC, they ALSO did the same with Google Drive. But then, one day, it stopped counting against your quota and the encryption disappeared.

> I think their design philosophy here is likely "let's err on the side of smooth UX

The UI was as smooth as it can be. It still is, for local backup.

I try not to attribute to malice that which can be explained by stupidity, deadlines or missing budgets, but in this case, there was a decision to remove encryption, that required negotiating with a rival. That decision had costs, and the process had costs. There is a benefit associated or this decision would not have been taken. I don't know exactly what it is.

Thank you for laying bare the specifics of this issue. It does seem indefensible.

The unencrypted backup system is almost certainly implemented mainly to give government authorities easy access to chat history when presented with a warrant whenever possible. Since this is offered as a helpful feature, very few think twice about the part in asterisk. This way they can comply with warrants without backdooring their app while continuing to sticker label "end to end encryption" and giving people comfort that their data is in "their" drive. There's no other reason really for such a weird arrangement.

That doesn't make much sense. There is no such legal requirement (hence the discussion). The Google backup option isn't universally used. And WhatsApp controls the key generation and management and could easily integrate any other sort of more robust backdoor. Such as their previous system, as described somewhere in this subthread, of storing the key on the server to allow recovery.

I'm not saying it's a legal requirement, I'm saying it's likely a smart tacit compromise.

On one hand - even after end to end encryption, we continue to see WhatsApp messages quoted in prosecution including in SEC actions and prosecutors are clearly getting access to messages even pre-arrest. The ability of prosecutors to compell unlocking after arrest in US is questionable. So how are they getting access to the messages if there's truly end to end encryption? Also there are any number of Governments in the important non-US markets that would simply ban WhatsApp if they didn't have access against a warrant (eg. See BlackBerry cases from some years ago)

On the other hand, no security researcher seems to have found or at least reported a back door in the WhatsApp apps that supports MITM undetectable to physical key verification which is the obvious weakness in an end to end encryption system based on centralised key repository system.

On the gripping hand, all high profile cases trying to compell tech companies to provide evidence have been against Apple. I've not heard of one against WhatsApp.

Ergo - the most like scenario is that this feature is deliberately built keeping messages un-encrypted so that when law enforcement shows up with a warrant or a muzzled warrant, they can just hand over the archive without weakening their app. People are convinced to opt in because it's the only way to change devices while keeping your messages. To give people comfort that unencrypted messages ate safe, it is sold as as "it's in your own Google drive"

So, Facebook is willing to give up data on its users, it’s highest competitive advantage, to its largest competitor, in exchange for... some commodity backup disk space?

They likely get more in return that we're unaware of ....

But, you know -- they gave the same kind of info (and much, much more) to firms like Cambridge Analytica for free for years. So perhaps they didn't even get more in return.

Facebook's purchase of WhatsApp was OKd by EU regulators subject to some constraints, e.g. that they don't use WhatsApp data to enhance FB graph and/or gain other unfair advantages. FB has since admitted to using WhatsApp metadata in violation of that constraint (and didn't even get a slap on the wrist) - and perhaps giving the data to Google is also part of playing dumb. I don't know, but I'm sure it has everything to do with benefit for FB and GOOG, and almost surely at the expense of user privacy.

I think this statement is being misunderstood by a lot of people here.

The Google Drive backup is most likely still encrypted, just not "end to end" because the key needs to be stored on a Facebook server.

That makes a lot more sense to me.

I don’t follow this closely, and this may have changed recently but As of 6 months ago, people have managed to download an unencrypted copy from Google Drive by impersonating the WhatsApp app.

Is that a GDPR violation?

I would guess this is actually done by many to conform to the regulation.

with all the daily (justified) outrage against FB/Google I struggle to understand why the community here is happy to applaud these companies for its innovation on specific projects or the work of specific engineers, ... whether it's project-zero or some fb framework we seem to be OK with praising their employees while at the same time hating how they use that innovation.

some serious cognitive dissonace is going on - either that or it's the effects of the news-cycle making us blind to it. Maybe we should think twice before applauding FAANG engineers in future posts because Technology isn't neutral.

note: a company can't be evil. it's just a group of people who can be evil and who hide behind the facade of the "legal person" statute. I'm happy to celebrate individual breackthroughs but let's not forget that without (metaphorically) nailing the culprits to the cross who are responsible for ICE contracts or privacy invasive tech there isn't even a point in complaining about all the horrid stuff these people (working for these companies) do.

Have you ever worked at a large company? Really the only thing that groups the whole thing together is the legal entity. Different teams within the same company are essentially different companies once you reach a certain scale. I see no reason to disparage the good work one group does, because of a different group of people they've never met before, for no other reason than they share a CEO. A CEO that neither group has ever met before.

this is a pretty simple way to absolve people of responsibility. Just because an institution has compartmentalized tasks, so that every single individual team appears to do nothing wrong, doesn't mean this actually absolves individuals, because following that logic obviously nobody would ever be responsible for anything.

It's like Kierkegaard's example of the mob who collectively intimidates but every member argues they have contributed virtually nothing, and so any sort of responsibility is simply dissolved.

Anyone who is responsible needs to be aware of the holistic system that they're part of and that they are enabling rather than just their own little world. That's no excuse.

>this is a pretty simple way to absolve people of responsibility.

Consider this menu of possible ways to be associated with a human rights violation:

- Do it yourself.

- Be a part of the chain of command that ordered it.

- Be commanded by the chain of command that ordered it.

- Be part of the same organization as the one that contains the chain of command that ordered it.

- Be a member of the society that contains the organization that contains the chain of command that ordered someone to do it.

- Be a member of another society, that could have tried to stop it, but didn't.

Where do we switch from participant to accomplice to bystander to "completely uninvolved?"

That one had already been answered:

First they came for the socialists, and I did not speak out because I was not a socialist.

edit: In case it isn't clear: You just chose sides right there in your comment.

Same could be said of an industry, and economy, or even a species. We are all part of systems at many different levels. Our critics might attempt to tie us to the bad parts of our systems and our proponents might seek to separate us from those same demerits. But I think either way of looking at it is just rhetorical.

> Have you ever worked at a large company?

yes - and I've done incredibly well during it. I also know change doesn't come for free. the sad thing is the bigger the companies the smaller the spines of the individuals working there.

How much change have you effected in all your time there that you did so well? How much power over projects you deemed immoral did you have? More or less than a random person on the street? Do you feel more responsibility for immoral projects that took place in this large company than for things going on within FB or GOOG?

Effect doesn't matter, it is the trying that counts. IOW: Don't give up so easily. Every bit of freedom we have took a big struggle to obtain. We've lost many freedoms and we will continue to lose more. Our struggle is far from over. If you could have done something you are responsible for the result. People all over are counting on you giving up. Don't make things so easy for them. Make them work for it.

Do you spend the entirety of everyday dwelling on all the world's wrongs? You, right now, can go dig wells, volunteer at shelters, adopt needy children, give blood... The list keeps going. There are a hundred thousand things that you can do to help others and fix things: Do you feel responsible?

Yes, every day. Are you suggesting I could make more productive use of my time?

We all put thoughts into our own well being. When sufficiently comfortable or successful at that we start caring for people we interact with on a daily basis. When that too is a reasonable success we start doing stuff for people we interact with less frequently. At some point you can chose to go look for people who need our help, do something for the town, the city, the country, humanity etc There is tremendous joy to be had in this process and it builds character. I admire people like that.

People who don't do it are missing out. They in stead keep telling themselves they can't do anything for others knowing they are lying. The endless rationalization of the irrational takes a huge amount of time until they graduate themselves in the DIY course in psychopathy. In their ego trip they end up feeling sorry for themselves. In their self inflicted blindness to what misfortune looks like the self pity knows no boundaries. Also: no one loves me! I wonder why that is? They keep trying to spend money to buy happiness but all they get is cheap thrills. Some end up in a pathetic cycle where all they can do is expand their wealth, ignoring even their partner and their kids. They don't have time for that, they have important other things to do. I pity people like that. If I can think of some way to help them with their condition I will do the responsible thing.

There's a long history of that. AT&T used to have a stranglehold on communications and had some seriously bad practices.

But they also produced some really cool tech. Unix being the obvious one to bring up on this site.

That era is romanticized to hell (including by me), and Ritchie, Thompson, and the like are folk heroes. Despite being engineers at the subsidiary of a giant monopolistic entity.

Ultimately, for people who think the tech is cool, it doesn't matter where it comes from. In 50 years people will be romanticizing the era of the FAANG companies (more than they already do) the same way we romanticize bell labs.

Maybe the people doing the lauding are different from those doing the criticizing? I'm inherently suspicious of statements which treat a discussion forum as monolithic.

If a company like Monsanto cured cancer, couldn’t you laud that effort even if you still disagree with other practices of the organization? Things, people, companies are rarely all good or all bad. We should applaud Facebook for their FOSS projects, while, for instance, pushing back against the walled garden and other data practices.

Both companies are huge and contain multitudes; they can easily do praise-worthy and scorn-worthy things at the same time. It's worth calling them out separately rather than trying to unilaterally love or hate the whole mess at once.

That said, with FB/Google these days the bad weighs more than the good.

That right there is why runaway consolidation and Monopoly building is a very bad thing even if they don't jack up prices like Ma Bell. If the leaders of these megacorps have no scruples, the public has little choice but to take the bad with the good. We've lost the ability to vote with our money when it turns out buying a mobile phone or joining a social club which makes use of a social network ends up funding antisocial and even evil behavior by other parts of those companies.

the problem with "huge companies" reasoning is they require huge civil courage and a large sacrifice from the public to be set straight once they mess up. big companies = big damage.

just because we made the mistake to let them become this big without giving them criticism doesn't justify why we should continue treat them with gloves. quite the opposite. they shouldn't feel comfortable at all in their position - they should feel like they're being held accountable (and act accordingly whether with rebelling internally or by leaving depending on the size of their spine)

But technology is neutral and it's important to keep it that way. Giving into cancel culture and groupthink is the worst of things. Even worse that FB and Google.

Cambridge Analytica was evil from the start. Blackwater was pretty evil from the start. Yeah, companies can be evil.

> WhatsApp, a Facebook owned app, will not give you read access to your own chat database - it’s encrypted.

On android? On iOS you can get a tarball of all your chats.

I just had to check this; the Android app has "Request account info" and specifically mentions "which you can access or port to another app".

As much as I'm happy to pile on Google, Facebook and WhatsApp, this appears to be untrue.

It also specifically mentions that doesn't contain all the messages, and takes up to 3 days. You can export individual chats as texts, but not all the attachments, metadata, etc - and you have to specifically select each one.

The statement that "Facebook gives an unencrypted copy of your chat database to Google but will not give it to you" is true; I'm referring to the SQLite database. You can back it up locally, but it's encrypted and Facebook will not willingly give you the key under any circumstance (though it is possible to trick them by impersonating the app). However, the Google Drive backup is unencrypted. But Google won't let you download it directly either (although, again, you can trick them by impersonating the app).

Thanks for clarifying, makes sense. This sort of behaviour really gets my goat. Just like trying to get my 2FA keys out of Authy so I could switch to Aegis involved having to use a third party tool written in Golang on my PC, to extract them using the API, or use some js hack on the deprecated Chrome extension to write them to the console.

Recently I was trying to switch my wife away from the free Keeper app to the one I use and backup to our own storage, they make it impossible to get your passwords without paying them or rooting your phone. I hand copied them one by one from her old to new phone. Honestly, screw any company that does this.

isn't whatsapp using e2e ?

if that's the case then facebook can't "give you" the key, because they don't have it, it's supposed to be on your phone and only there..

Is there something i'm missing here ?

It's using e2e during transmission. On your local device, it is plaintext.

They make a copies of that plaintext database for backup purposes:

One on google drive (not mandatory, but it IIRC by default), which is still plaintext but not accessible to yourself.

One on your local SD card (if you ask for it), which is encrypted with a key that's on your phone but was sent down from FB servers. If you switch your phone, and try to restore that database, it will contact FB servers to retrieve the key.

> if that's the case then facebook can't "give you" the key, because they don't have it, it's supposed to be on your phone and only there..

Regardless of the backup encryption, e2e doesn't mean that they don't have the key; it just means that it leaves one phone encrypted and enters the other encrypted without being decrypted along the way -- unlike, e.g. regular email, which is usually encrypted in transit by TLS, but gets decrypted and re-encrypted at every stop along the way.

That is just what WhatsApp stores about you on their servers i.e. contacts and groups.

But you can export all chats one by one.

On android you can select "export chats", select contacts and then send it to where you want it.

You can do that on iOS as well. That has been a feature for a long time.

Yes, individual chats, and not a complete record last time I tried (no "when received / when read" times; no attachments). There's a simple SQLite file containing everything; They upload it to google drive unencrypted, but neither google nor facebook will give you an unencrypted copy.

When you make a local backup, it is encrypted, and Facebook doesn't think you deserve a copy of the key or an unencrypted copy.

I think this is incomplete (it doesn’t represent replies or media properly) and the parent comment was talking about the SQLite database containing all messages

You can download your chat backups as a readable archive file. If you have iCloud mounted, these files can be found in your iCloud folders.

Or do you demand that any file under your control that contains chat should be accessible, rather than just one of them?

On Android, this used to be the case, but then Facebook and Google reached an agreement in which the WhatsApp backup goes unencrypted, and doesn't count against your Google Drive quota.

You can't download it without impersonating the WhstsApp app.

Do you know if the icloud backup is encrypted or not?

I have managed to extract the files from icloud but found that they are encrypted SQLite files. How do you access unencrypted whatsapp files?

I really wanted to like Signal and switch, but it seems to require a phone #. Are you OK with tying private conversations to an identifiable ID or did you find a work around?

I'm ok with tying some conversations to phone #. Others, not so much

Had to restart my iPhone 7, now waze works again.

Haven’t seen anything like this before.

Yes, it's definitely time for some slightly enhanced armchair activism!

Convert all active whatsapp contacts to signal? 80%? 50%?

Same for Facebook messenger!

It needs to be uncomfortable enough - if they want to communicate with us and are minimally tech savvy ("click link to install app") they have to switch to signal.

That's it.

I'm starting to send out messages over my morning coffee.

Let's do this together:


It's wild to me that

1. There are tons of replies in this thread. People have very strong feelings about signal.

2. Some people feel VERY STRONGLY that the pin numbers are stupid, and it has a terrible UX.

3. Some people feel VERY STRONGLY that the phone numbers are the simplest, most straightforward thing to use, and they would hate Signal if they didn't support it.

4. Some people feel VERY STRONGLY that they don't want their phone number used as an identifier, and it's a failure of Signal that they use them.

And it seems like many of these replies assume that their own position is the only possible one to take (or at least the only intelligent one!).

Imagine working on Signal and trying to create a product that pleases all these people, as well as even less technically minded folks, while having none of the usual analytics at your disposal to measure actual user interaction!

My hat's off to Signal. I admire their work but I don't envy their position.

I’d rather keep using messenger than switch to signal. Nothing that requires my phone number, and even worse, uses it as primary identifier, is something I want to switch to (and no, obviously I don’t use Whatsapp either). I realize that in this mobile-centric world that makes me a grumpy old man, but for me, my E-Mail is the important ID.

I switched phone numbers a few years back and I can't use Uber because my account is tied to my old phone number and the new phone number already had an Uber account. It's so dumb.

I’m curious why you see a difference between the two? Phone companies have been giving away your phone number for years, it gets more spam then my email address at this point, which has made me disable it for all incoming calls except those in my contacts.

At this point, none of us even use phones, my family pretty much use Zoom and at work everything is Google Meet, my phone function is basically a dead function at this point.

I don't truly own my phone number. I barely control it. I can move it from provider to provider, kind of, but it's still not _my_ number – the number is a side effect of having cell service. In particular, if I move to a new country, what happens to my phone number? (Having recently done so, I still don't know: for now, I'm paying two cellphone bills, but that's obviously not a long-term solution, and I don't know what is. Port it to Twilio, I guess.)

You can make the argument that I don't truly “owns” the domain name off of which my e-mail address hangs, either, but it's a much better situation than phone number ownership. There, at least, the constraints of ownership (rental) are clearly defined, as is my ability to move it between registrars.

Yeah, that's pretty much my thinking. Phone numbers are extremely brittle. And I didn't even think of changing countries, which might be in my future.

My phone number is tied to a geographical location. (Sure, I can keep it alive, but now I am collecting phone numbers as I move around).

My email represents me, and I fully own it with minimal upkeep costs.

I'm not sure I understand you. You ask me why I see a difference and then list reasons a phone number sucks?

It sounded like you saw phone numbers as something to protect, keep private. I was asking from that frame of reference, and then pointed out how non-private they are today.

Yes I think they suck, but do they suck more or less than en email? Not clear.

They're working on being able to use it without a phone number, though doing so in a privacy-friendly way is forcing them to overcome some challenges first: https://twitter.com/moxie/status/1281353119369097217

Phone numbers are an awful identifier.

I don't see why you're responding that to a comment about allowing non-phone number identifiers?

Maybe look into Status?


I'm a core contributor at Status, and the team I'm on is currently developing a desktop client[1] to complement the mobile client. We have alpha builds for Linux and macOS, and soon Windows.

Status uses the Waku[2] protocol.

[1] https://github.com/status-im/nim-status-client/releases

[2] https://specs.vac.dev/specs/waku/waku.html

All I want is messenger. Status seems to do way too much. I don't want a wallet, a browser or anything else (not even to mention that I don't know a single person on macos and barely anyone on iOS)

Sure, maybe it's not for you.

One thing: I don't think it's unusual for a user to have little/no interest in the wallet or dapp browser. In that case they simply use it as a chat app that's focused on privacy and security while ignoring the other features.

I haven't had an account on Facebook or the like for years, and I started using Signal when it came out. I've been evangelizing it, and I never used Messenger or Whatsapp or any of those services.

The uptake? Like ~30% of the people I'd like to contact bother with it. The rest, it's even worse - we get stuck with SMS. And a lot of conversations I just miss, period, because they're groups of people in one of those other ecosystems (and missing those is a price I'm personally OK with paying, but not everybody will be).

The situation is just really bad. Signal is a good baby step forward, but it's not really the answer for most people, because they're sticky to wherever their network graph of contacts already is, and they're not going to switch just because the one weird nerd in their life isn't available there.

We need to turn to propaganda. If you support tracking and insecure communication, think of your children who will get burnt by insurance companies having access to their data. Thousands of [company] employees will look at your nudes if you share it on there. They might look at your kid's photos that you don't want them to. The republicans and democrats (depending who they support, switch) will look at your kid's pictures. The FBI will look at it. The cops will have access to it and then will target your kids in future. Why are you ruining yours and my kids future?

are you OK with THAT? You support children harassment by cops?

Throw some biased media page calling big companies bad. Not a hard thing to do today or write an opinion piece on popular news journal site and reference it. Medium works too.

We also need to cancel people for supporting these platforms.

Always a good sign when guilt is used to coax someone into using a product. "Don't you want the best for your family?"

I've been converting people to signal for years, I've got everyone in my life to switch except for my parents and 2 coworkers.

I gave up after realizing that Signal will not deliver message for a couple of days and that message notification doesn't work. Why wasting personal credit on something that is clearly inferior from the usability point of view.

And after it started insisting on creating a PIN each time I start Signal, I finally deleted it from my phone.

You can forget about security if you can't get a seamless user experience.

Oh God, the PIN creation is so annoying. Whatever possessed them that that was a good idea? Note that even after creating the PIN the app doesn't quit nagging you, it 'helpfully' continues to remind you to re-type it so you don't forget.

Plus, it's just a terrible UX for a SMS client. I really wanted to like it, but it's just too obnoxious to use. I've certainly not recommended it, I've had better luck switching relatives to Linux mint (seriously, I am not exaggerating). Creating a group MMS is annoying, and there's a nag screen if you try to call someone from a message within the app. I understand that most of these problems are due to the secure nature of the app, but trying to use it as your full-time SMS client is just too much of a pain. I'll likely still keep it installed, as the 1 thing it does do is make it really easy to transfer files/send messages between devices (send messages to yourself, then download them using the client's for other OS's).

Since it's open-source and on GitHub, you can go there and raise an issue with this feedback. You can't do that with WhatsApp or other closed-source alternatives.

Personally, I don't feel the UX is "terrible", I can use it to do most of the things that I could do in WhatsApp. My friends and I also tested out the video call feature and found the quality to be the same, if not better.

> I gave up after realizing that Signal will not deliver message for a couple of days and that message notification doesn't work.

That has never happened to me.

It has happened as recently as last month with a relative of mine. On that phone, the messages would get delivered only when Signal was manually opened; if Signal was not the foreground application, messages for it would not get delivered (but WhatsApp delivered messages correctly on the same phone). It's as if the Signal server had lost the association between the Signal account and the GCM token used to deliver the notifications.

I think it used to be the case back in 2015 or so, I believe they meanwhile fixed the issue. For sure it hasn't happened to me for a couple of years now.

Ive done that too. But I've experienced a lot of people removing it again.

The result is uncomfortable, since Signal never knows the other side removed it, so messages land in thin air. Several times did I have awkward moments when someone was angry "I thought you'd let me know if it was cancelled" - "I did - I sent you a text about that two days ago! On Signal!"

To switch, or to use it? Many people I know contact me through Signal, but their primary means of communication still definitely is WhatsApp, and they'll have to consciously remember to open their install of Signal to contact me.

> Convert all active whatsapp contacts to signal? 80%? 50%?

They can start by removing the atrocious phone number verification first.

This is exactly why I never got further than their login/signup.

Being forced to give them one of my 'most unique identifiers' (my phone number) was an absolute 'no'.

What about the Matrix platform? I personally prefer something that doesn't require a phone number as an ID.

There are some good-looking and user-friendly clients


The only way to do this is to build an integration that makes the switch DEAD SIMPLE. If it involves someone being highly motivated to switch over themselves without any help its not going to gain steam.

If I'm gonna make it difficult to get in contact with me, I think I'd rather make them use a federated system rather than one platform.

I switched to my own xmpp server instead. Never going back.

While I definitely disagree with the way drive doesn't give access to users for their own data, I think giving access has it's own problems, as you might land in another Cambridge analytica type situation.

1. Shady addictive game requires full access to drive.

2. Shady game downloads all your chats with all contacts

3. FB is on fire since they allowed access to people's chats with others, even those who didn't download the game.

4. People get mad on FB/Google for not protecting their data.

The same list you gave already applies to data already readable on Google Drive - most people have data that can be used against them on those drives. Also, many apps ask for (And get) permission to read contacts and messages on your phone.

Claiming that this is the reason is sort of like rearranging the chairs on the deck of the titanic when you can see the iceberg.

Much more likely it is to stop import/export to a competing app. The fact they will let you keep a local encrypted copy, but not an unencrypted one under any circumstances, is telling.

The number of people who use whatsapp is an order of magnitude or more than the number of people who consciously use Google drive, as in people who know google drive exists. Even the people who have google drive, most of them have much less intimate data on it than in their private messages on WhatsApp.

If we are to take the position that all users are tech literate and should be fully in charge of how their data is used, then even Cambridge analytica wasn't bad. People explicitly gave it access to data that was accessible to them. However, the uproar proves otherwise. People don't like it when they click yes yes and some random app has access to their social media data or messages. This is why Gmail also started requiring independent audit for all gmail apps.

And you are still avoiding the equivalence that everything you describe is already happening with sms messages at least, and generally much more. None of your arguments carry any weight especially since they don’t explain why it’s ok for data to be unencrypted on your google drive in the first place.

That wasn't the only argument though. If the data was encrypted with a facebook held private key, the user would still not be able to see it. If it was E2E encrypted with the only key being on the device, people will get mad when they reset their phone and get a new one, only to find their messages are gone.

SMS had the same problems forever, but isn't owned by a company, thus you can't blame xyz company if random apps access your SMS. No company got ever investigated by multiple nation states over that.

You still haven't addressed my main question though, how will a random app scraping your WhatsApp be any less of a scandal than Cambridge Analytica?

The same way random apps scraping your sms, a common occurrence, aren’t. The sms app is by google. The WhatsApp app is by Facebook. What’s the difference?

That’s twice in as many months. They may need some better QA here although to be fair it does seem mostly people with an older SDK version based on the comments. But still. I guess they need to cotton onto whatever server side change happened and roll it back.

At least this time you can disable the SDK (at least one app developer has a kill switch for the SDK) unlike the last issue that happened during code initialisation you couldn’t skip.

> I guess they need to cotton onto whatever server side change happened and roll it back.

"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." -- Leslie Lamport, 1987 (https://lamport.azurewebsites.net/pubs/distributed-system.tx...)

> to be fair it does seem mostly people with an older SDK version based on the comments.

No, I don’t think there’s any “to be fair” in that, unless those SDKs have been officially EOL’d. Even then it shouldn't take down the whole app, Facebook should just break.

That this only happens with some SDKs certainly makes the problem less severe than last time, but only relatively.

> They may need some better QA

There's no such thing as QA at Facebook. Really. Developers are supposedly responsible for all testing of their own code, but all of the pressure is to "move fast" so that goes about as well as you'd expect.

There is, it just depends on the part of the company you're in.

If you have a team run by idiots, then you won't have QA.

If you have a team run by adults, then there is plenty of QA. I mean its total shite that this wasn't caught. I'd expect much better, especially as this is the second time in n months. But then perhaps working for a company run by a dick who can't listen is getting to people.

I work there (in infra). I have yet to interact with a team that has dedicated QA. Checked the internal jobs board; no relevant term is listed among titles, areas of work, talent profiles, or skills, anywhere in the company including WhatsApp or Oculus. This is consistent with the "developers are fully responsible for their own code" ethos we're all taught in bootcamp - same reason we all get to do oncall. I'm personally familiar with many real adults up to senior director level, so "if you have a team run by adults" is definitely not true. Given your characterization of MZ (whether or not it's accurate), I'd say you clearly don't have any actual knowledge of the situation and shouldn't be presenting your guesses as fact.

>Checked the internal jobs board; no relevant term is listed among titles

Engineer in test. QA engineering lead. (https://www.facebook.com/careers/jobs/?q=QA) You must have leet scuba skills with searching skills like that.

AR/VR, Oculus, portal and workplace all have dedicated QA engineering teams. I'm sure there are more.

> Given your characterization of MZ

if it was only zuck. Zuck has two faults: he's not able to articulate any single thought inside of 15 minutes and he consistently throws out integrity products that would have stopped bad PR.

Its not just me, you should look at the micropulse. confidence in leadership has taken a good battering recently.

The whole juneteenth buisness, we had an entire _day_ of people telling us the problem. Senior management then sat next to them and said in reverent tones: "thank you so much [[dabs a tear]] for telling us your lived experience" posts a selfie and then does fuck all else.

Or, having revved up the company to generate hundreds of ideas to tackle abuse on FB platform, do shit all with them, not even acknowledge they exist.

> You must have leet scuba skills with searching skills like that.

Nothing to do with scuba, and if you did any operational work you'd know that it's in the category of tools that are often flaky due to lack of QA. It's also worth noting that the link you provide only shows 41 listings, and most of those merely mention "familiarity with QA" in some boilerplate for what's obviously a developer position. Leet reading skills you have there. Percentage-wise, there are fewer true QA positions there than we have E-10s. For a company that hires 10K a year, that's indistinguishable from zero.

> AR/VR, Oculus, portal and workplace

In other words some small percentage of the company, none of it near me. All of the systems that the profitable parts of Facebook rely on - to provision systems, monitor systems, store data, move data around, analyze data, etc. - rely entirely on developer testing. So here's an amended statement to make you happy.

"99.9% of Facebook has no QA, and the rest has really bad QA."

Thats not what you said originally was it?

just over 10% of job openings have something to do with testing or QA. Of course QA is similar to software engineering, the whole point is to automate testing.

> [AR/VR, Oculus, portal and workplace..][..is] some small percentage of the company

yeah, go look at the headcount.

> none of it near me.

Now we are talking.

> rely entirely on developer testing

again, that wasn't the distinction I was making. It doesn't matter who's testing, so long as it gets done properly. Considering PE is a blended role. Its not a hard push to think that testing might be part of the role of a modern developer. Mind you, so is writing documentation, but thats not a priority either.

Many of the ads teams have dedicated QA, too.

I’ve had the same basic bug crop up 3x in the last two years while using the API - posts to Groups with dollar amounts just… fail. “An unknown error occurred.”

The first two times it took months to fix. It’s happening again currently, too.

Clear there are GAPING holes in their test suites.

Looks like it's going pretty well, Facebook is currently worth 700 billion dollars. Or is that not what you meant?

Close a great venture capital round, suddenly you've got a big pile of money in the bank and your company's valuation is really high. That means you're successful!

Make a lot of money selling lifetime subscriptions, then have nowhere to go for cash next year: look at all that money! I'm very successful now!

If all you're measuring and optimizing for is "valuation" or "revenue" you're not automatically going to get a good product or happy customers...

There are multiple dimensions you can use to measure success. Market capitalization is certainly one of them, but it is at best only tenuously related to engineering quality.

Workaround for users: Enable airplane mode and disable WiFi, or block facebook domains

Workaround for developers: Go to Facebook Developer page > Analytics and disable automatic Events.[0] Apparently this doesn't solve the issue for everyone.

Issue in Facebook's tracker: https://developers.facebook.com/status/issues/17391881029111...

[0] https://github.com/facebook/facebook-ios-sdk/issues/1427#iss...

Better workaround for everyone: Delete the Facebook SDK from your app and start caring about your users' privacy (and app stability).

That's not a workaround, that's final solution

Thank you now I can listen to cached Spotify playlists!

Do you know which domains one would need to block?

WhatsApp can work without these, but maybe a smaller subset would suffice. NextDNS[0] for iOS can be used for the blocking.

[0] https://nextdns.io/

Excellent, I was looking for this, thank you! Spotify launched on my iPhone immediately after I added these domains to my NextDNS denylist.

What if I want to keep using Messages app (I know I know, but I can’t avoid it)

connect.facebook.net graph.faceboom.com sdk.facebook.com

My personal anecdote: I’m parked over on the side of the road because my carplay habits of Spotify and Waze both kept crashing to the dashboard.

I couldn’t help but notice neither app worked even after a reboot, yet google maps did. I was thinking there was a carplay bug and then thought: “oh boy here we go, what are the odds, let’s check hackernews” and bam, of course it’s the Facebook sdk again

I’m thankful that googles iOS-chrome and google maps doesn’t use the Facebook sdk for something...

They never responded to my request for a postmortem on the previous blunder: https://github.com/facebook/facebook-ios-sdk/issues/1385.

Anyone reading this, please make some noise that request for postmortem. https://github.com/facebook/facebook-ios-sdk/issues/1385

I want a central document with clear steps on how to mitigate. Currently the best option for documentation seems to be wading through comment threads, which is very frustrating.

Maybe if Facebook were actually required to provide post-mortems, the developers would be more cautious about creating these outages in the first place. One thing's for certain, the silence is infuriating.

Being "required" to provide post-mortems doesn't always mean they will actually do it.

Stop using facebook's garbage!

Or if you must, use their oauth flow and API but _don't_ include code of theirs you don't control directly in your binary. It's just asking for trouble.


It's unfortunately against Facebook's Developer Policy (if you're offering "Sign In with Facebook" functionality). To use that capability you are required to use the official SDK! I filed a (probably futile) issue with them here: https://github.com/facebook/facebook-ios-sdk/issues/1437

And then FB will ban your app.

Ban it from where? Using their API?

Yes, Facebook requires that any mobile app using Facebook login do so via their SDK. Presumably they will disable your Facebook app (breaking login) if you don’t comply.

They told us that explicitly. We had been using the URL-based login on the app (i.e. open a browser with the URL and use the redirect to continue logging in), for almost two years. Then about a year ago, they e-mailed us just before a holiday and told us we had less than a week to move all of our apps to SDK otherwise they'd just disable us. Their argument was that "it's not providing the best possible experience to our users". There hadn't been a single complaint about it though.

We had nearly 100,000 users logged into our app via Facebook login so we were basically strongarmed into complying. We somehow pulled it off before the deadline for both Android and iOS but felt really dirty afterwards.

The above is correct. Facebook wants you to use the SDK.

How is it that a third party SDK is consistently causing apps to completely crash rather than, for example, displaying an error message? Is this a failure that cannot be caught? Sorry, just trying to understand. The dependency relationship just seems to be extreme.

>To stop crashes from the Facebook SDK, some devs tried commenting out any code that calls Facebook. Nothing worked.

>It turns out that by just including the SDK with your app, Facebook runs hidden code on launch. (FBSDKApplicationDelegate.m)

[0] https://twitter.com/sandofsky/status/1258288056399847425

... so if the app including the SDK has permission to access location data, the SDK could be passing that location data to Facebook?

Not only could it do that, but it also totally does do that

Didn't they also use to home in information about surrounding bluetooth devices? When iOS 13 beta dropped, half of the apps suddenly asked for bluetooth permissions because Google/FB SDKs attempted to use it on app launch.

Not sure about Bluetooth devices, but the Facebook SDK definitely phones home with some system data and generates an unique identifier on first launch which is then included every further interaction the SDK has. Both of these enable fingerprinting and persistent tracking.


Why are people using the facebook sdk? What does it give them?

When we looked at it, we saw several reasons, all of which are dubious at best:

1. It is required for a "login with Facebook" feature, apparently. I'd presume this goes for other feautures that integrate with facebook, such as "invite from your friends" or "share on facebook(messenger)".

2. It gives access to metrics, when added to your own, greatly enrich your own data collection.

3. It is required when working with Facebook ads - though I would not know how this ties in with mobile apps: on web it is clearer (where you are required to host "the pixel" on your website in order to have ads on facebook link to that website.)

4. It is feature rich, so as a pure SDK, it offers neat utilities; though for each of these, there are better alternatives found in separate libs.

Extending point 2, it’s essential if you’re running ads on Facebook and want to know if clicks on the ad in Facebook bring the user through to the app - ie so you can be sure you got a real conversion

@harryf there’s a way to send ads attribution via server API, so no need to have client side FB SDK for that

I'm using it for realtime analytics (dashboard like mixpanel) also this brings authentication and other FB Integration

So yes the definition of “freedom” is “give the government more power over both app developers who voluntarily integrate the SDK and users who voluntarily download the app”.

> voluntarily

I don't remember apps listing the SDKs they use. As a humble user, how do I voluntarily choose whether to download an app or not based on their usage of Facebook's garbage?

Edit: You can't have it both ways. You can't remove a bunch of power and information from the user in the name of "safety" and "UX" and then claim that users are making a bunch of choices they can't possibly be informed about "voluntarily"

In that case, pass laws forcing disclosure and you still let the users decide.


My guess in this particular instance is that it seems like the Facebook SDK is fetching some sort of remote serialized configuration, calling “count” on what it expects to be an array, and instead is calling it on NSNull (an object representing “null”) which is throwing an exception.

There are some intricacies on iOS surrounding null values and serialization, and as a developer it is important to understand that you may encounter NSNull. As a standard practice in my company, we type check every remote value before calling anything on it. Seems like that would have prevented this issue.

This is exactly what happens. Basically a null value is converted into an NSNull object and calling count on it will of course throw an exception... because there is no count method in that class...

It makes me wonder what all those engineers at FB are actually doing... ? Every time I tried to integrate or look into the FB SDK for simple functionality it was a total clusterfuck.

That's amateur hour stuff. Every developer in pretty much every language or platform learns VERY quickly what happens when you don't do a null check.

NSNull is not the null you think

It may not be a conventional null, but I'm not aware of any tech where calling "count" on a Null, .nil? or otherwise null-like construct won't end horribly.

The Facebook SDK is written in obj-c, where calling `count` on `nil` is perfectly valid and returns 0. It only doesn't work on NSNull, which is a weird different thing that does not work like the language's built-in null.

Sounds like a good reason to stop trying.

A bit of snark but also maybe stop trying to use a data collection engine to leverage anything for yourself?

Could also be a string that’s no longer there

Source: have had that issue before

Also maybe it’s time for them to start rewriting things in Swift, where this would be much less likely to happen

True enough. Swift Codable with optional properties handles this automatically.

I believe last time this happened it was because they were running code (including network requests) in a callback that gets called when they were loaded by the dynamic linker.

I'm curious about this as well. I've always written frameworks to be used with a 'Policy Layer -> Mechanism Layer -> Utility Layer' general architecture. Each layer captures and does its best to handle errors in a way you would expect that layer to. The only time errors move up towards the policy layer is explicitly because the next layer up will handle it.

Well SDKs may do their own initialization upon being loaded into the app which might be the cause here as the crash occurs on launch.

People are using third party code without fully understanding what they’re bringing in or the consequences of bringing it in.

Given how protective Apple is about the experience of apps on the App Store and this happening the second time, I'm surprised they have not come up with a way to prevent this from happening. Also, a monolithic SDK should not exist in this day and age.

Indeed, and if the FB SDK (as used by the Spotify app among others) does in fact call home, it might violate Apple's TOS and give rise to rejecting those apps on the App Store. I'm not up to date with Apple's policies, but wasn't Apple specifically rejecting apps that seek to establish a platform within Apple's platform? Now if Apple are going to play hardball on this one is another question alltogether.

A quick solution on Apple's part could be to add an easy to use domain blocker as part of there tracking blockers for Safari. Could be buried in the safari settings pages, but with a toggle to apply at an OS level.

It's a money game and Apple could totally crack down it, but they won't. They may long term but they take cowardly slow steps like everyone else does.

So exactly how much money does Apple make from free apps?

If Apple wanted to make money , it should make it harder for advertising supported apps to make money and force apps to either not exist or get people to pay for them.

I would be okay with that.

I guess Apple will have to weigh whether they'll want to tolerate the Fb app on iOS, but more probably they have a deal with Fb (else they would've kicked them out already). Considering Fb is said to be the main portal for "the web" as a Fb user might see it (which I find still very hard to believe for a number of reasons), if the Fb app (or worse, WhatsApp) isn't on iOS, with Safari and FF already blocking most tracking on facebook.com, Fb might block Apple devices altogether, and Apple might or might not stand to loose in device sales, depending on the intersection of Apple's with Fb's target demographic. If it were only for Spotify (which is a direct competitor to Apple Music), they'll have to weigh whether them kicking Spotify out will be seen as anti-competitive vs having a good excuse for eliminating a competitor on their own platform. But maybe my speculations are way off, and Apple will just tell Fb to fix their shit.

I would think that Spotify would want to remove the SDK. It doesn’t need the marketing tie ins anymore and it doesn’t use FB for advertising.

Libs over SDK!



The most popular apps on one of the most popular operating systems just plainly don't even open.

What has this industry become? How are we so goddamn inept at writing software? This is an industry where we can automate testing - not many other industries have this capability - and we still don't have a simple test regime of "check if the app opens" for any of those apps. Somehow, at the most valuable software companies of the world, nobody has set up a system that makes sure that you can't introduce a change that renders these apps unusable?


I have always been afraid of software engineering becoming a trade similar to medicine or law or (regular) engineering where significant barriers will be put up before you can enter the industry, but this sort of shit makes me feel like it could be for the better. I will have to go and get a degree, but that's the price I have to pay I guess.


Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized.


Years ago, I had one of the employee builds on my company phone, and it picked up an update and would "instacrash" - tap the icon, and it'd die immediately. Crash. Crash. Crash.

A couple of days later I caught up with the people allegedly responsible for this stuff in the bigger review meeting. "Shouldn't you open a SEV because you are shipping stuff that can't possibly work and aren't catching it pre-release?" The response was a lot of hand-waving and stupid manager-speak that boiled down to "no, go away".

I basically wrote them off at that point. If they could act that way and get away with it by way of implicit acceptance from upper management, that's just the way it was going to be.

I see the quality is as high as always.

Yep at such a large company there is such huge variance in quality. Wish you and many others were still here to help fight against this mindset. There is lots of high quality work done here, but it doesn’t matter if we end up with issues like this.

Curiously, this type of handwavy response reveals a not-quite-1:1 mapping between the sort of assertive, pro-admission, fail-forward balance that "move fast, break things" implies for me, and the apparent reality which seems closer to "haha crowdsourced QC go brrrrt"


The apps are broken serverside by Facebook. No amount of testing (automated or not) on the side of the app developers would have prevented this.

Facebook's own apps did not break since they are on the latest version of FBSDK.

What Facebook needs to do is to test whether their changes don't break other people's apps.

1. Server-side APIs can be versioned. If an old version is somehow changed, I would call it plain malpractice and in some other industries your license to practice could be stripped.

2. Client-side SDKs can be designed in a way that even if the server-side changes (which it shouldn't), it does not cause the whole application to crash. This can of course be hard depending at which level the SDK operates and how integrated it is into the app.

3. Are there no contracts between Google/Spotify/etc and Facebook? If I were FB and there were contracts on the line, I would make sure that there is no way something like this can slip through (automated testing).

> What Facebook needs to do is to test whether their changes don't break other people's apps.

I'd argue this is at the very core of why you test. I mean this problem is as simple as (n+1).m vs n.(m+1) in terms of versioning vs revisioning.

If that is not being tested then I'd say Facebook isn't actually testing at all - at least not in a practically relevant sense ...

... despite the shit-load of YouTube videos from FB devs lecturing about CI/CD and what not.

Sounds like you have it backwards. They use other people's apps to test whether their changes break Facebook.

It is not acceptable to have your client crash due to server-side stuff going on. When the server is faulty the client must handle that gracefully, like disabling the functionality or displaying a message about what is wrong if no further use is possible due to the error.

I disagree. While, yes, Facebook needs to test their SDK, app developers need to build resiliency into their apps to ensure third party dependencies don’t break their app. The developer adopted it into their app; it’s their responsibility to do the proper checks in cases like these where Facebook broke things on their end.

This is a native SDK that is crashing the process. There is nothing an app developer can responsibly do to prevent or recover from this crash.

Other than remove the SDK, of course.

Yes there is: a remote kill switch that prevents the third party SDK from loading. That puts you back in control of the software you’re shipping.

Fool me once, and all that.

You have to be technically competent enough to know how to dynamically load a framework, though.

And most SDKs are provided as static frameworks. You have to be technically competent to know how to wrap a static framework as dynamic, and then use dlopen to load them properly with a rather high one-time runtime cost. A mess.

For a tech company with a multi-billion dollar valuation this shouldn't be insurmountable

Facebook knows how dynamic loading works. (At least, I think so…) However, their many clients may not, as a lot are not multi-billion dollar companies.

So there is no way to try {} catch {} it, right?

Not in this case, as it's not code explicitly called by the client, but automatically loaded on app launch...

Yes, but 1. it’s hard to inject that as the crash happens very early at launch and 2. Objective-C exceptions of this type are not intended to be caught.

That's not strictly true, and anyway it's different for the app developer versus a library provider. If you're making a 3rd party SDK that can throw, you need to make it can catch those throws.

NSInvalidArgumentException is not intended to be caught. You can, of course, but receiving one indicates a serious run-time error.

If it's an objective C exception, you can catch those with an objective c wrapper function. Ive done that before in my Swift projects.

> Other than remove the SDK, of course.

Bingo! App developer voluntarily pulled in a dependency which they didn't fully understand. When you bring in a dependency, it's not still someone else's code/problem, it becomes your problem.

Ah yes.

If hacker news goes down while you are writing this comment, it is your responsibility to ensure you can still post your comment despite the service breakage.

Unless your app is designed for browsing Facebook, there's no reason that anything on Facebook's servers should affect if your app works.

Many of the apps crashing are using Facebook for a portion of their app, such as one login option.

Letting that take down the whole app is insufficiently defensive coding, whether in the app, sdk, or glue between them.

> there's no reason that anything on Facebook's servers should affect if your app works.

Absolutely, but the problem is that in order for you to use these features, you're supposed to use the official Facebook SDK. The additional problem, is that Facebook has no interest in defensive coding, because if you're not interacting with Facebook in an app, there's no benefit to Facebook for you using that app at all.

How do you defend against abort() in library code?

Don't include libraries which contain such code?

No abort here, the crash is coming from the runtime, which you’re not opting out of if you want your app to work.

Then we can agree the FB sdk should have been better written to avoid this, right?

Obviously, and now we agree!

So how could the app developers have prevented this, other then not using the SDK? (I'm not advocating for using the SDK.)

Would disabling SDK autoinit have fixed this? That was a suggestion raised back in March. Seems like a net win to avoid static initializers in the first place.

(Am not a mobile dev, but it wasn't exactly hard to find that, either).

Maybe (although some people report that not working), but not necessarily, as "autoinit" is just an if case checked in the same auto executed code in the Facebook SDK, which could still crash before the flag is checked.

Are you telling me that Facebook Inc. has no capacity or ability to test their own, one of the most popular SDKs in the world, in the wild? Like have a single phone connected by USB and open spotify app and simply login after they make a backend change?

By app developers I mean the consumers of FBSDK, not Facebook.

The truth is, 1st-party apps are well-supported but 3rd-party SDKs provided by these companies, especially the ones don't generate revenues (Facebook SDK, and many others) are simply not well-supported within the organization. They are not dogfooded, they don't develop with the latest and greatest tools available for 1st-party apps, some of them have very spotty CI, and often their release schedule is not as rigorous (many 1st party apps adopted weekly release schedule).

I would disagree that the 3rd-party SDKs don't make revenue for Facebook. They likely provide additional targeting and behavioral data that feeds into their giant ad machine and lookalike audiences.

That is a long chain. Internal promo doesn't work that way. That's why it is easier to get promoted when you work directly on ads.

Maybe don't include the Facebook SDK (Facebook Spyware Development Kit) in your tier-1 app?

1. Mock dependency

2. Throw exceptions

Don't tell me "No amount of testing" just because you aren't responsible for your own dependencies.

You can't mock it, that is the entire point. Simply loading the shared library of the SDK will execute the broken code.

90% of people in this thread think “native code” is a react plugin

I don't, but I'm fairly sure there is a solution and others just don't want to try hard enough

Can validate, no reasonable way.

Removed the SDK myself from a top 100 App Store app.

So an app (which you control), is told to load a shared library, and there's no way to handle that? Even by delaying the loading to only when it'll be used?

It's not "is told", it is "it is writen in the binary itself that the kernel/libc runtime should load the shared library".

To be fair, you don't have to do it that way. (Library loading is done by dyld, FWIW.)

This is the second time.

Recently tested an enterprise marketing app with an SDK ... and we tested if the endpoint wasn't available. Nothing happened -- it worked just fine. Failed quietly and gracefully -- the user would never notice.

"We" know how to do this.

Facebook hasn't done anything to fix this problem since it happened a few weeks ago.

But it's Facebook -- I hope they keep it up. Can't wait for them to AOL themselves.

The problem is not that the endpoint is not available, rather that the endpoint is returning garbage on which the parser chokes. Of course you should always write parsing to be 100% resilient against any kind of garbage input (even malicious), but my point is that your marketing SDK could still be a ticking timebomb.

True, yes. But this is for a financial services mobile app... one of the first concerns was the user experience. The org is well versed in the other back-end security and data issues that also have to be looked at.

But if you're Facebook, and you're pushing your SDK as a great solution and it's embedded in high profile apps and you are letting this happen a couple of times, I get to make fun of you. (a dollop of sarcasm, to be sure..)

>I have always been afraid of software engineering becoming a trade similar to medicine or law or (regular) engineering where significant barriers will be put up before you can enter the industry, but this sort of shit makes me feel like it could be for the better. I will have to go and get a degree, but that's the price I have to pay I guess.

I can almost guarantee that the programmer who wrote this had a degree and passed a very rigorous testing process to get their job at Facebook. Doesn't mean anything when the company tells them to cut corners to iterate faster.

Stop blaming engineers for the mistakes of management.

Yeah it's not like most programmers wouldn't prefer to be writing good software but the "let the user test it" cycle of quick turnaround sprint culture is always going to have this issue.

There is more complexity in this.

Proximate Issues

The app makers cannot easily prevent the crash since it is happening in Facebook provided code. Perhaps the only thing they could do is to code in kill switches. Another possibility would be to modify the Facebook code. That introduces maintainability issues.

The crash happened because of a server side change that triggered roughly the Objective-C equivalent of a null pointer exception.

Objective-C is an old unsafe language.

Ultimate Issues

Client-side libraries have a poor history in the industry. They are often written by server engineers who aren’t expert on the platform.

For instance, if it was really a static load method that called the server, then this was a poor design decision.

Furthermore, weaker server engineers are notorious for breaking backward compatibility. They don’t design nor build in tests to ensure backward compatibility. The backward compatibility must include backward data compatibility.

The above issues are indicative of organizational issues.

Facebook is a little bit of an outlier. It was notorious for badly written client libraries stretching back to the Facebook platform days. The joke was that their libraries may have been written by interns even though so many companies were dependent on them.

The Facebook libraries and the ilk are used for sign on through Facebook and ad attribution. The trade off an app maker needs to make it a whether to run ads on Facebook and whether to support sign in.

Facebook is not the only big company with these issues. Google’s equivalent also crashed albeit at a much lower rate due to a much more convoluted issue.

Apple introduced the Swift programming language to address the deficiencies in Objective-C. That these companies are running into these issues makes me wonder if they never transitioned to Swift. They are certainly way behind the latest version of the libraries.

To summarize, it is a cascade of issues some technical and some organizational rather than a single coding error. These issues span companies.

> The crash happened because of a server side change that triggered roughly the Objective-C equivalent of a null pointer exception.

> Objective-C is an old unsafe language.

This was a safe crash, not an unsafe one. Swift can and would crash the same way.

You can’t safely call a method on a nil value in Swift. You need to explicitly force that in your code with an exclamation point to assume that the variable is not nil.

Looks like the server passed empty value in JSON which got converted to NSNull in Objective-C. It's a common mistake to assume null is returned instead of NSNull.

The Swift equivalent of the JSON library should return nil which you then need to explicitly check for non nilness before using unless you use the unsafe exclamation pint.

  $ swift
  Welcome to Apple Swift version 5.3 (swiftlang-1200.0.16.15 clang-1200.0.22.41).
  Type :help for assistance.
    1> import Foundation
    2> // IIRC this used to throw an exception.
    3> // Now a respondsToSelector: check is silently inserted and we get a surprise nil.
    4> (Int?.none as AnyObject).count + "call Swift's bluff".count
  Fatal error: Unexpectedly found nil while unwrapping an Optional value: file repl.swift, line 4
  2020-07-10 18:03:43.815959-0700 repl_swift[73029:930570] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file repl.swift, line 4
  Execution interrupted. Enter code to recover and continue.
  Enter LLDB commands to investigate (type :help for assistance.)

You need to call Swift's bluff with AnyObject. That's unsafe and was only necessary to bridge to Objective-C where you can call any method on any object.

I think we are in general agreement?

When you write an iOS app you're constantly interacting with Objective-C. The moment you're using JSONSerialization–and unless you're shipping your own JSON parser, you're not avoiding it–you're getting back a nice Any that you are almost certainly going to work with, possibly implicitly, using AnyObject. My point was that switching the SDK to use Swift would not help fix this issue, and I think I've shown that you'd run into similar issues even without the use of the force unwrap operator as you implied previously.

Fair enough.

Google has recently pushed a payload that crashed any app using Google Maps SDK https://issuetracker.google.com/issues/154855417

This one was a good one because the app cache was poisoned so the only solution was to ask customers to reinstall the app (after Google deployed their fix, which took a while) or upload an update that nuked the SDK cache on startup.

>>> I have always been afraid of software engineering becoming a trade similar to medicine or law or (regular) engineering where significant barriers will be put up before you can enter the industry, but this sort of shit makes me feel like it could be for the better. I will have to go and get a degree, but that's the price I have to pay I guess.

This would still happen because A) the programmers that made this mistake probably have the knowledge to pass any certification exam you can throw at them, B) most of the time we are not just building something that has exploded 1000 times before, we are creating new stuff.

The engineering process was the issue here, and sadly the only way to prevent this is not to make personal barriers of entry higher but to slow down the process with more QA, testing or whatever you can throw at it. In other words, make software more expensive. And keep in mind that this would only give better numbers but it would not solve the issue either (see Boeing 737 MAX).

> A) the programmers that made this mistake probably have the knowledge to pass any certification exam you can throw at them

Very unlikely. You probably don't realize how tough some professional exams are in some fields. We are talking dedicated months of study here.

> B) most of the time we are not just building something that has exploded 1000 times before, we are creating new stuff

Most software out there is just CRUD stuff, not really new stuff...

>How are we so goddamn inept at writing software?

Wow who would have thought no standards, free access to learning, fake-ass meritocracy, and exorbitant profits and market control would have led to such a thing

Pick a standard, whether it's a degree program, a cert, or apprenticeships. Hurry up! Just give us a solid target to aim for.

The fact that so many popular apps have to pull in a full-blown SDK from Facebook to get support for even a subset of its functionality is insane to me, too. That 'log in with Facebook' button on your app comes with an entire suite of stuff that Facebook can use to slurp up any data it wants.

> I have always been afraid of software engineering becoming a trade similar to medicine or law or (regular) engineering where significant barriers will be put up before you can enter the industry, but this sort of shit makes me feel like it could be for the better.

There's a good chance that the one who wrote this bug had a degree.

Surgeries in medicine still get botched despite all of their degrees and barriers. Degrees don't produce good code, good developers produce good code.

There's no standardized process for producing a "good developer". Every company has a different process and different standards. I think that's a part of the problem and I agree that it's not one academia can solve, but I wonder if there could potentially be another way.

These histrionics are ridiculous. There has never been an era in software engineering where people didn't write bugs. There has never been an era where people didn't miss bugs. You can scream and moan at the top of your lungs all you want, but shaming and calling people incompetent does not contribute anything.

> I will have to go and get a degree

Was not aware that having a degree made you immune to writing bugs.

I try to avoid software in general as much as possible.

Bugs happen. Tests miss edge cases. Shit happens.

This will be a fart in the wind and lessons will be learned.

Maybe lessons will be learned, maybe not.

The industry seems to have very low standards. Let's let these mistakes happen over and over and let's just patch them as we go. No long-term thinking.

Someone should collect all these horrible incidents and start a university course on just plainly what NOT to do.

Industry has low standards because the users have low standards. If the application doesn't start, they are mildly annoyed and go do something else. Eventually the bug will get fixed and they use it again. This has been the case for like 95% of software written in the past decade.

Some of the very brightest software engineers in the world work at Facebook. As much as I would love to rip on Facebook, I also understand that integrated software development is hard, and I guarantee you that Facebook is "doing it better" than most.

Don't sit in an ivory tower and throw stones in glass houses. Can you guarantee that your own SDLC house is in order? Because it might be, right up until a sneaky edge case nips you in the butt, and then suddenly it isn't.

If an industry leader with "the brightest software engineers in the world" is unable to write their software to a higher than average standard, then I think I am in the very reasonable position to point it out and raise discussion about it. Don't place me in an ivory tower for that.

Why should we just sweep this under the rug and call it a day? There is something very important that can be learned from this. Apparently, FB themselves have not learned, because this is not the first time this has happened. Why was this allowed to happen more than once? Did Apple, Google, Spotify, Facebook etc not take any actions?

"Shit happens" is one way to look at it. Another way is to question whether we as software engineers are really doing our job properly.

Why are you so angry about Facebook, Apple, Google, and Spotify having a tiny outage? They are not life saving services and only cause a mild inconvenience when they fail which practically never happens for most people especially compared to "average software".

> FB themselves have not learned, because this is not the first time this has happened.

They have tens of thousands of engineers likely split into hundreds of different teams/microservices focussing on different parts of their software stack. A ton of them are new to it and nobody knows every part of the stack so shit can happen.

What is the biggest engineering organization size you have worked for and what was your uptime?

> Why are you so angry about Facebook, Apple, Google, and Spotify having a tiny outage?

The linked issue makes it very clear that the "tiny outage" affects literally every app that has the Facebook SDK merely linked, for a frankly dumb reason (static initialization) that they should have learned from the last time it happened.

It's not a SaaS. "Uptime" for a linked library is stupid.

Let's avoid lionizing/denigrating here.

They're using static initialization and it bit them in the ass again. They should stop using it because it is prone to things like this which result in apps not launching at all. I'd like to see them commit to stop using this method to initialize themselves, and take responsibility for the crashes. Then, IMO, we'd be good.

In a scenario like this, I'd hope they'd want to make this bug's recurrence impossible.

It was horrifying to see how many config changes you have to do to basically beg facebook not to send back data to their server. Most of these were hidden in various corners of the documentation and I still suspect there's crap being sent back without permission. I would love nothing more than to disable the SDK entirely but we have a few hundred thousand users logged into our app via Facebook.

Here is our Android config in case anyone's interested:

<meta-data android:name="com.facebook.sdk.AutoLogAppEventsEnabled" android:value="false"/>

        <meta-data android:name="com.facebook.sdk.AdvertiserIDCollectionEnabled"

        <meta-data android:name="com.facebook.sdk.AutoInitEnabled"

Isn't Facebook the place that forces engineers to write code in a open bullpen environment? How can high quality code be written in a madhouse?

Bright engineers sometimes do some very stupid, arrogant things.

The brightness at places like FB tends to be diverted towards making money, not ensuring software quality. (This is also true at places like Apple, just differently so.)

> Some of the very brightest software engineers in the world work at Facebook

Like they work in many other companies.

In any case, this is not about being bright but about being diligent.

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact