The important thing to note here is that these endpoints were scheduled for retirement at future dates; some in July 2018, some in Dec 2018. Today, with no prior announcement, these endpoints were abruptly retired.
Instagram had previously published a deprecation schedule for these endpoints, and threw that schedule out the window without notifying even Instagram Partners.
Edit to add: these endpoints are all related to the profile data and behaviors of non-oauthed users. This is the exact data that ML researchers (myself included) would normally use to analyze social graph behavior, interests, and audiences of Instagram users, and also is exactly the type of processing activity that GDPR restricts by requiring explicit and revocable consent. I see the abrupt retirement of these endpoints as a reaction to the pressure FB is facing as of late. This is a positive for personal privacy and will reduce the potential for abuse, however many types of legitimate and responsible businesses will be affected as well.
Good grief. When did we forget the meaning of "deprecate"? Deprecation and retirement are two drastically different things.
Their change log reads "deprecated immediately" which is silly, but should be read as "we don't know what 'deprecated' means and are retiring these APIs immediately."
We've been scrambling to get migrated to the Instagram Graph API, as we have a lot of paying businesses using our app to manage their Instagram presence, and that's now not possible with the 'deprecation' of the API.
What's worse is, we can't even migrate to the Instagram Graph API, as Facebook has:
1. Temporarily stopped app reviews, so there's no way for us to go live with our Instagram Permissions on FB Graph API.
2. Blocked access to the Instagram Graph API, even for our development and staging apps.
So they expect us to migrate to the new Graph API, but there's no way to do it since we can't even develop and test the API given that access is blocked.
It looks like Instagram is starting to feel the shockwaves of Facebook's issues. Here I was hoping that Instagram would not go down the same path as Facebook, but I guess that was only a matter of time. As a developer building and maintaining IG apps, it's extremely frustrating not just for me but also my clients who depend on apps we build to manage their Instagram.
This also comes on the heels of IG dialing down their API limits from ~5000 per hour to 200 per hour [0], which is ridiculously low. If my app has 20 IG accounts connected to it, each account can only hit 10 per hour. Crippling.
I always thought these rate limits often to be ridiculous. Same applies to Twitter -- they have quite a restrictive API with few opportunities to get the query rate increased.
Honestly, do they want people to build scrapers using sets of IP addresses?
Rate limits aside, Twitter's APIs are horrendously unreliable. Plenty of timeouts, 500s, and over-capacity errors. Are all these big tech companies spending way too much money on employees and not enough on their actual product, infrastructure and security of such product?
> Are all these big tech companies spending way too much money on employees and not enough on their actual product, infrastructure and security of such product?
No. They are allocating their money where their business is. The idea of "ecosystems" with 3rd party integrations has been shot down nearly everywhere - all the "big tech companies" have one thing in common: they want people to be locked in their environment as long as possible by any means neccessary and legal, as this is where the tasty ad revenue comes from.
A 3rd party Twitter client, for example, cannot show ads (and, so, generate no revenue for Twitter). Therefore, it is perfectly logical that Twitter makes their API as bad as possible for 3rd party clients (e.g. no polls, images in DMs, group DMs).
Correction: they want their content consumers to be as locked into their environment as possible.
Content producers, on the other hand, are the ones who are making your platform more attractive to your content consumers. You want to make it as easy as possible to put high-quality content on your platform, and a key component of ease of access / publishing is a powerful API.
On the upside, it's blatantly obvious there's a massive opportunity for new platforms with liberal APIs that stick to keeping them open. From my observation in the years since Twitter turned against their ecosystem, I think developers have grown increasingly desperate for this.
> I think developers have grown increasingly desperate for this.
Developers have. But unfortunately, most that the users (aka our customers) want is a place where they can share cat videos, have instant communication with friends and (distant) relatives and, depending on allegiance, Trump or Clinton memes in peace. There are only two things that can not be done on the social networks: gambling and porn. Everything other has been slowly absorbed, especially by Facebook.
Also: when you're working 10+ hours a day or more, you most often don't have time to maintain your own blog, much less a full blown web server. There's the reason why Instagram, FB and Twitter are so damn successful: they're essentially frictionless after the sign-up process.
The sandbox limits are what bother me. They make the API basically useless for side-project type experiments.
I really enjoy Instagram the app to post photos of my travels but as the years go by it's feeling more and more locked down. I'm not really sure what the alternatives are, Flickr?
> This is a positive for personal privacy and will reduce the potential for abuse, however many types of legitimate and responsible businesses will be affected as well.
I mean, sure. But every day these are open is another day that something dangerous could happen. Letting people use these APIs would only be responsible if there was a guarantee that everyone using the APIs was acting in good faith and could adequately defend against data breaches at Facebook's level, on their own. The latter was never really true and the former has been blown apart by the CA scandal, so good riddance.
I think it's kind of hilarious that the users of HN are outraged that FB and Instagram are even a thing, while simultaneously outraged that they're taking steps to close potential leaks of the personal information of their users.
> I think it's kind of hilarious that the users of HN are outraged that FB and Instagram are even a thing, while simultaneously outraged that they're taking steps to close potential leaks of the personal information of their users.
Yeah, it's almost as if HN was a plurality of diverse individuals with different needs and opinions and not a single entity with a single opinion on everything. Who would have thought?
Look, this is a cute response and all, but downvotes and upvotes expose the posts that the "group" (in this case, the larger HN community) tend to believe in the most. If 2 posts rise to the top, it suggests to me that there's an overlap in people that are upvoting those 2 posts.
I find it concerning that Facebook now has an excuse to lock in their data monopoly by effectively refusing to give up their users' data about themselves. For example, it appears it would now be impossible to build an IG alternative platform that piggybacks off of your existing follow graph. So switching costs to a competitor suddenly skyrocket when there's no possibility of an "import" feature.
So basically Facebook's data monopoly just got stronger. Am I missing something?
If they give their data to other people, people will complain. If they don't people will complain. It seems like no matter what they do someone will be mad.
There have been some third party services [1] that allow a data dump of your Instagram photos... which probably just lost their functionality in the API deprecation.
(Though I think they were really _just_ the photos without the other data like your stories, drafts, comments, followers and followings graph, etc.)
Perhaps locking it down puts pressure on IG to provide an official solution since others can't.
Or... one could write some code to run a headless browser and scrape everything that can be loaded on desktop.
That's fair from their perspective. Frustrating from a competitive standpoint: the most common reason for a user's "Why wouldn't I just use Facebook for that [instead of some new product]?" seems to be that all their data / friend graph is already on FB.
Does anyone know of any kind of community effort to democratize social data? e.g. ideal product: API-ified dashboard where I can view all my FB friends, IG followers, Twitter followers as well as every single piece of content (tweets, photos, etc) that I have permission to view on each respective platform. Standardized in a public format, live updated, and consumable by other apps (if permitted by those apps' users).
Super technically challenging (given that much of this data would be inaccessible without well-maintained brittle web scraping bots), but seems technically possible.
But I guess such an effort could run into the same scrutiny as the reasons IG shut down these endpoints... :\
(Not sure if this is related to the Cambridge Analytica situation. Maybe the timeline was sped up, but IG's APIs have been deprecating for the past 2-3 years now.)
The problem is that they threw out the deprecation schedule that they originally published and immediately retired/shutdown many endpoints with no prior announcements. Not even Instagram Partners were notified.
For those reasons, I absolutely believe it was related to the Cambridge Analytica situation.
Also important to see this in the context of the Cambridge Analytica story. Giving out user data about all of the followers / followings of a user, etc., including simple things like bios, hashtags, etc., is how these databases of user info get seeded.
Facebook have an API to export data - team privacy are loudly outraged, team data-portability are quietly happy.
Facebook lock down their APIs - team privacy are quietly happy, team data-portability are loudly outraged.
The end result being that no matter what happens, there will be a fairly consistent supply of outrage - the only question is do you want it from one side, the other side, or a bit of both?
Were developers notified at all this was going to happen?
Purely speculating--but I bet they discovered that their APIs were open for privacy abuse, and rather than just disable the abusing accounts, they shut down the APIs until they can be fixed.
Many of these endpoints were scheduled for retirement in July and December of this year. Developers were NOT notified in advance that they would be abruptly shut down today.
I also believe this is a reaction to all the recent privacy issues that Facebook is facing. They identified Instagram as a huge risk for behavioral mining (it is) and abruptly shut down all features that can be used to analyze behavior and infringe on privacy at a large scale.
This was not a deprecation. They immediately and without prior announcement revoked access to many endpoints that had been scheduled for shutdown later this year and next year.
Silly indeed. I applied for it twice, having submitted a screen recording explaining what I was using it for. They gave me partial API access, pretty useless functionality. Not what I needed which was clearly explained in the second video.
The application felt like I was begging. Not something I'd ever do again.
I never understood how this logic became so trendy.
- All software above the OS layer is built on someone else's platform.
- All websites unless hosted on your own servers are built on someone elses platform, most likely using libraries built by someone else, on someone elses platform.
- All websites run in a browser which you don't own.
You build a business that 100% depends on facebook. Facebook is a commercial company. At any point they can shut it down, and you are back to 0. Bad.
You build a website that depends on "a browser". There are at least 4 popular browsers desktop browsers, with several variations on mobile and more exotic browsers. All 4 are run by different companies. For most websites you would still function if even 3 of the 4 major browsers were shut down. Furthermore, desktop software is downloaded and cannot just vanish (let's ignore app store and that kind of magic, as not all browsers come in like that).
So I think you just proved the point on writing a "website" is a safer bet then writing a "tool for facebook"
With FB you have a single point of failure: "Hey Jerry block api_key." That is one field updated in one table.
For Firefox to block your website/web app they may have to force an update on the millions of browsers out there, which is not equally easy. If your hosting company shuts you down, you move to another (hopefully you had backups). If Google stops supporting Angular maybe you move to React. If Apple removes you from the appstore and people really want your app, maybe they may find ways to download. If Instagram says "no more access to our users from your app," well your app just became useless if it depended solely on Insta.
Yes, and if Buffer (Twitter), SimilarWeb (Chrome extensions, yes I know you buy extensions, i’m on to you!!), AppAnnie (App Store), Hootsuite (fb/twitter) and Heroku (aws) all listened to you, their founders wouldn’t have been rich.
Not surprising - as someone who has worked in the Facebook APIs for going on three years, the amount of undocumented breaking changes or abrupt API retirements is insane. Their overarching principle here has been "we're doing it now so you figure it out."
Got that notice too. Indeed, deprecated isn't "we're in panic mode and gotta act erratically to show those poor souls who have actually never read the T&C that we care".
We invested $50k in analytics and client experience. Going to the trash can.
Agree with quite a few comments here. The most profound one was an observation I made too: FB is effectively strengthening its monopoly by using the Cambridge debacle. Feels very much like post financial crisis behavior of the too big to fail banks (I worked for one).
But perhaps more importantly: Are "Instagram" partners actually affected? Beasts like Salesforce.com and Oracle consume huge amounts of data. There are dozens that consume audience data etc. Pretty sure that they are paying license fees to continue to operate with much more sensible API permissions and limits.
Thoughts?
It still works in the sense that their app needs to use it. But AFAIK Instagram stepped up server side security a few months ago, and it’s now nearly impossible to use that API from a data center IP. There’s a big thread on it in the issues of that repo you linked, but I’m on mobile atm can’t find it.
Thanks Charlie, I'm trying to get an export of old messages from Instagram. Scrolling up takes ages and I met my SO there, I'm hoping to get access to those first messages again one day!
No probs. I spent quite a long time experimenting and instaLooter was the best I could find.
Instagram's gone from a place where I posted the odd picture to a serious attempt to document my life (mostly to be consumed by myself). Throughout last year I became increasingly nervous about not having access to all the data!
Missing from their changelog is the most basic endpoint which has been retired. /media/ which now returns {"meta": {"code": 400, "error_type": "APINotAllowedError", "error_message": "This endpoint has been retired"}}.
To think the modified rate limit with no announcement was a critical issue--the effect of this shutdown will be seriously widespread. Despite the deprecation schedule, this seems drastic.
Considering the number of fires this is going to cause, application-wise...yikes.
Sooo, can I still cross reference customers from my database to see if they are influenecers on instagram or does GET /users/{user-id} and GET /users/self/followed-by ruin that?
But they have abruptly eliminated those API endpoints before plan.
"Deprecate" means that you've marked something for removal at some point in the future. From the other comments here, it seems that the endpoints have been abruptly shut down, effective immediately.
Instagram had previously published a deprecation schedule for these endpoints, and threw that schedule out the window without notifying even Instagram Partners.
Edit to add: these endpoints are all related to the profile data and behaviors of non-oauthed users. This is the exact data that ML researchers (myself included) would normally use to analyze social graph behavior, interests, and audiences of Instagram users, and also is exactly the type of processing activity that GDPR restricts by requiring explicit and revocable consent. I see the abrupt retirement of these endpoints as a reaction to the pressure FB is facing as of late. This is a positive for personal privacy and will reduce the potential for abuse, however many types of legitimate and responsible businesses will be affected as well.