Wait so Twitter allows corporations to access and search public tweets, which are published by people so that they can be seen publicly? Oh the humanity!
Next thing we'll learn some company is indexing public websites and allows people to search in those too!
Not sure how Zuckerberg being rich matters, but at least in FB case one could argue that social graph data - i.e. list of friends - can be considered at least somewhat private (not sure about FB settings but looks plausible). Content of public posts is definitely not private without any doubt.
I'm not sure what un-obvious implications are there? That if you write something on the Internet in public service, people could read it? Isn't that the whole point of publishing it?
I think the implication is that people are fine when it’s used for advertising, but political campaigns is a whole different ballpark. As far as I understand, this is what makes the CA case different.
People not only were OK when Obama campaign used social networking data - they actively cheered it, including on HN and tech press, as a new and smart way to reach people. I remember reading a number of articles lauding them for it myself. Now a mundane activity of getting access to searchable public data is presented as some dark conspiracy.
I personally can't see any logic why political ads would be different than coffee machine ads - both try to convince me to do something. It's just there's no organized movement united by hate of coffee machines, so nobody makes a fuss about coffee machine ads.
And, that folk, is how regulations are born, when companies abuse the people.
In a few decades, "future HN"-reader will talk about how those regulation are stifling innovation by rising the bar of entry to any business managing user data, leaving the big players with an unfair advantage. Surely the world would be a better place if information was free to share ?
Facebook's public response destroyed every social media company's presumption of good faith.
> “In 2015, GSR did have one-time API access to a random sample of public tweets from a five-month period from December 2014 to April 2015,” Twitter said in a statement to Bloomberg. “Based on the recent reports, we conducted our own internal review and did not find any access to private data about people who use Twitter.”
Facebook kept changing their message regarding the affected accounts, from nothing to 87M and now possibly way more.
There's a possibility that Twitter might have sold DM information as well. Under any other circumstance they would have some presumption of good faith, but given what happened with Facebook there's a good chance we will hear more.
Everyone is in agreement that public tweets are public. Twitter also has Direct Messages [1], which they represent as private:
> Direct Messages are the private side of Twitter. You can use Direct Messages to have private conversations with people about Tweets and other content.
I think @stingraycharles was surprised by the response because the discussion surrounded publicly accessible information. The poster might not have realized that Twitter had information that is reasonably considered "private" as well. If DM information (metadata or raw messages) were made available, that would be surprising to millions of Twitter users. And I think we will see more scrutiny of Twitter and other companies in the coming months regarding the use of "private" information.
Unless, of course, you believe that SV and tech companies are not to be trusted in the first place, in which case none of this would be surprising or shocking.
Because they don’t offer DMs or DM metadata for sale. This Kogan guy had no special connections at Twitter or even Facebook for that matter. He was an independent developer using off the shelf offerings of Twitter and Facebook to get what he was after. If DMs aren’t for sale (they’re not) then he didn’t get them.
They don't "sell" DMs - they just hand a DM firehose over to the feds.
Haha - you'd be stupid to think that twitter doesn't do this.
Hell, library of congress archived the full firehose from 2010 until now - and now it will only archive "select content"
Gee - thats because POTUS is using it as his primary platform and a POLICY platform.
So -- it would seem clear that they are using 6-degrees from any and all that comment and RT POTUS tweets, and any other's that they want.
but just because lib of congress is no longer slurping that firehose up -- does not mean that other 5Eyes are not. I think it would be myopically foolish to think that after the NSA was caught intercepting HW and installing spying tech into it, that they wouldn't be doing it for twitter+DMs as well.
Library Of Congress Will No Longer Archive Every Tweet
Since 2010, Library of Congress has been archiving every single public tweet: Yours, ours, the president's.
But today, the institution announced it will no longer archive every one of our status updates, opinion threads, and "big if true"s. As of Jan. 1, the library will only acquire tweets "on a very selective basis."
The library says it began archiving tweets "for the same reason it collects other materials — to acquire and preserve a record of knowledge and creativity for Congress and the American people." The archive stretches back to Twitter's beginning, in 2006.
It's interesting how we call Facebook's data either public or private depending on how we want to regulate it. If we're worried about user data privacy, it's private data that CA stole. If we're worried about racial tensions in Sri Lanka its a public forum that needs to be moderated.
Not that it's wrong, but Facebook seems to exist in this grey zone of semi-private content which gives it both sorts of challenges and none of the usual defences.
Facebook users expected a certain level of privacy. They didn’t realize quiz apps would harvest a huge amount of data about their profiles. There is also the question of what kind of data they gleaned based on being friends with people who installed said apps.
Twitter is 100% public like a website and the expectation isn’t the same. When you friend or unfriend someone on FB you are somewhat implying who you give access to your feed.
>> They didn’t realize quiz apps would harvest a huge amount of data about their profiles.
> They would've known, had they read the dialog they had to accept to access their much desired quiz.
You do remember Facebook would share your friends' data with the quiz along with yours? Your friends never got a vague click-thru warning when you accessed your "much desired quiz."
So now average users are expected to be able to interpret adversarial-y written EULAs? If Joe Nobody should be able to disentangle a web of lawyer jingo and ass covering from what may very well be a team of lawyers with combined decades of experience, then why do we even have lawyers in the world?
>If you can't read and understand that I'm sure you can collect disability checks from the government
When it comes to computers there is a WIDE divide between can't read and doesn't read. I'm sure nearly every person in tech has at some point in their life had an encounter with a non-technical user who, given an error message that clearly states the problem, has selective blindness and doesn't read the error message but rather just clicks 'okay' and can't figure out what the problem was. They could read the error message but have effectively been trained not to.
Similarly facebook users could read that message detailing how their privacy is about to be violated, but have probably been trained not to care.
I'm of the opinion that this is definitely something that has occurred and it has been done very, very deliberately. The media and ad industry and companies like fb work(ed) very hard to encourage people that:
* they should give up privacy in return for (carefully metered out dopamine hits) shiny things.
* they should give up more data and privacy because they've given up x much already
* They've given up all this stuff, fb/et etc already knows everything so they don't have any privacy so why bother protecting stuff _now_
* They gave up all this stuff and look, nothing bad has happened, so if you want privacy, you must be hiding something/a terrorist/a paedophile/etc.
And thus we arrive where we are now. Trying to convince people to take back some privacy and control to un-fuck the whole situation is an uphill battle, because those whose interests are in retaining control and lack of privacy are very well resourced and have gone to great effort to hide negative effects from the users and convince them of their non-existence.
Log into? The 'typical user" interpretation of what "access" means in this context is probably worse than what was actually done. "Access" isn't a word that sounds innocent, if the app told them it was going to "'access' their bank account" any user who actually took the time to read it instead of blindly clicking through certainly would have second thoughts to say the least.
The uncomfortable truth of the matter is that facebook users using these kind of quiz apps simply didn't care about privacy. When presented with that dialog they shrug and clicked through. Privacy was something they didn't take seriously at the time, if ever. The two big fuckups by facebook, imho, are:
1) Allowing such privacy invasion, knowing that foolish users would fall for it. They should have protected their users by forbidding this sort of thing in the first place even if their users were too foolish to desire such protection.
2) Allowing careless users to give away information about their friends, many of whom would have avoided such "quiz apps" but got sucked up by the apps because of their careless friends.
It doesn't take much skill to read lawyer lingo. The hard part is writing it, and that's why lawyers make money doing that.
It's the same for any language. For example I can read French and Portuguese but I can't write them. Also I can read very fancy English texts but my writing is very basic.
>When you friend or unfriend someone on FB you are somewhat implying who you give access to your feed.
It may feel that way, but it is an unrealistic expectation. Something shared with hundreds of people is effectively public information. To believe otherwise is a fantasy.
In “the real world”, we intuitively understand this: a secret told to two dozen people is a secret no more. But the illusion of control on social media sites seems to convince people otherwise online.
Most users accepted this as a fact. Behind the scenes and not transparent to the user FB was allowing much more access. Now they are understanding the implications.
Not everyone is a highly technical person and to lump everyone into this category is not right. Try explaining this to elderly users.
I think the right way to think about audience controls on social media posts is as controlling who you affirmatively include in a part of your life, not as a way to actually keep any secrets.
For example, you might post pictures of your college reunion, and not share them with work colleagues because you want to maintain a more professional kind of relationship with them. But you shouldn't expect any confidentiality for things that happened at a 200-person event.
A real world example is meeting up with your friends at the bar and running into a coworker. You aren't prohibiting your coworkers from seeing you out with your friends, but you also aren't affirmatively sharing that part of your life with them. And what is appropriate to coincidentally be seen doing is different than what is appropriate to actively share in certain contexts.
In the real world, something you tell a couple dozen friends is no longer secret, but you don't expect it to wind up in corporate/government databases or on the front page of the paper. That's the model that's misdirecting people about Facebook.
It's weird, but the world in general consists of grey zones. There are no clear black-and-white distinctions.
Facebook clearly is a public forum. It clearly has shared private data with CA. Both of these issues deserve to be discussed, and both will have different, possibly even conflicting outcomes.
And really, that same thing could be true for, say, cable companies as well - they provide a public service, and they are regulated by a local franchising authority. We don't currently see content regulation, but if your local cable provider started, say, providing a channel dedicated to cannibalism, you might see changes in that. At the same time, the subscriber lists are considered private, and sale of them is regulated.
The main reason you don't see much of a mix here is that pre-Internet communication is mostly of the broadcasting type, which has only very few areas where private data actually exists.
Facebook is not clearly a public forum. My close friends do not constitute the public, yet CA got access to data that only those close friends would have otherwise seen.
Many aspects of Facebook with different expectations of private/publicness. Examples: comment form on public website powered by FB, FB page for å product, public FB group, private FB group, FB direct messages, FB group messages, personal FB wall/feed.
In light of these, categorical statements about all of Facebook is not very meaningful.
Stupid but obvious "what-if-question": How can two private citizens attain the equivelant of "attorney-client privilege" -- What is the reason that such privacy 'rights' are exclusive to conversations between an attorney and a person/entity.
Sounds like 'honor among thieves' when talking simply about attorney-client privilege is only honored between and in only those in the legal field?
> How can two private citizens attain the equivelant of "attorney-client privilege"
By one of them employing the other as an attorney (attorneys are private citizens).
(There are other similar but not strictly equivalente privileges, such as physician-patient, accountant-client, clergy-penitent, and marital privileges.)
> What is the reason that such privacy 'rights' are exclusive to conversations between an attorney and a person/entity.
Attorney-client privilege is needed because effective assistance of counsel requires people not to fear increasing their legal jeopardy through honest communication with their attorney.
For how much time? When you consider youporn sends ping to twitter throught a weird domain (with a cookie on https://syndication.twitter.com/), it makes me wondering how much time it will takes before the porn life of everyone on twitter will be reveal.
I don't want to be the alarmist but it's not like twitter has a good track record when it comes to database leak and god knows how many influent politics are using it ...
Seems like every web page pings all the social networks nowadays. The app world is worse, much more difficult to see who is phoning whom.
You need to install three extensions just to battle all this abuse. Getting a browser ready to surf the web in 2018 is like preparing to invade Normandy, and quite absurd.
This title is misleading. By saying “Twitter also sold data access...” it implies that Facebook sold him data access. This of course is not the case. Facebook didn’t sell him anything, he produced an app and abused the platform.
Didn't they get more than what was expected from the API? The user accepted his profile information but they also got his friends information.
Based on Wikipedia:
> Abuse is the improper usage or treatment of an entity, often to unfairly or improperly gain benefit.
That seems like they used the API improperly to access his friends information and got the friends information.
For sure Facebook shouldn't have given that information, but it was also clear that information wasn't supposed to be there, that Cambridge Analytica were aware of it and abused it.
The user accepted his profile information but they also got his friends information.
That was expected and agreed to behavior by all parties (users, developers, and Facebook) at the time. I know it’s out of fashion in 2018 to acknowledge that TOS exist and that users should have read them before agreeing to them (or at least before complaining after the fact), but this is the reality.
The “abuse” part was that you were supposed to only access data that was necessary for the operation of your app. Accessing friends’ profile information to make an app that shows people who work at the same company as you, for example, was completely acceptable and a good use case. Collecting it for purposes unrelated to the stated purpose of the app was and is abuse. The Obama For America app also abused the platform in this way, but on a vastly larger scale - they pulled the entire US social graph (about 4x as many profiles as Kogan had) after less than 1 million people actually authorized the app and used it in all sorts of ways outside their app. This technique was widely celebrated in the press, right up until the “wrong” candidate won.
> I know it’s out of fashion in 2018 to acknowledge that TOS exist and that users should have read them before agreeing to them (or at least before complaining after the fact), but this is the reality.
It's not "out of fashion," it's unrealistic and unreasonable.
If everyone fully read and understood every TOS and EULA they're affected by, along with all the unilateral changes they also usually "agree" to be bound by in the future, everyone would spend so much time reading them that they'd die because they'd have no time to eat, let alone actually use the services tied to the documents.
I actually agree with you that it's not as big of a deal as the press has made it out to be...certainly not Congressional hearings big. But it was a violation of the developer policies in force at the time to use data for any purpose outside of the app.
Twitter also gives away their data to anyone that wants it for free. You can be streaming millions of tweets per day within minutes of creating an account.
That used to be the case, sure. But AFAIK these days you won't get access to the firehose without full authorization from Twitter (and there's a price tag involved).
> Twitter also gives away their data to anyone that wants it for free. You can be streaming millions of tweets per day within minutes of creating an account.
That's news to me. I maintain the Twitter client library for Go[0], and I still haven't been able to get through the process for getting access to the new streaming API (for testing the library). It requires an application stating your intended purpose and usage. I applied explaining the situation and was rejected (with a request to provide more information - I still have to resubmit that, but it's time and I just haven't gotten around to that yet).
It's possible to stream millions of tweets per day, but good luck trying to do it "within minutes of creating an account". And it's definitely not free.
For full context: I'm already a verified user on Twitter (ie, "blue check mark") and was clear in my application that I was requesting access so I could develop and test the library that other users of the Twitter API develop with. I'm not exactly complaining that they didn't approve the request initially, but if that's not sufficient to get access, it's pretty clear that they're not "giving away their data to anyone that wants it for free".
Maybe you are referring to their paid service, Gnip? Yes, you need to pay for that one, but they have a free version that you can indeed access within minutes. Try Twitter4J.
> Maybe you are referring to their paid service, Gnip? Yes, you need to pay for that one, but they have a free version that you can indeed access within minutes.
The streaming API is heavily rate-limited, so you can't realistically stream "millions of tweets per day within minutes", in the context of the original article.
I think we are talking past each other. I am not talking about the Account Activity API or Site Streams. I am talking about their FREE streaming API that let's you stream millions of tweets per day. I am just trying to raise awareness about how easily people can pull user data en masse from Twitter, which is very relevant to the article. It's public, but people still don't realize just how easy it is to do.
> I am talking about their FREE streaming API that let's you stream millions of tweets per day. I am just trying to raise awareness about how easily people can pull user data en masse from Twitter, which is very relevant to the article. It's public, but people still don't realize just how easy it is to do.
Yes, that's an endpoint off stream.twitter.com. It's not only limited in terms of how much relevant info you'll be able to access (in the context of the original post), but also heavily rate-limited for clients.
I agree that it's confusing that Twitter has so many different components of their API which use overlapping names, but I've looked into this extensively because, as I mentioned at the outset, I wrote the client libraries for these and still maintain them.
It is very easy to setup multiple streams to get a LOT of data. Not everything, but a metric shit ton. I have collected over 10 billion tweets for Twicsy over the years, and that is only because I was targeting pictures. If I had cast the net wide I could easily have gathered 10x that, and if I was super aggressive maybe 50x.
It is up to 1% per stream. With 2 streams you can stream up to 2% (minus overlaps). This is theoretical, practice isn't quite that good, but not far off. In reality you can collect about 4-5 million tweets per day with a stream. If you look for common terms like http,and,for,the, etc you can collect 10's of millions per day easily with 20 or so streams. If you take the time to develop some tech and pull hundreds of streams designed to have minimal overlap you can pull 100+ million per day.
And how do you pull more than 1 stream, you ask? The limit of streams isn't per app, it is per authorization. If you develop an app that has sign in with Twitter and you collect auth keys for 100 people you can have 100 streams because the auth limits are per user, not per app.
Interested dummy here: You can get a 1% stream per user authorisation (sounds weird to me)? How do you minimize overlap?
I once had access to a 1% stream but thought it was a fire hose test version (and everyone would get the same 1% to avoid combining like you describe).
There is a sample stream which is supposed to be 1%, but then the keyword based filter streams are also theoretically up to 1% of tweets. You can setup streams for different words and broaden your coverage.
(less one-wordily, 3500 posts or 30 days history per user, if I'm remembering right. (or something to that extent) There are a bunch of ratelimits. And if you really get into it, the "firehose", the full stream of all tweets, certainly isn't available to a normal user.
But I assume all these work arounds would be against the terms of service. So Cambridge Analytica were paying for legal access to this data. I think that makes it a bit more significant (though the situation is still very different than Facebook).
"The ethics panel member added: "Facebook is rather deceptive on this and creates the appearance of a cosy and confidential peer group environment, as a means of gulling users into disclosing private information that they then sell to advertisers, but this doesn't make it right to an ethical researcher to follow their lead.""
A bot of a tangent, but a year ago I loved Twitter and found it engaging. I tweeted mildly interesting comments and sometimes mildly amusing anecdotes.
Then the algorithms took hold. Now most of what I tweet goes totally unnoticed. Now perhaps I am more boring now (even though I have a lot more folowers), but it doesn't seem to be the case, I think I was just as inane before.
On the other hand, if I get maybe four or five friends to retweet me, all of a sudden I have exposure and I get read, liked and retweeted a lot. This has just pushed Twitter more into the hands of bots. If I had a bot army of maybe 10 bots I would get a fair bit of exposure.
There is also a momentum thing, if I take a couple of weeks off Twitter it doesn't love me any more when I come back.
It's really too bad, up until a few months ago the Twitter timeline was an ok place to see what other people whose work I admire were up to, but now it's a torrent of irrelevant spam so why bother using it. So frustrating to see something good be ruined, goes to show how little control over your experience, and indeed everything else, you have with these services.
I am pretty sure you can disable the algorithmic feed in the settings. It's under "Show the best Tweets first" checkbox in the "Account" page of the Settings.
At least I thought carefully about what I tweeted since I knew it would be public. I'm much more upset that private Facebook data, never intended to be distributed to the public, was taken from me.
I’m with you; github is my social network. I love that their business model is about supporting open source products but providing infrastructure for pay for companies that need privacy. That way, I’m not a product, my work is shared with the world, and they get to make money.
Sorry I don't understand the severe downvote? I was referring to all the "under the table" things that happened with Cambridge Analytica and similar stories, that will surface in the next few weeks/months. Hence the crumbing house of cards...
Furthermore complaining about being downvoted is against the guidelines.
Not saying you complained but some people seems to use that as an excuse for downvoting every metaquestion that touches into these stupid markers of approval/appreciation or whatever it signals : )
Agreed. I'm at -7 and I didn't do anything to deserve this. It's quite rude and unfriendly. But it clearly makes some people feel good to have the power to do that, certainly because they can't do it IRL. Poor sapiens.
I understand how this can be frustrating, but it is not true that you didn't do anything to deserve this. You published several comments with high noise-to-content ration (and yes, complaining about downvotes is also noise). You got downvotes for that. Take them, move on and do better next time. It's just bits on some server, and they mean nothing except showing you how to improve your participation on the site.
Now is the moment. The moment where you eat your tie/ hat/whatever. Yes, you, who blindly defended Twitter that "did no harm" and "Twitter is not like FB".
Come on people, company that hosts insanely large free webservice 24/7 for years doesn't sell user data to the highest bidder? Do you even believe it?
http://support.gnip.com/apis/
https://developer.twitter.com/en/docs/tweets/batch-historica... (new doc for Gnip API)
https://developer.twitter.com/en/docs/tweets/search/overview... (new doc for Gnip API)