

Google Fires Back at Twitter: You Took Yourself Out of Search - rhufnagel
http://mashable.com/2012/01/11/google-twitter-search-3/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Mashable+%28Mashable%29

======
dpritchett
_Sullivan said he thought Google had enough permission to include links from
networks like Twitter in its search results, Schmidt said: “That’s your
opinion. If you could arrange a letter from Facebook and Twitter to us, that
would be helpful.”_ [1]

Sounds like the reporter expects Google to be able to scrape and index all
tweets in real time. The whole point here is that (1) that's a big deal and
(2) Twitter would prefer to charge dearly for that access if they allowed it
at all. In negotiating terms, Google+ provides Google with a great BATNA in
this situation where previously all they had was "pay whatever Twitter asks to
index tweets or go without".

Frankly, the "I believe you have permission, make it happen!" argument hinted
at in the above quote sounds just as naive as the Congressman who says "We can
do it! We have the technology!" about SOPA.

[1]
[http://www.youtube.com/watch?feature=player_detailpage&v...](http://www.youtube.com/watch?feature=player_detailpage&v=o3FEILaTP3o#t=110s)

~~~
Zirro
"Sounds like the reporter expects Google to be able to scrape and index all
tweets in real time."

Do we know how this worked on the technical side "back in the days" when
tweets appeared on Google? Did they share a database, or did Google actually
scrape in close to real time? (I can't imagine the second alternative being
particularly efficient for any of them, though)

~~~
smackfu
Google used to have access to a firehose of live tweets, but then their
contract expired.

[http://mashable.com/2011/07/04/google-realtime-search-
suspen...](http://mashable.com/2011/07/04/google-realtime-search-suspended/)

~~~
Zirro
"While we will not have access to this special feed from Twitter, information
on Twitter that’s publicly available to our crawlers will still be searchable
and discoverable on Google"

"A special feed", that's the information I was looking for. Thanks for the
link.

------
jroseattle
The posturing going on right now between Google, Facebook and Twitter is
childish and hypocritical. Each company is only interested in leverage against
each other, pure and simple. And when that approach stalls, they attempt to
curry favor among the public with their situationally-convenient comments.

The disingenuous collective here is ridiculous. Google and Twitter haggle over
dollars, Twitter says no thanks, and Google says Twitter "took themselves out
of search"? Google goes with what it has for real-time data (Google-plus) and
incorporates that into search results, and Twitter calls it "bad for people"?
For all the smart people involved in these organizations, I find this public
conversation makes everyone involved look really small and petty.

Similar to the game of adding "in bed" to every fortune from a fortune cookie,
I now append "because we want to be in control" to every statement from
Google, Facebook or Twitter.

~~~
calloc
Twitter is returning it's pages with all of the links marked as rel=nofollow.
This means Google can't index the content that is linked by Twitter users.

~~~
ljlolel
That's not true. It just means that it won't contribute to pagerank. But the
GoogleBot has discretion to crawl or not crawl the links.

~~~
Karunamon
So you're saying that the Google spider should break with expected and normal
behavior (honoring nofollow) in order to crawl pages that someone else has
gone to the trouble of making noncrawlable?

Yeah, that won't get them into any PR hot water or anything! </sarcasm>

------
mladenkovacevic
If Google had to pay every web-service out there for the privilege to index
their otherwise free content, that could cost Google more money than is
possible for them to spend.

Then again, Twitter and Facebook are certainly large enough that some kind of
deal could have been negotiated. But if they did that, would other social
networks uproar in protest because now it's not just Google monopolizing their
power.. it's the 3 largest websites out there doing it together.

~~~
cbs
_If Google had to pay every web-service out there for the privilege to index
their otherwise free content, that could cost Google more money than is
possible for them to spend._

It would also be a disturbing direction to point the web in. The reason that
google and other search engines are useful is because their bots can crawl the
web like any user, they just also obey when a website asks them to stay away.

Using robots.txt or nofollow to keep bots away unless they cough up cash
concerns me greatly because it means they're turning what was a "count me out
please" mechanism into a paywall. A paywall that the public doesn't see, but
goes up between the public and finding public content with a search engine.

------
nohat
I do think we should be worried about letting google get too strong a walled
garden. The most worrisome thing about google is that we depend upon so many
google products, that it might be difficult to see what we are missing if they
started to overly favor google offerings. I don't think google has gone there
yet, but it is something to keep track of. That being said, twitter has no
excuse to complain if they did simply remove themselves from google's results.
I suspect that in fact they negotiated and got a better deal going exclusive
with bing.

------
rodh257
I think that Twitter search is a real missed opportunity, I was hoping that
Google with all their resources would be able to capitalize on it, but looks
like Twitter isn't playing along.

I did a university project (retreave.com) which used the streaming API to
receive tweets, index the pages they linked to, and then provide a lucene
based search engine for it. It's really cool to search for say 'design books'
and see what your twitter network has reccomended that relates to it (and of
course, the search doesn't just search the tweet text, but the page text as
well). It ranked results based on social stature in addition to relevance (ie
a link tweeted by lots of people is higher ranked)

Unfortunately the User Streams API only scales to a few users, and I've been
waiting for 9 months for an invite to the Site Streams Beta, but looks like
they don't want me to make their search useful either.

------
SeanLuke
I don't understand why there is a deal at all. Why isn't the situation simply
that Google indexes things (like tweets) unless they have a rel=nofollow?

~~~
kanamekun
As of June last year, Twitter was churning out 155 million tweets a day:
[http://radar.oreilly.com/2011/07/gnip-twitter-firehose-
realt...](http://radar.oreilly.com/2011/07/gnip-twitter-firehose-realtime-
data.html)

It's not easy to index find that much fresh data in anything resembling real-
time. And with real-time data, seconds and minutes matter.

------
dredge
Once upon a time it was possible to get a clean (not rel=nofollow) link from
one's Twitter profile: <http://www.davidnaylor.co.uk/twitter-backlink-
tip.html>

Presumably for anti-spam reasons, Matt Cutts (of Google, and regular Hacker
News commenter) implied in a tweet that Twitter would want to fix this:
<https://twitter.com/mattcutts/statuses/865610396>

I think this implies - in my opinion rightly - that Google want or wanted
links on Twitter to have the nofollow attribute set.

~~~
abraham
Twitter was already implementing rel=nofollow. Matt was just letting them know
they were doing it inconsistently.

------
grannyg00se
Does it not make sense that Google would want to create as much synergy as
possible between their offerings?

I don't understand the concern here. It looks to me like they are going to use
a small amount of screen space to let users know how many relevant Google+
links were found in the search. All of the Google+ links will be collapsed at
the top. The other 95% of the page layout and result is the same.

------
Forrest7778
I don't mean to go off topic - but for some reason the imagine of the Google
'G' and the Twitter 'T' just looked like the word 'Git' to me, so I was
hopelessly waiting for more controversy involving Github to make it's way into
the article...

[http://8.mshcdn.com/wp-content/uploads/2012/01/google-vs-
twi...](http://8.mshcdn.com/wp-content/uploads/2012/01/google-vs-
twitter-360.jpg)

------
mcgwiz
It all depends on how you look at the overall system Google has created. If
one considers the entirety of Google+ to be a mere component of Google Search
(like the now-extinct Friends section of Netflix was), then one can better
reconcile this situation with supposed fairness. After all, what's to stop
Bing or Yahoo from creating a social component to their products which affect
search?

So the issue in is not one of ethics, but of how Google is marketing Google+.
When all that existed was the +1 button, no one complained about ethics.
Nothing has changed since then except a reworking of the information
architecture of the same +1 data and processes. Google seems to believe that
Google+ needs to exist independently of Search. If they're gonna stick with
that route, they need to open the Search platform to 3rd party integration.

------
espeed
This fight is going to spill over to Facebook too -- Google has been trying to
get Facebook to open up for a while.

------
dos1
I can't help but be reminded of early 90's Microsoft when I read about what
Google is doing lately. They're definitely on the "extend" phase of embrace,
extend, extinguish. They're creating their own little insular community.
Search results return Google+ results first and prominently at the top. Their
browser has features that no other browser supports (native client, Dart etc).
Their previously free APIs are now going to cost lots and lots of money
(Maps). None of these things are bad in and of themselves, and all of it is
certainly their choice. However, if this is the direction they're headed then
I want no part of it.

Edit: For those of you who find my comment off topic - did you even read the
article or just the summary? Here are some relevant quotes:

"Twitter had criticized Google’s new social search feature, which it calls
Search plus Your World, on Tuesday. As we’ve seen time and time again, news
breaks first on Twitter, its statement said. We’re concerned that as a result
of Google’s changes, finding this information will be much harder for
everyone."

"Sullivan told Schmidt he thought one feature of Search plus Your World, which
recommends relevant people to follow on Google+ but not other networks, is the
equivalent of saying ‘hey, you can only find information about finance on
Google finance. You cannot find information about finance anywhere else."

Clearly people are upset about Google trying to block out competitor's
results. I believe my comment is completely ON topic.

Edit 2: Wow, I had no idea how many Google apologists were out there. I had
high hopes that the Hacker News community might actually be objective and not
just fawn all over Google. Christ was I mistaken.

~~~
rryan
> native client, Dart, etc.

You forgot SPDY, WebRTC, etc. This is nothing like ActiveX. These are open-
source technologies and other browsers are free and _invited_ to implement
them. They're trying to push the web forward.

~~~
dos1
Sure they're free and open source. But they aren't STANDARDS. You seem to be
unable to look objectively at the situation. While ActiveX was closed source,
by extending the web with non standard technology the end result will be the
same, open source or not.

~~~
yanw
But they are adhering to standards as well, more so than anyone else for that
matter, point to an instance where they're not. Also it takes ages for stuff
to get standardized, are you suggesting that no technology should be
applicable until it's the standard? I see no issue in introducing new cool
tech specially if it's opensource.

~~~
jrockway
I think people complaining about this are upset about the network effect,
rather than Chrome adding new features. Say you love Firefox. One day, your
favorite website decides to switch entirely to SPDY, and you can't use Firefox
anymore. It's natural to blame Google for this, since they invented SPDY and
put the client into production. But really, it's that site's developer's fault
for using nonstandard features.

This happens all the time; Firefox adds non-standard CSS transforms, Microsoft
adds non-standard Javascript functionality, and Apple adds proprietary video
codecs to Safari. When your web app depends on these things, you hurt your
users. But it's your fault for using nonstandard features; it's not Apple's or
Google's or Microsoft's fault for making their browser support them. If this
upsets you, consider software in general. To use Facebook, you have to use
Facebook. That's nonstandard! To use an Emacs extension, you have to use
Emacs. That's nonstandard! And so on.

Google and Mozilla are especially innocent, since Apple and Microsoft can
easily steal any features they want from Chrome and Firefox. They're the only
ones out in the open, and for that, I think they deserve to try new things for
the web. If they stick, the other vendors can easily support the new features.

~~~
nudded
first you say "One day, your favorite website decides to switch entirely to
SPDY, and you can't use Firefox anymore. It's natural to blame Google for
this, since they invented SPDY and put the client into production. But really,
it's that site's developer's fault for using nonstandard features."

then "Google and Mozilla are especially innocent, since Apple and Microsoft
can easily steal any features they want from Chrome and Firefox. They're the
only ones out in the open, and for that, I think they deserve to try new
things for the web. If they stick, the other vendors can easily support the
new features."

How would these features stick, if developers using them are to blame?

~~~
jrockway
By degrading gracefully, of course. It's why you _can_ read GMail in IE6, even
though the experience is a lot better in Chrome 17.

