
Google Has Ended a Bunch of Free Data - Brajeshwar
http://finance.yahoo.com/news/google-gone-dark-search-giant-142955454.html
======
gfodor
It's kind of stupid to react positively to this news just because you hate
marketers or ads. The truth is that incoming keywords from Google provide a
whole host of functionality and analytics for site owners as well. (I realize
they offer Webmaster Tools, but the fact is that getting incoming keywords for
individual visitors provides all kinds of leverage for personalization,
improving SEO, and understanding user behavior that aggregated, offline
reports will never come close to doing.)

The fact Google is still providing this data to people who do paid advertising
shows this isn't about privacy at all, but is about closing off a valuable
source of information for some other reason, most likely just under the guise
of privacy.

At a high level, it's not clear that this type of information being "leaked"
to sites was ever anything other than a function of the way Google worked from
v1, where the query was passed as a GET parameter. So it's debatable if this
was a good design decision in the first place. But Pandora's box was opened.
Due to this design decision an absolutely immeasurable about of code has been
built around this source of information (for good and evil) and calling this a
"data apocalypse" isn't really that hyperbolic for people relying upon it.

~~~
mortehu
> The fact Google is still providing this data to people who do paid
> advertising

By "this data", I presume you mean "traffic data grouped by keyword"? Or are
you saying that the search query is passed unfiltered along with the request
for individual ad clicks?

AdWords can provide aggregate keyword traffic data easily without compromising
privacy.

In order for Google Analytics, and similar software, to provide aggregate
keyword traffic data, the data would have to be bundled with each request. The
more data a website receives about you simultaneously, the less privacy you
have. Previously, a website would get your IP address, your user agent and
your search query, and at the same time have the ability to inject a tracking
cookie. The IP address I use is registered to me in RIPE, so in my case it
would be trivial to connect my search term to my name and address. If I
visited unencrypted sites, secret agents monitoring the Intertubes would be
able to store my search queries. Is this kind of data really available in
AdWords?

~~~
gfodor
I actually haven't verified personally if the keywords are passed along with
ads, I was going by the claims made in the article.

If you are concerned about your keyword data being aggregated and stored based
upon referrals and associated with IP, I have bad news for you. This data has
been being aggregated and cross referenced not only by sites and advertisers
but also third party aggregators for more than a decade.

------
jezclaremurugan
People are freaking out -> Marketers are freaking out. Ended a bunch of free
data -> Increased users' privacy.

Seems like a win to me.

~~~
dmix
My side business depends on SEO, but I also care enough about privacy to use
DuckDuckGo (which also doesn't share keyword data), so I'm ok with the
decision overall. Webmaster tools is pretty sufficient.

Privacy has trade-offs. Marketers/data miners will have to adapt to it if
their customers demand it.

~~~
ams6110
I almost never click on links, I retype them or copy/paste them into the
search bar. You really have no idea where you're about to go or what extra
data will be tacked on otherwise.

Edit: sorry this is unclear; I am talking about search engine result links. Of
course within a site or app I do click links though even then I often glance
at the status line to see what's about to happen.

~~~
vdaniuk
What? How is that possible without JavaScript? And javascript is trivial to
switch off or control.Copying and pasting links. That's nearly as hardcore as
browsing websites Stallman way.

~~~
ams6110
I clarified above, I'm mostly talking about search engine result links. Yeah
with most javascript-heavy sites, doing this everywhere is impractical.

------
anaphor
It looks like they just enabled SSL for all users, which is defnitely a GOOD
thing. These SEO people are confused because I guess it limits some data they
would've normally got, but it's a byproduct of them turning on SSL. It's not
some campaign by Google to limit SEO data except insofar as it gets limited by
SSL being on.

~~~
dangrossman
This isn't a byproduct of turning on SSL. If both Google and the site linked
to by Google are secure, then the referrer would be intact, and contain the
search terms, which is where Google Analytics and every other tracker has
picked them up for 15+ years.

Google purposely added another level of indirection between their search
result page and the search result links themselves. When you click, a bit of
JavaScript rewrites the link to a script at google.com/url in order to wipe
the real referring URL. Should Google want to do that for reasons other than
masking search terms, like recording clickstream data from their SERPs, they
could easily add the search term to that google.com/url link's query string.
That's exactly what they do for ads -- they transfer search terms through the
redirect's referrer even on SSL connections.

~~~
anaphor
It's been this way for a quite a while though, now the only difference is that
SSL is enabled for a larger number of people. Also adding the search terms to
the url would defeat the whole purpose of using SSL for search at all.

~~~
dangrossman
If the _whole purpose_ of these changes was to hide the search term from
website owners, then turning on SSL everywhere wouldn't be required at all.
That was already accomplished with the redirection between the SERP page and
the clicked link, which was put in place before SSL was turned on everywhere.

There are other purposes. Like preventing governments from recording your
long-term search history without a warrant. Frustrating that purpose requires
SSL but doesn't require wiping the search term from the referrer as well. It
would never be transmitted across an encrypted->unencrypted connection
transition per the HTTP spec.

------
mfincham
If this discourages "marketers" from using the web it sounds like a win to me.

~~~
vdaniuk
Not again. If marketers stop using the web there is no web. There is no
Google, bing, twitter, Facebook, reddit,stack overflow, hordes of startups
that use ads to monetize, plethora of blogs and therefore VC companies and
whole startup ecosystem would bust.

Do I want a more efficient system of transfer of commercial information? Yes.
Is there is better system than marketing at the present moment? No.

One of my hats is a marketer and I am shocked how the industry is
misrepresented here on HN and on reddit. The level of arguments is comparable
too.

~~~
quesera
> If marketers stop using the web there is no web.

There was a web before marketers.

I liked it.

~~~
vdaniuk
Then you are one selfish dude or dudette. I will endure some cognitive noise
aka ads if that means more web for everyone.

And you know what? 50% of ads I see are from edx,new relic, code
school,treehouse, engineyard, heroku and I am very ok with that,

~~~
quesera
No need to be hostile. I'm not threatening your livelihood.

Nor am I trying to change your opinion or influence your thoughts to suit my
agenda.

But I do confess: I wouldn't be disappointed if that was a common courtesy.

~~~
vdaniuk
Any oral or written communication between humans or machines has a probability
P to influence thoughts or opinions of human agents engaged in communication
or machines that are able to learn.

Marketers are increasing P to reach their business goals. Actions taken to
increase P may have positive or negative externalities. For example, edX uses
Google Adwords and remarketing to spread awareness about free moocs from
Harvard, MIT, etc. This is an example of an action with a positive
externality. Thus I demonstrate that subset of marketers that are trying to
change opinions is beneficial to society, even if they are pursuing their own
agenda.

I respectfully reject your statement that you are not trying to change my
opinion or opinions of other HN readers. See my first argument for
justification.

~~~
quesera
Even the most heinous of brutalities can have positive externalities. You're
really wrapped up in the defense of your point, but you're not arguing it
effectively. I wish your example was relevant.

Don't worry, everything is going to be fine. This decision was made long ago.
I remember the day. Nothing I or anyone on HN can do will change that.

However, I do take issue with your assertion that the web was built on
marketing. It really wasn't.

~~~
vdaniuk
Yeah, perhaps I am not arguing my point effectively. I am not trying to
convince you, though. I am trying to convince other HN users that might see
our discussion.

And I never asserted that web was built on marketing. Marketing is an emergent
property of social systems in general and web in particular. You cant banish
marketing, but you can amplify the good aspects. And to do that you've to have
a rational discussion on the subject. Which I dont see _even_ on HN.

Your condescension is not lost on me and thats fine. But your second line
sounds just like something fictional illuminati member in a Dan Brown book
would say. What day do you remember, again?

~~~
quesera
The day I remember is the day when the NSF announced their scheduled
withdrawal from oversight of the Internet, and the corresponding end of the
prohibition of commercial use of same. 1994?

You asserted a tight relationship between the _existence_ of marketers and the
_existence_ of the web. The direct quote is visible above, but I see that it
can be interpreted more than one way. Let's assume you meant to say that "the
web would be radically altered if it wasn't an advertising medium". Then, we
can agree.

You have a good point: the amplification of the good aspects of marketing is a
worthwhile goal. It's not going away, so is it possible to make it suck less?

The problem with your argument, in my eyes, and the cause for my lapse into
condescension, for which I apologize, is that your examples are so
extraordinarily cherry-picked that they seem absurd to bring up.

Advertising, even at its best, is a distraction from my intent. This is
_generally_ a negative thing. It's pretty important, but I'm not so fragile
that it's going to derail my whole afternoon. But it's not ingratiating, and
it's not necessary. You were arguing for the "necessary" angle, and I think we
won't reach agreement on that one.

~~~
webmaven
This may be an overly pedantic point, but marketing != advertising.

------
logn
That's good news.

I'm not sure which scares me more. Marketers amassing surveillance data or the
government. At least with the government, we might reign in the practice.
Private industry seems to have some blessing from the public to do anything
and everything profitable.

~~~
brudgers
Governments have an exclusive sanction to employ violence. Private entities do
not.

~~~
uptown
Blackwater (and other similar entities) would probably disagree.

~~~
quesera
You buried your point in pithiness.

Blackwater (XO) is acting at the behest of, and on the payroll of, government.

------
mwww
Google wants website owners to use Webmaster Tools so that it can connect
websites with individuals. The end goal is to make it harder for people to
manipulate search results in a way that would result in an inferior user
experience for searchers.

For many years, Google has been a gold mine to a profession called "SEO". It
still is, but every single day it is getting harder to extract value from it
if all you do is trying to get your site to the top of the SERPs. Eventually
it will be very expensive to trick Google and this is why I see many
professional SEOs starting work at big brands or doing consulting for them.
It's easier to get good rankings for a site that provides great value to users
than for another spammy endeavor.

I was also involved in this business many years ago, but mainly to rank my
sites that were getting traffic anyway because of the value they provided to
users. Instead of being angry, I think that SEOs should be thankful that
Google made it possible for them to take advantage of their service and even
helped them along the way — I am. There is nothing wrong in charging people
money for services that they use to make money themselves.

SEO should always only be a tool that helps you reach users with your quality
product, not your main way of making a living because in itself it creates no
value to society.

~~~
vdaniuk
Ugh. Someone is wrong on the internet and I just can't ignore that.

SEO doesn't create value for the society? That is an extremely bold claim.
Guest posts, infographics and other form of viral content are a very common
tools for so these days. They create immense value for content consumers and
that's the point. SEO is very much white hat these days in the anglosphere due
to superior Google search spam detection algorithms. There are black hats, of
course but they are not a defining market force for a long time already.

------
lstamour
"If a website is accessed from a HTTP Secure (HTTPS) connection and a link
points to anywhere except another secure location, then the referrer field is
not sent."
[http://en.wikipedia.org/wiki/HTTP_referer](http://en.wikipedia.org/wiki/HTTP_referer)
whose source is "Hypertext Transfer Protocol -- HTTP/1.1: Encoding Sensitive
Information in URI's (RFC 2616 § 15.1.3)". IETF. June 1999. Retrieved
2013-03-20. "Clients SHOULD NOT include a Referer[sic] header field in a (non-
secure) HTTP request if the referring page was transferred with a secure
protocol" [1]

Easy fix: Enable SSL everyone, stop "freaking out".

[1]
[https://tools.ietf.org/html/rfc2616#section-15.1.3](https://tools.ietf.org/html/rfc2616#section-15.1.3)

Edit: I see comments further up indicating that Google redirects from SSL to
non-SSL, thereby obfuscating the referrer and that this is new. If so, I'll
have to take back this statement, sigh.

~~~
dangrossman
The search term is not in the referring URL, SSL or not. Google removed it
from organic search, by a combination of moving it out of the query string
into a URL hash, and by adding a level of redirection between the search
result page and the linked website.

~~~
lstamour
Ahh true. Instant did change things, didn't it? I'm still used to omnibars
that hide the URL from me on Google searches that I forget it's not all
/search?q= any more. Good point, good point.

Though, given Google's native apps, perhaps this was inevitable? I mean, there
was always a percentage of traffic that would seem organic yet be from
Google...

------
woah
>It's a contradiction, according to Rishi Lakhani, a search consultant:

>... their idea of privacy is ridiculous to say the least. You can't offer
privacy, but still SELL the data to AdWords advertisers. It's the same user.
It's the same action.

The idea that clicking on an ad is the same thing as clicking on a link is
where I'll have to disagree with this guy.

~~~
gfodor
This is a pretty shallow analysis of his point. His point is that _insofar as
it related to the privacy of propagating keyword information_ , the action of
a user clicking an ad or an organic search result doesn't seem any different.
If the keywords a user searches for is private information, then why should
this privacy be off the table when they click on an ad?

------
guiambros
If this is about privacy, what about stop capturing search data _at all_?
Eliminate all user personalization and search history, as if you weren't
logged in. No more privacy issues, NSA court orders, etc.

Of course, this won't ever happen. This is not about privacy. This is about
_posing_ as a company that cares about privacy. It's easier to deal with some
upset marketers rather than governments pushing for stronger regulations and
anti-tracking measures, given the total lack of self-regulation (third-party
cookies, anyone?).

------
samsolomon
FYI search terms have been disappearing since a Google update in October 2011.
More than 95 percent of keywords have been marked as "Unknown" on my sites for
the last 6 months.

You can still make a good guess at what terms are leading people to your site
by the number of impressions on Google you get for a certain search term. To
discover this go to Analytics > Acquisition > Search Engine Optimization >
Queries.

------
r0h1n
So do unto Google as Google does unto others, in a way at least.

I perform all my Google searches while remaining logged out from Google. More
importantly, I use Firefox/Chrome plugins that rewrite Google search results
minus their tracking. In a way, I've gone "dark" to Google too.

