It's kind of stupid to react positively to this news just because you hate marketers or ads. The truth is that incoming keywords from Google provide a whole host of functionality and analytics for site owners as well. (I realize they offer Webmaster Tools, but the fact is that getting incoming keywords for individual visitors provides all kinds of leverage for personalization, improving SEO, and understanding user behavior that aggregated, offline reports will never come close to doing.)
The fact Google is still providing this data to people who do paid advertising shows this isn't about privacy at all, but is about closing off a valuable source of information for some other reason, most likely just under the guise of privacy.
At a high level, it's not clear that this type of information being "leaked" to sites was ever anything other than a function of the way Google worked from v1, where the query was passed as a GET parameter. So it's debatable if this was a good design decision in the first place. But Pandora's box was opened. Due to this design decision an absolutely immeasurable about of code has been built around this source of information (for good and evil) and calling this a "data apocalypse" isn't really that hyperbolic for people relying upon it.
> The fact Google is still providing this data to people who do paid advertising
By "this data", I presume you mean "traffic data grouped by keyword"? Or are you saying that the search query is passed unfiltered along with the request for individual ad clicks?
AdWords can provide aggregate keyword traffic data easily without compromising privacy.
In order for Google Analytics, and similar software, to provide aggregate keyword traffic data, the data would have to be bundled with each request. The more data a website receives about you simultaneously, the less privacy you have. Previously, a website would get your IP address, your user agent and your search query, and at the same time have the ability to inject a tracking cookie. The IP address I use is registered to me in RIPE, so in my case it would be trivial to connect my search term to my name and address. If I visited unencrypted sites, secret agents monitoring the Intertubes would be able to store my search queries. Is this kind of data really available in AdWords?
I actually haven't verified personally if the keywords are passed along with ads, I was going by the claims made in the article.
If you are concerned about your keyword data being aggregated and stored based upon referrals and associated with IP, I have bad news for you. This data has been being aggregated and cross referenced not only by sites and advertisers but also third party aggregators for more than a decade.
My side business depends on SEO, but I also care enough about privacy to use DuckDuckGo (which also doesn't share keyword data), so I'm ok with the decision overall. Webmaster tools is pretty sufficient.
Privacy has trade-offs. Marketers/data miners will have to adapt to it if their customers demand it.
I almost never click on links, I retype them or copy/paste them into the search bar. You really have no idea where you're about to go or what extra data will be tacked on otherwise.
Edit: sorry this is unclear; I am talking about search engine result links. Of course within a site or app I do click links though even then I often glance at the status line to see what's about to happen.
What? How is that possible without JavaScript? And javascript is trivial to switch off or control.Copying and pasting links. That's nearly as hardcore as browsing websites Stallman way.
I don't get how copy-paste is more onerous than crippling one's browser. I C/P links all the time too, to prevent click-jacking, and to know what site I'm hitting, and what type of content I'm likely to expect.
Yeah, I hate those dirty marketers who wrote the HTTP spec. I'm sure glad Google is thwarting the evil IETF's schemes.
Let's be real here: the biggest marketing company in the world still has all this data and way more. They are simply forcing site owners to pay to get at it now.
Yep, while I have objections to most Google criticisms, I totally agree that killing off search referrers is bonkers. I guess there will be some starttyups springing up that will try to correlate your search rankings and page visits, creating fuzzy but actionable stats concerning an search keywords.
Just because you own a web site doesn't make you a marketer or even looking to turn a profit.
Google is going dark because they are scraping web sites and keeping their visitors on google.com. Take a look at Google Images today verse a year ago. You can also see glimpses on certain mobile searches related to wikipedia keywords where they outright scrape the page, and the information boxes they show on the right hand column for certain terms on desktop.
Just because you own a web site doesn't make you a marketer or even looking to turn a profit.
Owning a web site also doesn't give you any right to know how people are finding your site without you having to ask them to volunteer the information. That's what businesses in the real world do: they ask you, and you choose to tell them or not.
I think comparing it to real world business is disingenuous. When a customer walks into a store there is a whole wealth of information that you can get without asking the customer that would cause an uproar if made visible online.
Physical appearance... what else? That's hardly a wealth of information, and if the store owner is too obvious about documenting the physical appearance of customers who are just walking around they could get into some trouble. Security cameras are an accepted necessity, for security, but not most people wouldn't accept them for marketing purposes. This brings to mind the ads in Minority Report, that scan your eyeballs to identify you and show you a personalized ad. That's technically possible today, but I doubt you'll see it anytime soon.
In a real-world business, you don't get actual information from a visitor until that person buys something and becomes a customer. At that point you'll often get a credit card which gives you a name and relatively unique id, and you'll have the opportunity to interact with the customer and ask questions like "Did you find everything you were looking for?", "What brought you in today?", and "Is there anything else I can help you with?".
Depending on the type of store, you can have salespeople wander around and ask visitors those questions too, but they can't be pushy about it without violating the person's privacy and guaranteeing that the visitor won't become a customer.
Gender, age, ethnicity, level of interest, general idea of pedigree (e.g. accent and vocabulary), ballpark of income based on accoutrements, possibly information about your family situation (wedding ring, kids), what kind of car you drive, apparent level of sophistication in relation to the business — and in most cases you can casually ask visitors questions without disrupting the transaction.
The referrer header has always been optional, and nothing governs the specifics of its contents beyond the host and page. Google never had to provide anything beyond "Google search results page" as the referrer, and probably never should have. After all, the search page is a form, and form submissions generally use POST rather than GET. Both are valid, but if Google used POST then the search terms would never have been part of the url for the results page, and never would have been passed on in the referrer urls.
Yeah, marketing is bad right? Google is evil unless it takes something from marketers, then its good. The fact that startups use the same data, who cares? Actually, I think it will fuel some interesting innovation on traffic sources attribution, but I just can't comprehend this hate for "marketers".
perhaps, if you are so concerned about privacy, you shouldn't be using the site period; are you now going to claim that the site owner has no right to see how people use their website?
It looks like they just enabled SSL for all users, which is defnitely a GOOD thing. These SEO people are confused because I guess it limits some data they would've normally got, but it's a byproduct of them turning on SSL. It's not some campaign by Google to limit SEO data except insofar as it gets limited by SSL being on.
This isn't a byproduct of turning on SSL. If both Google and the site linked to by Google are secure, then the referrer would be intact, and contain the search terms, which is where Google Analytics and every other tracker has picked them up for 15+ years.
Google purposely added another level of indirection between their search result page and the search result links themselves. When you click, a bit of JavaScript rewrites the link to a script at google.com/url in order to wipe the real referring URL. Should Google want to do that for reasons other than masking search terms, like recording clickstream data from their SERPs, they could easily add the search term to that google.com/url link's query string. That's exactly what they do for ads -- they transfer search terms through the redirect's referrer even on SSL connections.
This would be amazing. Can you cite a source I can use?
I made the mistake of launching a [previously heavily optimized] site on the very week the secure switch took place. Explaining this to the client has been a complete disaster for me.
It's been this way for a quite a while though, now the only difference is that SSL is enabled for a larger number of people. Also adding the search terms to the url would defeat the whole purpose of using SSL for search at all.
If the whole purpose of these changes was to hide the search term from website owners, then turning on SSL everywhere wouldn't be required at all. That was already accomplished with the redirection between the SERP page and the clicked link, which was put in place before SSL was turned on everywhere.
There are other purposes. Like preventing governments from recording your long-term search history without a warrant. Frustrating that purpose requires SSL but doesn't require wiping the search term from the referrer as well. It would never be transmitted across an encrypted->unencrypted connection transition per the HTTP spec.
No I believe SSL is a separate feature that was always-on a while back (or at least optionally so).
This feature creates a redirect which hides the original keyword in the referring URL. Which has always been available at https://encrypted.google.com but now has become default at google.com.
The encrypted subdomain confused people, considering it's only adding a redirect, not more encryption.
Plus it's not like it would have really been helping the SEOs all that much. Though they really shouldn't have been relying on it as mush as they were clearly.
Not again. If marketers stop using the web there is no web. There is no Google, bing, twitter, Facebook, reddit,stack overflow, hordes of startups that use ads to monetize, plethora of blogs and therefore VC companies and whole startup ecosystem would bust.
Do I want a more efficient system of transfer of commercial information? Yes.
Is there is better system than marketing at the present moment? No.
One of my hats is a marketer and I am shocked how the industry is misrepresented here on HN and on reddit. The level of arguments is comparable too.
Any oral or written communication between humans or machines has a probability P to influence thoughts or opinions of human agents engaged in communication or machines that are able to learn.
Marketers are increasing P to reach their business goals. Actions taken to increase P may have positive or negative externalities. For example, edX uses Google Adwords and remarketing to spread awareness about free moocs from Harvard, MIT, etc. This is an example of an action with a positive externality. Thus I demonstrate that subset of marketers that are trying to change opinions is beneficial to society, even if they are pursuing their own agenda.
I respectfully reject your statement that you are not trying to change my opinion or opinions of other HN readers. See my first argument for justification.
Even the most heinous of brutalities can have positive externalities. You're really wrapped up in the defense of your point, but you're not arguing it effectively. I wish your example was relevant.
Don't worry, everything is going to be fine. This decision was made long ago. I remember the day. Nothing I or anyone on HN can do will change that.
However, I do take issue with your assertion that the web was built on marketing. It really wasn't.
Yeah, perhaps I am not arguing my point effectively. I am not trying to convince you, though. I am trying to convince other HN users that might see our discussion.
And I never asserted that web was built on marketing. Marketing is an emergent property of social systems in general and web in particular. You cant banish marketing, but you can amplify the good aspects. And to do that you've to have a rational discussion on the subject. Which I dont see even on HN.
Your condescension is not lost on me and thats fine. But your second line sounds just like something fictional illuminati member in a Dan Brown book would say. What day do you remember, again?
The day I remember is the day when the NSF announced their scheduled withdrawal from oversight of the Internet, and the corresponding end of the prohibition of commercial use of same. 1994?
You asserted a tight relationship between the existence of marketers and the existence of the web. The direct quote is visible above, but I see that it can be interpreted more than one way. Let's assume you meant to say that "the web would be radically altered if it wasn't an advertising medium". Then, we can agree.
You have a good point: the amplification of the good aspects of marketing is a worthwhile goal. It's not going away, so is it possible to make it suck less?
The problem with your argument, in my eyes, and the cause for my lapse into condescension, for which I apologize, is that your examples are so extraordinarily cherry-picked that they seem absurd to bring up.
Advertising, even at its best, is a distraction from my intent. This is generally a negative thing. It's pretty important, but I'm not so fragile that it's going to derail my whole afternoon. But it's not ingratiating, and it's not necessary. You were arguing for the "necessary" angle, and I think we won't reach agreement on that one.
Those big centralised choke points for information (Google, Facebook, Twitter et al) we could do without. I'd strongly favour a non-commercial distributed solution to those problems.
I generally don't like SEO and I'd agree with you if I didn't know so many clueless small business owners (I mean that in the least offensive way possible -- marketing is just not their strong suit).
So many people have something really cool to offer but their message is unfocused or overly complicated or just not elegant. Keywords offered a relatively simple and cheap way to see what people who might be interested in what they have to say/sell are looking for and hone their shop window to the world.
Now they get to pay Google for the privilege.
It is nice that Google is trying to get rid of the disease I just hope they don't take too much of the good flesh with it.
Did having the visibility of search keywords ever really help people "optimise" their page text for increased business?
I am far from an expert in this, but wouldn't you only get to see the search queries of people who found the site already? How do you get from that to knowing what to change? :)
I did always enjoy looking at the AWStats output for my old web pages to see what hilarious pornographic keywords people were using to accidentally find me.
People who ran websites used to be able to link specific searches to specific sales or subscriptions. That means you can assign actual revenue numbers to different groups of search terms. Now, you have no way to make that link. You know which search terms bring traffic to the site, but not which among all those terms are linked to your revenue.
With the first data set, you can see that people searching for "red widgets" buy your product, while people searching for "blue widgets" don't. You stop writing about blue widgets, make more content about red widgets, go out and blog about red widgets, run ads about red widgets, and both increase your traffic and your revenue.
With the second data set, you see "red widgets" and "blue widgets" are bringing people to your site. Maybe "blue widgets" even brings much more traffic, and you assume that's where your sales are coming from. You could spend months optimizing content, building audience and running marketing campaigns about blue widgets before you realize they weren't your customers at all.
The kind of query visibility they need for that has not changed at all. Google still reports search volume for specific sites in Webmaster Tools, and for the search engine as a whole through the Keyword Planner tool. It's only the ability to link search terms to individual people that's been lost... unless you pay for it, as advertisers still get that data.
No, it won't have that affect at all. They weren't creating pages based on search traffic they were already receiving, but search traffic they could potentially receive. You can still use the Google Keyword Planner to get lists of millions of keywords and their search volume. Webspam isn't going to be reduced by this change.
I'm not sure which scares me more. Marketers amassing surveillance data or the government. At least with the government, we might reign in the practice. Private industry seems to have some blessing from the public to do anything and everything profitable.
This is true, but not really. Governments only have sanction to enforce the law. Anyone can behave illegally or unethically. Increasing your surface area is not street smart.
Just the opposite is true. People that live in large cities (and subject to random crime) adopt a strategy of blending in, not one of "sticking out and relying on their lawyers"...
The article said they're making some attempt to anonymize the data. Anyhow whether that's wholly true or not, I don't like google having the data either. However, I do get that it's fair on some level since I'm directly using their service and it's been always apparent they monetize it through the data we inherently give them.
Where marketing and private intelligence bothers me is when it's not obvious to consumers when they're providing data. And also it's not always expected the ways in which it's used. And in the increasingly interconnected world of marketers selling and sharing data, the full implications won't be realized for some time.
edit:
Recently it's bothered me more and more that the software world tries to find hidden ways to charge users. It would be like highway construction companies tracking traffic patterns by embedding data sensors in their roads to collect consumer data and while they're at it, share it with the government to win some favors. Or like your carpenter going ahead and embedding sensors into the drywall to collect your data to resell and help them generate more revenue. People have paid for stuff directly for a long time and been happy to. But internet companies have this thirst to get millions of users by offering something for free, gambling investor dollars, and hope it pays off huge (which it occasionally does).
Google wants website owners to use Webmaster Tools so that it can connect websites with individuals. The end goal is to make it harder for people to manipulate search results in a way that would result in an inferior user experience for searchers.
For many years, Google has been a gold mine to a profession called "SEO". It still is, but every single day it is getting harder to extract value from it if all you do is trying to get your site to the top of the SERPs. Eventually it will be very expensive to trick Google and this is why I see many professional SEOs starting work at big brands or doing consulting for them. It's easier to get good rankings for a site that provides great value to users than for another spammy endeavor.
I was also involved in this business many years ago, but mainly to rank my sites that were getting traffic anyway because of the value they provided to users. Instead of being angry, I think that SEOs should be thankful that Google made it possible for them to take advantage of their service and even helped them along the way — I am. There is nothing wrong in charging people money for services that they use to make money themselves.
SEO should always only be a tool that helps you reach users with your quality product, not your main way of making a living because in itself it creates no value to society.
Ugh. Someone is wrong on the internet and I just can't ignore that.
SEO doesn't create value for the society? That is an extremely bold claim. Guest posts, infographics and other form of viral content are a very common tools for so these days. They create immense value for content consumers and that's the point. SEO is very much white hat these days in the anglosphere due to superior Google search spam detection algorithms. There are black hats, of course but they are not a defining market force for a long time already.
"If a website is accessed from a HTTP Secure (HTTPS) connection and a link points to anywhere except another secure location, then the referrer field is not sent." http://en.wikipedia.org/wiki/HTTP_referer whose source is "Hypertext Transfer Protocol -- HTTP/1.1: Encoding Sensitive Information in URI's (RFC 2616 § 15.1.3)". IETF. June 1999. Retrieved 2013-03-20. "Clients SHOULD NOT include a Referer[sic] header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol" [1]
Edit: I see comments further up indicating that Google redirects from SSL to non-SSL, thereby obfuscating the referrer and that this is new. If so, I'll have to take back this statement, sigh.
The search term is not in the referring URL, SSL or not. Google removed it from organic search, by a combination of moving it out of the query string into a URL hash, and by adding a level of redirection between the search result page and the linked website.
Ahh true. Instant did change things, didn't it? I'm still used to omnibars that hide the URL from me on Google searches that I forget it's not all /search?q= any more. Good point, good point.
Though, given Google's native apps, perhaps this was inevitable? I mean, there was always a percentage of traffic that would seem organic yet be from Google...
>It's a contradiction, according to Rishi Lakhani, a search consultant:
>... their idea of privacy is ridiculous to say the least. You can't offer privacy, but still SELL the data to AdWords advertisers. It's the same user. It's the same action.
The idea that clicking on an ad is the same thing as clicking on a link is where I'll have to disagree with this guy.
This is a pretty shallow analysis of his point. His point is that insofar as it related to the privacy of propagating keyword information, the action of a user clicking an ad or an organic search result doesn't seem any different. If the keywords a user searches for is private information, then why should this privacy be off the table when they click on an ad?
If this is about privacy, what about stop capturing search data at all? Eliminate all user personalization and search history, as if you weren't logged in. No more privacy issues, NSA court orders, etc.
Of course, this won't ever happen. This is not about privacy. This is about posing as a company that cares about privacy. It's easier to deal with some upset marketers rather than governments pushing for stronger regulations and anti-tracking measures, given the total lack of self-regulation (third-party cookies, anyone?).
FYI search terms have been disappearing since a Google update in October 2011. More than 95 percent of keywords have been marked as "Unknown" on my sites for the last 6 months.
You can still make a good guess at what terms are leading people to your site by the number of impressions on Google you get for a certain search term. To discover this go to Analytics > Acquisition > Search Engine Optimization > Queries.
So do unto Google as Google does unto others, in a way at least.
I perform all my Google searches while remaining logged out from Google. More importantly, I use Firefox/Chrome plugins that rewrite Google search results minus their tracking. In a way, I've gone "dark" to Google too.
The fact Google is still providing this data to people who do paid advertising shows this isn't about privacy at all, but is about closing off a valuable source of information for some other reason, most likely just under the guise of privacy.
At a high level, it's not clear that this type of information being "leaked" to sites was ever anything other than a function of the way Google worked from v1, where the query was passed as a GET parameter. So it's debatable if this was a good design decision in the first place. But Pandora's box was opened. Due to this design decision an absolutely immeasurable about of code has been built around this source of information (for good and evil) and calling this a "data apocalypse" isn't really that hyperbolic for people relying upon it.