
Official Google Blog: "This site may harm your computer" on every search result?? - Anon84
http://googleblog.blogspot.com/2009/01/this-site-may-harm-your-computer-on.html
======
mechanical_fish
_Unfortunately (and here's the human error), the URL of '/' was mistakenly
checked in as a value to the file and '/' expands to all URLs._

Wow, one character. That was one expensive character.

~~~
mechanical_fish
Incidentally, I'm glad to know that I can now tell my clients that I have a
_Google-class_ QA procedure: I make my changes directly on the live site,
without testing them, and I promise not to blow up the home page for more than
an hour!

Six sigma, baby!

Seriously, though: We've all made similar mistakes before (though, after the
first few times, we usually try to catch them on the dev server or the staging
server). The trick for Google will be to ensure that this one is not made
again.

~~~
kwamenum86
I'm not sure it's practical to test this on a dev server with each update. All
they really need to do is make sure "/" or sites ending with "google.com" and
maybe some other white listed sites are never included in the file.

~~~
mechanical_fish
_I'm not sure it's practical to test this on a dev server with each update._

I tell myself that all the time. And then I pause, and sigh, and load up the
dev site anyway, just to make sure the home page has not exploded. Because
_sometimes it has_. Typos happen. Brain farts happen.

Obviously I wouldn't necessarily expect Google to test a dev server by hand
every time they change some code. That procedure is for little people like me.
[1] For something as important as Google I'd expect there to be a procedure
whereby each new change gets pushed to a small group of boxes, which then run
a few really simple automated acceptance tests ("If I do a search for a random
term, do I get back a page with links that can _actually be followed_?")
before the thing gets pushed live fifteen minutes later.

Apparently I expect too much. Perhaps I'm overengineering this. Maybe it's
okay if, a couple of times a decade, we just cripple half the Internet for 30
minutes and give 5% of the world's computer users a virus scare.

\---

[1] At least until I get lazier (in the Larry Wall sense of the word) and
write some more scripts.

------
kwamenum86
This could be a PR nightmare for Google. On top of the Epic Google Fail, they
could not even come up with an accurate blog post:
[http://blog.stopbadware.org/2009/01/31/google-glitch-
causes-...](http://blog.stopbadware.org/2009/01/31/google-glitch-causes-
confusion).

It will be interesting to see the stories that come up when the regular news
cycle begins again.

~~~
nostrademons
Interestingly, the StopBadware blog is now down - I can't get a response from
it. I guess they got GoogleDotted.

~~~
kwamenum86
Well StopBadware got screwed twice then. First, their site was overloaded
because every Google user who attempted to view a search result url was taken
to the interstitial page, which links to their site...bad news. Their site is
still down as of this comment: <http://stopbadware.org>

Now I suspect their blog is down because Google linked to their blog.

The blog post basically pointed out some inaccuracies in the original version
of the Google blog post.

------
snprbob86
This is a risk you run when you have _continuous_ deployments. Luckily,
incremental roll out, sufficient unit testing, and other safeguards mitigate
the problem, but sometimes things are still going to slip through. I'd bet
that significantly man more problems have occurred, but this is simply the
first big, obvious one worth talking about.

This goes back to what I was preaching earlier about Microsoft and Google's
varied approaches to testing. If this sort of bug got burned on to a disk and
shipped to a million customers, it wouldn't have been a 40-minute problem...
Sadly, Microsoft's testing culture bleeds into its ailing web culture and is
becoming less effective in the modern auto-update world. See: Google grows
impatient, so they patch Chrome with a workaround for Hotmail. Meanwhile, the
Hotmail team is waiting for the next scheduled release. Lucky for Google,
Chrome is mostly useless without an active internet connection useful for
pushing updates.

------
jacquesm
Someone should make the case for a separate test / dev environment to the big
G :)

Seriously though, I wonder how many programmers / site builders are going to
use this as an excuse for their own goofs....

"Look, if google can be down for half an hour due to one slash, you really
shouldn't be angry with me for [insert some terrible mistake here]."

------
tptacek
Isn't StopBadware Niels Provos' Google 20% project?

------
cadalac
I wonder if that cost him his job.

~~~
a-priori
I sure hope Google is more forgiving than that... in the grand scheme of
things this is a very minor screw-up.

~~~
sgk284
As a former Google employee, I can assure you that there have been far bigger
screw ups than that... and it's just part of the learning experience. The
bigger the screw up, the more money you've invested in training them :) (i.e.
When I was there, one man single handedly instantly brought down 10,000
machines in a hard reboot... fortunately because of the way Google replicates
and distributes things, the world was none the wiser. It really gave me great
respect and trust in their infrastructure and no he wasn't fired.)

------
veghead
Wow. Someone broke the Google by slashing on it.

