

Website outages and blackouts the right way - amirmc
https://plus.google.com/u/0/115984868678744352358/posts/Gas8vjZ5fmB

======
corin_
Another, in my opinion, great example of the need for subdomains on HN
submissions. I surely wasn't the only one who thought this title meant Google
were endorsing blackouts?

~~~
pferde
Can't you just hover over the link and see the entire url on the bottom of
your browser window?

~~~
Tim-Boss
Not on my phone!

~~~
pferde
Good point, didn't think of that.

------
pierrefar
Hello all

I'm the author of this post. There are some suggestions for Javascript-based
alternatives. A well-implemented Javascript overlay for the blackout message
is a valid option, but keep in mind the following when thinking about it:

1\. Googlebot does run some types of Javascript. We blogged about this
recently: [http://googlewebmastercentral.blogspot.com/2011/11/get-
post-...](http://googlewebmastercentral.blogspot.com/2011/11/get-post-and-
safely-surfacing-more-of.html)

This means that the content included via Javascript may be indexed until the
next time we crawl and process the page and find it's gone.

2\. Consider how the JS will affect your pages' instant previews. Overlays are
likely to show up in the previews, and these will take some time to update as
we re-generate the previews.

3\. Consider your users. Some webmasters are suggesting keeping the blackout
overlay visible without any means of hiding it. That may not be the best user
experience, and may be annoying to your non-US based users.

I'm happy to answer any other questions.

Thanks, Pierre

~~~
wahnfrieden
You should be able to help googlebot index the right content by using
different HTTP status codes in the response. Also, although googlebot executes
JS, in my experience it doesn't use it for indexing content. My guess is that
it's primarily used for the instant preview and for verifying that you're not
cheating by hiding SEO keywords on load or whatnot. This could change any day,
of course.

------
tripzilch
Who is this guy now? I keep seeing announcement posted on Google Plus by
people speaking for Google, but they never bother to mention as much as their
function or position within Google.

They ought to start of with "Hi I'm XYZ, lead PQR coordinator of Google ABC."
or something.

At least this time it says "google" if you mouseover his photo, the last one
didn't even that.

~~~
johnbatch
You can go to his about page.
<https://plus.google.com/u/0/115984868678744352358/about> "I work at Google as
a Webmaster Trends Analyst. This is my personal profile.

What do I do at Google? I help webmasters build better websites, so you'll see
me talking to webmasters on our and other forums, writing blog posts, saying
things like "I'll ask internally" and the like."

~~~
larrys
"I work at Google as a Webmaster Trends Analyst. This is my personal profile."

Security wise a bad idea. For example email of a actual company employee (at a
company that offers email) might normally might be name@corp.yahoo.com or
name@corp.google.com not name@yahoo.com etc.

By posting in this manner nothing to prevent someone from posting the wrong
information.

------
wiradikusuma
If HN has "sticky posts", this thing should be on top at least until 18th of
Jan to avoid people from hurting their website's SEO too much for the
solidarity action.

~~~
amirmc
Until I saw this post, I hadn't even considered the possible SEO implications.

------
CWIZO
This "technique" is highly recommended for your error pages also (50x). It
also helps if you set the "Retry-After" HTTP header (value in seconds), that
tells google (and hopefully other crawlers) that they shouldn't bother with
crawling for another X seconds. Helps with the load if you are experiencing
problems with your server.

Retry-After is also usefull for "down for maintenance" pages since you usually
know how long your page will be down.

------
sambeau
This technique would not be needed if Google joined the blackout and didn't
crawl for a day.

~~~
random42
Even if Google joined the blackout, only the web frontend would be blacked
out, not the backend crawling.

------
insickness
Can someone explain how to change a site to return a 503 HTTP header?

~~~
jagira
Put _Redirect 503_ in your Apache configuration file.

~~~
pquerna
I wrote this up awhile ago for how to do this in pure httpd configuration:

[http://journal.paul.querna.org/articles/2009/08/24/downtime-...](http://journal.paul.querna.org/articles/2009/08/24/downtime-
page-in-apache/)

------
sambeau
Am I alone in thinking that if Google won't join the blackout than the next
best thing would be a Google search full of anti SOPA messages?

Will a 503 make it into the listing? If so be sure to put your message in the
title.

~~~
driverdan
Except that doesn't work. You lose your rank for your existing keywords if you
replace your site with an anti-SOPA message, which is what this is about.

If you just add a banner or message to your site and keep the rest of your
content intact then sure, it may show up.

------
PaulMest
Wouldn't it be possible to detect if it's a web crawler visiting your site
(e.g. user agent) and then let them crawl the site as normal? If it's not a
web crawler, then display the "blackout" version of the site?

~~~
wmf
That's cloaking and can cause you to be de-indexed.

~~~
PaulMest
Interesting. This site claims that the intent behind the cloaking is taken
into account: [http://www.smart-it-
consulting.com/article.htm?node=148&...](http://www.smart-it-
consulting.com/article.htm?node=148&page=103) ... Do you agree? Or have you
had a different experience?

~~~
pdkp
Yes, at least in the past, Google has taken into account intent when it comes
to cloaking, letting some big sites get away with it and still rank well in
serps.

However, I can see how your initial suggestion of showing full content to
Google, could be viewed as solely for preserving rankings in an artificial
manner.

As an example of cloaking, some News Sites let Google index all their pages,
while requiring actual users to login/register to view it.

Typically, if the user has a Google Referrer, they can view the page one time
for free and then need to login/register to view anything else.

Visiting the page directly or with a non-google referrer shows a
register/login page.

New York Times was one that does(did?) this. I stopped visiting them when they
started. I think Washing Post, or one of the posts, was doing it too, as well
as a number of other sites.

Experts Exchange used to basically be the same way, although I think they are
doing it differently now, and they were slapped by Google a long time ago for
cloaking, so changed to a different method of cloaking...

------
adrianwaj
Any day we're set on, Black Friday? Weekly, bi-weekly, first of every month?

~~~
Tim-Boss
Afaik this is all for the 18th January 2012.....or Wednesday as we mortals
call it

~~~
adrianwaj
Not enough. I think there needs to be something preventative in place so this
sort of thing can't raise its ugly head again.

If they want censorship, let's give them censorship.

~~~
driverdan
You're free to blackout your own website for as long as you want. No one is
going to stop you. Good luck with your regular traffic/visitors disappearing
though.

~~~
adrianwaj
A little bit of medicine can cure you, a lot can poison you.

------
dugmartin
While I'm no fan of SOPA I think people that are outraged enough to blackout
their site but and at the same time are concerned with SEO implications remind
me of protesters that don't want their pictures taken. You want to show up and
feel good but don't want anyone to know about it later.

Why not set the page titles and meta descriptions for all your pages so that
"This site was blacked out in protest to SOPA" shows up in your search results
for a while? That would honestly probably have more impact and Google will get
around to indexing you again.

~~~
corin_
It's not the same as not wanting your picture taken on a demonstration, it's
the same as not wearing a tshirt for the next year that says "I was on this
demonstration".

Doing a blackout for twelve hours harms your business, fucking up your SEO
harms it again. Obviously in this, as in any protest, there is a decision to
be made on what to do. Why not only blackout for one hour? Why not blackout
for an entire month?

------
jbhelms
Since GoogleBot doesn't run javascript, does that mean that an all Javascript
solution would work without disrupting indexing?

~~~
cbr
It does run javascript: <http://www.jefftk.com/news/2012-01-11.html>

~~~
jbhelms
That is good to know. I guess this helps catch SEO cheats with javascript

------
infocaptor
Another alternative is to simply use jquery like framework to create a modal
dialog window and display the message. Hide all the close buttons. You can put
this script in the wordpress header and then all pages will show the dialog
when the document is done loading. This has the least impact on your domain
and content wise.

~~~
adrianwaj
I wonder if a JS redirect would work too to some central page that any number
of people can use (maybe a 302 redirect would be better.) It can display the
desired page in an iFrame. I don't know what could be on that central page,
but it could get a ton of traffic if done right. Donations, petitions, forum,
guest book, chat, politicians contact details.. etc. Perhaps it can be
displayed as the requested url somehow.. is that possible?

The other thing it could do is redirect back to the source site every x
visitors, so one could set the ratio: 1/3 visitors gets the redirect. Search
bots don't get the redirect.

It'd also make a good point for media coverage, providing some metrics on the
effect of the blackout.. so long as people use it.

"xx,xxx internets were censored in the last yy seconds/minutes/hours from zz
domains"

