
Google+ Is Built Using Tools You Can Use Too - planb
http://highscalability.com/blog/2011/7/12/google-is-built-using-tools-you-can-use-too-closure-java-ser.html
======
xentronium
> Google+ Is Built Using Tools You Can Use Too: [...] _BigTable_ , _Colossus_

Right.

~~~
tmcw
This. This article is half-speculation about the simplest parts of the stack
that Google still isn't revealing ('probably uses Java servlets').

Google is doing a pathetic job of transparency and an even worse job of open-
sourcing the real parts of its infrastructure, and from everything I've read
so far, it's hurting them as much or more internally than it starves the
developer community.

~~~
andos
Can you elaborate? I don't understand why they should open-source their
infrastructure.

~~~
tmcw
For the same reason why everyone else should open-source their infrastructure
and why others do - Yahoo's Hadoop, Facebook's Open Hardware, Rackspace's
OpenStack. Because others are building similar technology and cross-
pollination (and honest trials) of BigTable with, say HBase, or their web
stack with node, would mean that they break out of NIH and start improving
both the state of the industry and their own code.

That said, I'm not saying that they need to, but that nobody should credit
them with transparency or open-source goodwill in terms of actual core stack
when what we're actually seeing is zero transparency and lots of in-house
closed code. To the point that even what they're using is a question mark, and
our guesses like BigTable often meet with a smirk and an 'oh, no, we're on to
InternalProjectX, which is so much better.'

~~~
andos
Thank you. Like others, I believe Google keeps its infrastructure secret
because that is a competitive advantage. Their infrastructure is their
business. It makes sense for Facebook to openly publish its datacenter design
because that's not what Facebook is about: they could be running from AWS for
all we care -- the social graph is kept very much closed, however.
Opensourcing seems to be the radical approach to commoditizing complements.

But I would love if Google opened up about that, anyway. I am an avid reader
of their service failure post-mortems because they reveal a tiny bit of what
goes on inside their data centers. And because the way Google stuff fails is
admirably beautiful.

~~~
DennisP
Here's an ex-Google engineer who thinks Google's software infrastructure is
obsolete, compared to opensource offerings: <http://rethrick.com/#waving-
goodbye>

~~~
nostrademons
When this was discussed on Quora, a bunch of other Googlers chimed in and said
basically "no, not really":

[http://www.quora.com/Google/Is-Googles-software-
infrastructu...](http://www.quora.com/Google/Is-Googles-software-
infrastructure-obsolete)

I wasn't one of them, but I agree. Google's internal infrastructure has its
warts, but it's generally light years more stable and reliable than anything
you can get open-source. I have a teammate that worked with Hadoop in previous
jobs; he said that there's no comparison between it and MapReduce. I used JSON
for my previous startup's core data storage; were I to do it over again, I'd
use protocol buffers, which are much, much more efficient. I use both Closure
and JQuery on a daily basis; I consider them as occupying different niches,
and wrote a bit more about that last night:

<http://news.ycombinator.com/item?id=2758317>

~~~
DennisP
Interesting, thanks.

------
famoreira
I wonder if the Closure library (and related tools) will gain more traction
now that it has been used for Google+. Any reason why more companies/projects
use them?

~~~
udp
Yes, I'd actually never heard of Closure until I read this. I wonder how it
compares to jQuery/jQuery UI?

~~~
ma2rten
EDIT: It is definitively only for full fleshed java-script web apps, not for
quickly adding some gimmickry to a static webpage.

As it happens I just started looking into it a few days ago. Here is what I
found:

The closure compiler really is it's biggest advantage. It compiles your code
to really optimized javascript, with not one character too much. Only the
parts of the library that you can actually use are compiled into your app.
Also the library is pretty much extensive. For example their i18n code looks
quite good.

The downside (in my mind) is that it forces a quite verbose java-like
programming style on you (classes with getters/setters etc.). You also have to
annotate your code with type information (a la javadoc) to take full advantage
of the compiler. Also the DOM APIs are really ugly. They basically look like
the bare browser apis.

Their is not really a big community around it either. But also means that
nearly every line of code is actually used in a google application.

I havn't decided yet if I should use it for my next project.

~~~
murz
> You also have to annotate your code with type information (a la javadoc) to
> take full advantage of the compiler

I consider this an upside, especially on large JavaScript products with many
engineers contributing (like the ones Google builds). Having your JavaScript
checked at compile time helps ensure quality code, and helps to avoid errors
at run time.

~~~
ma2rten
I guess you are right, especially if you work in a team and want to build well
documented, maintainable code. On the other hand it can be a bit tedious, if
your main goal is to build a rapid prototype kind of thing.

------
nikcub
the most interesting part of this was the first comment, which was a rebuttal
of the ex-googler blog post talking about how new open source projects and
ideas are more advanced than the 'dated' (his quote) Google stack.

~~~
carbonica
The ex-googler who compares Protocol Buffers to MessagePack is either lying
about his understanding of Protocol Buffers, or hasn't looked at MessagePack
beyond a simple glance.

First of all, MessagePack throws up a benchmark on their front page bragging
about how it can deserialize _3 integers and a string_ faster than Protocol
Buffers. Lovely. Of course, besides being a silly microbenchmark, that's in an
array - not a keyed map, and since your protocol is going to evolve, you'll
need a keyed map. Since MessagePack serializes map keys _with_ the message,
you're going to take a big hit there. Let alone when you have a protocol that
grows over several years and has to support a service with over a hundred
keys. Eventually, your string keys will become too big of an overhead, and
you'll switch to integer keys. Then you have to roll your own translation
layer between your space-saving integer keys and useful names for your keys.
It's roughly at this point you wish you had picked the "creaky" solution which
has done the right thing™ for almost a decade (though of course not open
source for that long).

------
jsilence
"Your neighbours' Ferrari is built from atoms you can use too!"

SCNR!

The title somehow implies that everyone can have their own Google+ clone with
little effort. Which is unfortunately not the case.

-jsl

------
chime
> HTML5 History API

There is a bug with their API that I can't replicate on demand which
absolutely messes with the history in latest Chrome. It happens sometimes when
I go to someone's posts page, then about page, and then try to go back. It
keeps switching between posts and about pages. When I click-hold on the back
button, I see the wrong titles for many of the pages I visited previously but
the links work. Pretty weird and annoying.

------
nl
_Google probably uses a custom Java servlet container_

Google has previously released their old servlet container: [http://google-
opensource.blogspot.com/2009/01/opengse-releas...](http://google-
opensource.blogspot.com/2009/01/opengse-released.html)

I believe they use Jetty internally (certainly that is what AppEngine uses)

------
galactus
Java servlets? I wonder how the average HNs user (who loves google+ and hates
the java platform) feels about it.

~~~
nl
There is no "average HN user" - and I think most people here realize how
important Java is to pretty much any company that is of interest on HN.

People who spend a lot of effort hating Java (or any platform) on general
principle don't know what they are talking about. The Java platform has a lot
of problem - and they are worth discussing - but mindlessness biases are
pointless.

