

Microsoft TerraServer (1998) - dictum
http://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx

======
fsckin
I dug around a bit and found a research paper[0] about TerraServer. It's got
all sorts of neat details not in this link.

TerraServer hardware configuration parameters:

Max hits per day 40 million/day

Max SQL queries per day 37 million/day

Max image downloads / day 35 million/day

Bandwidth to Internet 200 Mbps = 2 Terabytes/day

Concurrent web connections = 6,000 connections

Web front ends: Six 4-way 200 Mhz Compaq Proliant 5500, .5GB ram

Database back-end 1 8-way 440Mhz Compaq AlphaServer 8400 10GB ram, 3.2 TB
raid5

 _324 9GB Ultra SCSI disks_

324 x 9GB disks for 2.9TB? You could store 1.2PB with the same number of 4TB
disks. Doesn't seem like Moore's Law is holding up for storage, nor for
internet speeds.

[0]
[http://www.math.uaa.alaska.edu/~afkjm/cs401/project_fall_04/...](http://www.math.uaa.alaska.edu/~afkjm/cs401/project_fall_04/terrarep.pdf)

~~~
frozenport
I wonder if 324 on Lustre would have more throughput than a 1 4TB disk?

~~~
fsckin
They would be leaps and bounds faster. Especially comparing random IOPS.

Let's just assume they were using a Seagate Cheetah 9LP. That's a 9GB, 10k
RPM, 1MB cache UW SCSI disk that was widely available in 1998, for a
whopping... $1200 each. $390,000 worth of disks... wow. That could buy 1300 *
4TB SAS drives today with a total raw storage of 5.2PB!

324 * 9LP's would have around 5.5GB/sec of total read/write throughput. That
easily beats a single 4TB disk with around 200MB/sec read/write performance
and terrible IOPS in comparison to even a single 9LP.

SSDs compare much more favorably here. 10 x 1TB SSDs for ~$6500 could easily
reach 5.5GB/sec throughput and triple the capacity of the 324 x 9GB drives.

~~~
mariusz79
You're comparing off-the-shelf disks to enterprise-class storage.

~~~
theatrus2
"Enterprise" 4TB disks don't have orders of magnitude more IOPs than consumer
versions.

~~~
mariusz79
But they are more expensive, so for comparison to be right one needs to take
that under consideration.

------
mentat
This was so much fun to use. I built perl scripts around this (learning at the
same time differences between lat/lon and UTM) to map wireless access
points[0][1].

0\.
[http://wigle.net/phpbb/viewtopic.php?t=9](http://wigle.net/phpbb/viewtopic.php?t=9)
1\.
[http://wigle.net/gps/gps/main/download/](http://wigle.net/gps/gps/main/download/)

~~~
eksith
Thank you for sharing this.

Do you have a snapshot archive since you first started this? I wonder how much
the landscape has changed since then.

~~~
mentat
I can see if there's anything in any of my backups. Nowadays I'd just use
Google Earth's history feature (optionally feeding in time based KMLs of
access points). I think the WiGLE interface supports time filtering, but I
don't know if that's officially exposed.

~~~
eksith
Thanks! It probably isn't exposed, but it would be nice to see these grow over
the years. It may even expose other things like how community hubs have
shifted and what demographics are being catered to. Overlays from census data
or other insights might reveal some interesting things.

------
ryanburk
> Tom Barclay, Robert Eberl, Jim Gray, John Nordlinger, Guru Raghavendran, Don
> Slutz, Greg Smith, Phil Smoot Microsoft Research and Development

a number of those folks went on to build some impressively large scale
services. hotmail, skydrive, etc. I wish jim was still around...

~~~
andyjohnson0
Wired article from 2007 on the use of satellite imaging in the search for Jim
Gray.

[http://www.wired.com/techbiz/people/magazine/15-08/ff_jimgra...](http://www.wired.com/techbiz/people/magazine/15-08/ff_jimgray?currentPage=all)

~~~
rob-alarcon
I didn't know about this, is a Sad story.

------
bnolsen
The company I worked for back ended some of these datasets. Definitely a bunch
of usgs imagery. At the time several racks of solaris servers were required to
serve up the imagery using tomcat for the requests. Those machines were in
canada. Windows was used for arcgis and oracle. Shortly after opteron was
released and ibm came out with a 64bit jvm I moved the whole thing except for
the esri portion over to a simple load balancer running apache and two
mirrored dual opterons running fedora 64bit (2002). A total half rack which
handled about a million image reqs a day or so. Each req involved decomposing,
reading and stitching image mosaics together and sending out in under 10s.
Unfortunately we were back ending more than just terraserver so I don't
remember who had what specs. With all the memory alloc/dealloc going on inside
the java process we were fortunate if it didn't require a full restart every
12 hours.

The funniest part of the whole thing was that I rewrote the engine from java
over to a c++ stitcher and got request times under 1s on a dual p3 600. That
was bypassing the esri server, oracle database and java stuff. At that time
the umn mapserver started getting interesting as well although their image
processing was glacial at the beginning. And then I was retasked and I left
before being able to finish up and deploy.

~~~
codeulike
Wait, so you're saying that the Terraserver project that was supposed to show
Windows and SQL Server capabilities was back-ended by a third party company
using tomcat, oracle, java and linux. That doesn't make any sense. The OP-
linked paper describes how it was all done with NT, IIS and SQL Server.
[http://msdn.microsoft.com/en-
us/library/aa226316%28v=sql.70%...](http://msdn.microsoft.com/en-
us/library/aa226316%28v=sql.70%29.aspx#msterra_hardware)

------
corywatilo
"A terabyte holds 250 full-length movies. It is a lot of data." =]

~~~
eksith
My, how times change!

Come to think of it, a terabyte is a lot of data still (as is a gigabyte).
Just not for movies these days though.

~~~
MichaelGG
Most torrents at 720p (and seemingly fine quality to me) come in around a GB.
So it's enough for almost 1000 movies.

~~~
RobAtticus
Good quality 720p should definitely be more than a gig for movies. Even high
quality 480p rips came in at over a gig for most movies. I'm sure the torrents
are watchable, but there are sacrifices being made somewhere.

------
chris_wot
Funny how things change a bit. EMC now owns Clarion. DEC no longer exists. EMC
now owns Legato Nerworker. Seagate's backup management group was bought by
Veritas, who then merged with Symantec. Such is the potted history of storage
companies!

------
codeulike
I remember this. I think at the time they said just by turning the thing on
they were doubling the amount of internet-accessible data in the world.

------
CervezaPorFavor
Yet another entry in the list of missed opportunities.

But I suppose it's easy to criticise people with the benefit of hindsight.

~~~
jaggederest
I don't think it's wrong to criticize them for missing this as an opportunity.

I remember playing with it at the time, it was really revolutionary. There was
literally nothing like this available to the general public before this.

And who owns mapping these days? Google. It's pretty sad that Microsoft wasn't
able to turn a six year head start into a reasonable business.

~~~
santoshalper
That's sort of been the story of Microsoft for the past 15 years or so, lots
of cool ideas, then a failure to execute and make a great product out of them.
I wonder if young engineers these days realize how formidable Microsoft used
to be. There was a time when having MSFT in your rearview mirror meant you
were in deep shit. Now, they're just another big IT company.

------
wiradikusuma
Reading "..combining five terabytes of image data.." and "Figure 1. The
TerraServer hardware" while looking at the portable 2TB external HDD sitting
on my desk... how technology advances rapidly. Makes me wonder how (storage)
technology can be in 10-15 years.

~~~
mariusz79
Yet our smartphones can't seem to get over 64GB limit. Personally, I'm afraid
that the cloud will effectively slow-down the progress in the storage space,
at least in the consumer portion of it.

~~~
akgerber
SD cards are available with up to 256GB of storage. They just cost $400 or
so[0], so I don't think enough people see the value of paying for that much
storage in a phone.

[0][http://www.amazon.com/Lexar-Professional-256GB-Memory-
LSD256...](http://www.amazon.com/Lexar-Professional-256GB-Memory-
LSD256CTBNA600/dp/B0090BEWKY)

~~~
mariusz79
What about the phones that come without sd card support?

~~~
akgerber
The flash memory chips used in phones are essentially the same as those used
in SD cards, and phones often use SDIO for internal communication. The SD card
is just a point of reference for the cost and availablity of flash memory
unbundled from a phone.

------
bluedino
10GB in that Alpha server seems like a massive amount of memory at the time.
But DEC claimed capability of up to 28GB. An 8400 configured the way they had
it was probably a half a million dollars, not including the disks of course.

You can see how far behind Intel hardware was at the time - the quad-CPU
Pentium Pro boxes only had 256MB of memory! Sure, that was only a $40k box
from Compaq...

Fun fact: At the time, Quake engine licensees were buying 4-way Alpha and
Intel servers to do the level processing for first person shooters.

------
dmead
i was so confused about the big deal with google maps when this had been
around for like 7 years.

~~~
estebank
I remember both. Google maps was by far much more enjoyable to use. Just like
between Hotmail vs GMail.

------
zmonkeyz
I remember trying to visit the maps on this thing way back in the day. It was
always broke.

