Key target GIG technologies include:
Very large scale data storage, delivery, and transmission technologies that support the need to index and retain streaming video and other information coming from the
expanding array of theater airborne and other sensor networks. The target GIG supports capacities exceeding exabytes (10^18 bytes) and possibly yottabytes (10^24 bytes) of data.
So someone had a vague plan for a storage centre and couldn't put a finger on how much it would store to within 6 orders of magnitude! I don't count this as evidence that the NSA has more storage capacity than everybody else on the planet put together.
So yottabytes may be feasible, but it's a ways out there.
That would be enough to store a few years of everyones life in considerable detail : videos, photos, phone audio, GPS, financials, dna, web surfing, ...
You would probably need to take the complete visual and audio stream that went into a persons eyes and ears over the course of a year of their life, to start to fill that up, and do that for every living person.
I just wonder how much of that data they really need to sift through, in order to find likely terrorists ?
Finding terrorists is a red herring, in this case it's all about mapping and measuring the movements of every person and device.
Maybe NSA is doing something undiscovered, but this feels more like sensationalism then realism. I'm sure they have a lot of storage and can probably rival google on data storage, but I doubt they have some sort of magic god-like storage and search facility
So... how do they do it?
Assuming they were consuming 10 Tbit/sec of inbound data it will still take about 30,000 years to fill a yottabyte of storage.
With that, it would only require 1.639*10^6 cubic meters.
> as a 2007 Department of Defense report puts it, the Pentagon is attempting to expand its worldwide communications network, known as the Global Information Grid, to handle yottabytes (1024 bytes) of data.
Sounds like the yottabyte is a global/total capacity or a figure they aim to be able to handle in the future instead of a figure that they're currently hitting.
That said the Wired article just cites a DoD report which say their goal was to have Global Information Grid handle yottabytes when Utah was up and running.
Except you'd probably be wrong. Your filesystem almost certainly can't handle that much space. ext4 maxes out at 1 Exabyte. NTFS might barely make it if you use GPT and oversized sectors and don't run into any unforeseen problems.
Maybe the NSA has access to one of those high-density holographic optical storage systems we've been hearing about this years?
Based on data from [http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-...], a back of the envelope calculation shows that if the Utah data centre was really storing even a single Yottabyte using SD cards, they'd need 90 floors of servers.
Going off the only public images I could find of the Utah centre, it appears to only be 2 or 3 floors. If this is the case, then NSA electronic storage is at least 30 times denser than current commercial tech.
I looked around a little bit. Heck if I could find what comes after "yotta"
1 quad = 1 terabyte (10^3^4 bytes)
1 kiloquad = 1 yottabyte (10^3^8 bytes)
1 megaquad = 10^3^12 bytes
1 gigaquad = 10^3^16 bytes
And so forth.
While in Star Trek no direct comparison between quads and bytes is ever explained, this scale seems appropriate.