

Data, data everywhere - helwr
http://www.economist.com/specialreports/displayStory.cfm?story_id=15557443

======
mootothemax
This is a huge problem for the intelligence services, and is an excuse
frequently made for why the CIA failed to stop 9/11 and MI5 7/7. With the
sheer amount of data logged and available to the services, how can one
reliably wade through such huge amounts of data?

Being perfectly honest, I wouldn't be surprised if a Google Translate-style
statistics package was in use right now. You, sir, fit the profile
GeekWithGuitarAndCat72.

------
netcan
_In recent years Oracle, IBM, Microsoft and SAP between them have spent more
than $15 billion on buying software firms specialising in data management and
analytics._

That's huge. Does that sound right?

~~~
shrikant
$15bn actually sounds conservative to me (I'm in the business intelligence
space).

Yep, check out [http://www.bi-
verdict.com/fileadmin/FreeAnalyses/consolidati...](http://www.bi-
verdict.com/fileadmin/FreeAnalyses/consolidations.htm)

 _EDIT_ : And that table doesn't even include IBM's billion-dollar SPSS grab.

~~~
david927
Wouldn't it be smarter to use a couple million of that to create a seed fund
for aspiring technologies?

~~~
dagw
Why not do both? The major disadvantage with the seed fund approach is that
you have no idea what you'll get out of it or how long it will take. If you
but SPSS, it might cost you more, but you know exactly what you're getting and
when you'll be getting it, which is a huge advantage if you feel you're in a
hurry.

~~~
david927
By "a couple million out of that [15 billion]," I meant both.

------
andr
The one thing shows like 24 get perfectly right is that intelligence and
warfare is now much more about computer scientists analyzing data than people
on the ground.

------
RichClaxton
Huge data sets is the new Oil, processing this new huge data is where the big
money will be in the future.

~~~
kixxauth
After the money made processing the oil there will be plenty more to be made
with the distribution networks and gas stations themselves. While processing
data is an interesting problem for geeks, the more mainstream problem will be
visualizing data. I don't go down to the refinery and pull my car up to the
spigot. I buy my gasoline at a pump where I can also pick up the milk and get
a cup of coffee at the same time.

------
spoiledtechie
It is very true. Intelligence Agencies are always wondering how to view the
data better. I know for one example, we have 150 TB's of video data coming in
every day from Iraq and Afghanistan and we are trying to find ways to view it.

One number my boss says a lot is the CIA wishes they had 14,300 Analysts to
view through all this information at once. These of course would be 14,300
people looking at video 24/7... Its a huge problem that doesn't have a
solution at all yet.

------
ThomPete
As always with data when it comes to dynamic areas such as economics.

"The map is not the territory"

------
ableal
This has been here a few times before:
<http://searchyc.com/Data%252C+data+everywhere>

(Just for the record, this seems the first time it got traction.)

~~~
kungfooey
I'm glad it finally hit the front page. It deserves some attention.

