

How Big Data Became So Big - sew
http://www.nytimes.com/2012/08/12/business/how-big-data-became-so-big-unboxed.html?_r=1&hpw&pagewanted=all

======
crayola
The buzz around big data, let's not forget, is partly fuelled by bullshit
generated by vendors of big data solutions and analytic firms.

It probably does not really need to be repeated, but I guess it cannot hurt.
Data being big rarely causes it to be much more interesting. There can be much
more actionable information in a few hundred records than in billions,
depending on context, proper experimental design, and so on.

Also, an often overlooked point is that precision of statistical estimators
(e.g. width of confidence intervals) typically scales with the square root of
sample size; yet the cost of processing an additional observation is typically
constant (e.g. with distributed systems like hadoop). So in many cases there
are decreasing returns to data size.

------
edhallen
Good look at the history of the marketing term big data. I think this fits
squarely in the realm of big data hype, or at least a review of why it is big
data hype (something I've blogged about here
[http://www.klaviyo.com/blog/2012/07/16/the-curse-
analytics-b...](http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-
big-data-hype/)). The SAS discussion makes it especially clear that it's
viewed as a trend.

But - posts like these frequently jump to the top of hacker news, and there's
clearly a feeling that access to new analyses and more data is going to change
the world. I think this is probably right. I'd love to hear more discussion of
how big data has already started this process.

Examples I can think of (and I'd love to hear more): \- weather analysis for
farmers (weatherbill / the climate corporation) \- marketing (the target
pregnancy example - [http://www.nytimes.com/2012/02/19/magazine/shopping-
habits.h...](http://www.nytimes.com/2012/02/19/magazine/shopping-
habits.html?pagewanted=all))

More examples?

~~~
asanwal
CB Insights (<http://www.cbinsights.com/mosaic/>) is assessing the health of
private companies using public data.

B2B sales teams, supply chain/procurement professionals and investors use
CBI's private company analytics to sell more, purchase smarter and invest more
intelligently.

disclosure: I'm the founder so I'm very biased. We're hiring btw.

------
drucken
Painful reading an HN front page article like this. An entire article around
use of two-word (marketing) nomenclature with no informative value in the
article at all...

------
Tichy
I'm already getting allergic to the term. Every couple of months there is a
new term for the same thing to fuel a new hype cycle (machine learning,
computational intelligence, data mining,...).

Not that it isn't an important thing, and certainly more and more data is
becoming available. Now lets get back to work.

