

Ask HN: Big Data Software, What's in it? - notastartup

Hey guys I keep hearing a lot about &quot;Big Data&quot;. I have no idea what it is other than people doing sentiment analysis on Twitter, and Business Analytics Software.<p>Is this a niche that a freelancer could take up? If so, what are the prerequisite knowledge or credentials you need to earn?<p>Also what is this big data software about anyways? It seems to me just a fancy charting&#x2F;graphing software. Is there an open source version of it?<p>Curious to learn more.
======
karterk
Big data, in my opinion, technically should be data that's atleast 5-10 TBs.
But that hasn't stopped people from calling data that's an order of magnitude
smaller to be "big data".

Example of big data could involve transactions, logs, click stream, sales
data, or even GPS data (say in telecom).

If you're looking for some opportunities here, a good safe bet would be Hadoop
and HBase. You can treat them as open source implementations of the original
Google papers on Map Reduce and Big Table respectively.

Today so many companies are realizing that they have data locked up in logs
and other largely static databases that they can analyze using map reduce
jobs. You can generate reports, trends, patterns and more from these analysis
and can use them to guide business decisions.

