

Ask HN: Show me how to perform raw data analysis more effectively - ghotli

I've been a ruby guy for a few years now. The elegance of the language is the biggest draw for me. Unfortunately, I feel like every time I pull a ton of data from sql, or a set of flat files in to memory that I'm plagued by GC collection pauses and that it takes _much_ longer than it should to process the data.<p>That said, I'm looking for a better language or at least a better framework for just doing exploratory data analysis. We've done some big data transformations with hadoop (pig) and post processing the output with a little bit of C. Those datasets were absolutely huge and the problem warranted using something a bit more heavyweight like that.<p>For the small / medium sized datasets I'm looking for something a bit more elegant. What good tradeoff between speed and elegance have you found? Got any supporting code samples to make your case?
======
maxdemarzi
It all depends what you want to do. "Exploratory data analysis" means a lot of
different things. If the data is already in a relational database then I just
stick with SQL. You can do a lot with it and a rapid visualization tool to
help make sense of it. Try something like Tableau
<http://www.tableausoftware.com/>

------
starkfist
Your question doesn't make sense unless you give some examples of what you
want to get out of the data.

