BigQuery is the best interface for it. Can resolve queries on the entire dataset in less than a few seconds (however, you only get 1TB processing free per month. Since the full dataset is ~285GB, you only get 4 queries per month. Plan ahead on the May 2015 dataset, which is only 8GB.)

That's only true if your 4 queries need to read every single column.

One of the big advantages of BigQuery's column-oriented storage is that you only pay to read the columns that are actually needed to answer your query.

For example, this query to extract the top 10 authors only cost me 19GB to run (and took 7.0s):

  SELECT
    author,
    COUNT(*) AS COUNT
  FROM
    TABLE_QUERY([fh-bigquery:reddit_comments], "table_id CONTAINS '20' AND LENGTH(table_id)<8")
  GROUP EACH BY
    author
  ORDER BY
    COUNT DESC
  LIMIT
    10;

  author,COUNT
  [deleted],228425822
  AutoModerator,3677774
  conspirobot,575576
  ModerationLog,547671
  autowikibot,402076
  PoliticBot,388395
  imgurtranscriber,360248
  dogetipbot,358093
  qkme_transcriber,301968
  TweetPoster,293309

But it doesn't specifically address collection and distribution of API results as a dataset.

(From the page:)

A licensing agreement is required in order to:

* use the reddit API for commercial purposes. Use of the API is considered "commercial" if you are earning money from it, including via in-app advertising or in-app purchases. Open source use is generally considered non-commercial.

* use the reddit alien logo ("snoo") in your app or for its thumbnail. Any new apps you create must be approved as well before usage. The circular "r" logo is reserved solely for use by reddit, Inc.

"You retain the rights to your copyrighted content or information that you submit to reddit ('user content') except as described below."

And the exceptions just state that Reddit has a perpetual irrevocable worldwide license.

The most downvoted comment[2] is ironically in iAMA, by a mod of iAMA (ironic because of the recent drama).

I'd find the top ten most upvoted, but I ran out of free bandwidth on BigQuery :(.

[0] https://gist.github.com/alexggordon/7b56353dcf8044a7a5f9

[1] https://drive.google.com/file/d/0Bzxo-UKxFmN-eWticy1BR2tCRDQ...

Relationship between Reddit Comment Score and Comment Length for 1.66 Billion Comments [0] and the Github repo [1]. Reddit cliques N°2 - deeper into the subs [2]

[0] - https://www.reddit.com/r/dataisbeautiful/comments/3cjyvb/rel...

[1] - https://github.com/minimaxir/reddit-comment-length

https://bigquery.cloud.google.com/table/fh-bigquery:reddit_c...

I improved on it a lot with a whitelist of subreddits and some machine learning to select the best thread. But I was only touching on what is possible with that data.

https://kiwiirc.com/client/irc.snoonet.org/mybots

EDIT: Reddit changed their api, give me a moment to fix it.

I saw one analysis of subreddits that checked for negativity. /r/PathOfExile came in as one of the most negative subreddits, which could easily be turned into a narrative about gaming culture.

I get this screen

Welcome to BigQuery! What is BigQuery?

I just did a query for my reddit handle and it took 6.5 seconds to retrieve all of my comments. Kind of a snowden moment but this is super interesting, first time I played with Big Query.