
Search startup closing: what should I do with 1PB of social media content? - blockch4iner
Wanted the advice of HN here.<p>I have a search company I&#x27;ve run for the past decade which I&#x27;m shutting down in the next 60 days.<p>We had a good run but got hit by 2-3 major issues at once which makes the business model non-viable in the long term.<p>We have &gt; 1PB of content. Mostly social media content.<p>This is all public data. Nothing private or nefarious like private email addresses or passwords.<p>I don&#x27;t want to just delete it but I also realize it&#x27;s insanely expensive to  move around.<p>I also think it would be very irresponsible to just publish it openly.  There are plenty of bad actors who would love to do something if they could get this much data for cheap&#x2F;free.<p>However, the data is very valuable to researchers.  Especially in light of the  2016 election.  We have social media data for the last 4 years.<p>I want to preserve it but want to do so responsibly.<p>Any suggestions?<p>I&#x27;m also blockch4iner@gmail.com if you&#x27;d like to reach out directly.
======
blockch4iner
I'd like to not democratize bad actors. By publishing it free I'm taking
something that could cost like $300k to build and giving it away for zero.

This prevents malicious actors that are underfunded from doing something with
it.

------
londons_explore
If it really is public data, just make a big torrent file and publish it for
free.

Bad actors would have just scraped it anyway.

------
deca6cda37d0
[https://archive.org](https://archive.org)

------
londons_explore
On physical disks? FedEx it to the web archive?

