
Digital hoarders: Our terabytes are put to use for the betterment of mankind - feross
https://arstechnica.com/?p=1667508
======
joyj2nd
I hoard because shows, youtube videos (youtube-dl) and websites disappear. I
use some GTD methods to organize my data (e.g. folders a,b, c...z) I use
recoll to find data
[https://www.lesbonscomptes.com/recoll/](https://www.lesbonscomptes.com/recoll/)

I would like to use an online backup service but I would like to have my data
encrypted.

I tried [https://www.hetzner.com/storage/storage-
box](https://www.hetzner.com/storage/storage-box)

But data throughput to an encrypted storage was abysmal. I tried several
methods, even included running an encrypted filesystem in a 10TB file. I tried
[https://www.cryfs.org/](https://www.cryfs.org/) but nothing worked good and
reliable.

I looked into
[https://www.infomaniak.com/en/kdrive/](https://www.infomaniak.com/en/kdrive/)
but the encryption method seems like snake oil.

Any suggestions?

~~~
kcolford
I use restic with backblaze. It works great and super cheap. Kinda slow, but
you can always order one of their portable drives and ship it back to them for
the initial upload.

~~~
joyj2nd
60 USD for unrestricted per year? Can I store 10 TB there?

~~~
balladeer
You can. I haven't checked their products in a long time. Look out for:

[1] Old file versions and deleted file retention - default was 30 days I
think. You pay extra to make it a 1 year, and more to make it forever forever.

[2] Keep connecting the external drives or lose that data. Because the last
time I checked if you don't connect your drive (or maybe the computer/laptop
too, not sure) regularly the files are marked as "deleted" and [1] will kick
in even though you'll keep paying (unless you are paying extra).

[3] They didn't have restore option from the app - either use their website (I
am not sure how the partial/interrupted restores were handled from a browser)
or they will send a restore drive to your home (it was free iirc).

If it works for you you can give the service a shot. Their Mac app, at least,
is really really good (way better than competition including the relic called
Time Machine).

------
shantara
I don't understand the negativity the article associates with "hoarding" data.

It is the nature of the internet that any content or website you see can
disappear at any moment without a prior warning or an explanation. From an
author deleting their earlier work they were later dissatisfied with, to
copyright claims, to people losing interest in their hobby project, to site
owner forgetting to extend their domain, to website becoming impossible to
find in search engines.

It always pays to be properly paranoid and have backups in case the worst
happens. Storage is cheap, so what's exactly the downside?

My personal archive may not bring "the betterment of mankind", but it saves me
from disappointment when looking up older content that has since disappeared,
and also allows me to help other people find what they're looking for.

~~~
zouhair
When you think about it is quite insane that you can find rock tablets from
thousands of years back with text on them but we can lose all the data on the
internet just like that.

~~~
freehunter
Is that insane? It’s not like every stone tablet or parchment or scroll
survived, many were lost. Some were lost immediately, some after a few years,
some after a few centuries. Not every word on the Internet survives for long
but not every word written on stone tablets lasted very long either.

------
hashhar
I "hoard" because there is a lot of media I own that I can't find elsewhere. I
had stopped buying physical media and ripping them to my disks in the last 2
years and moved full time to streaming - Spotify for discovery, Apple Music
for maintaining the library (both are cheap here in India).

But recently I saw that a lot of music I owned was missing from both services.
That made me decide to go back to my old ways. Also, having spent the effort
in cataloging and ripping my collection has made me intimately familiar with
my collection and has been an activity I can partake in to relax/wind-down.

~~~
callmeal
>But recently I saw that a lot of music I owned was missing from both
services.

It's not just that, it's also that apple "transparently" changes the version
of the music that you "bought". I had a version of a musical album with a
particular song in it. When my ipod died and I re-downloaded that album on my
new computer, I found that they had helpfully "upgraded" it to the newer
release (without the song I wanted on it). And there's no easy way for me to
get it back unless I find a physical copy somewhere and buy it again.

~~~
Mediterraneo10
Added to that concern is that streaming services often feature only recent
versions of albums that are pitched as “remastered”, and they might replace
earlier audio with whatever the label now sells. The supposed remastering is
not any improvement on earlier digital-era releases, it just reduces the
dynamic range because labels know that people are listening to music on their
phones in noisy places these days. So, if you have a nice stereo and listening
environment and you want to get a version of the album with actual room to
breathe dynamic-wise, your only choices are buying a used CD or torrenting it.

------
throw32098
I do not like the shaming tone of this article. Petabyte server today is a
small box that fits anywhere. Meanwhile some people have hundreds of shoes or
clothes, but nobody calls them "hoarders".

Anyway, I have offline storage, because my internet connectivity is slow. I do
not like waiting, so all my content is downloaded overnight with a few cron
scripts.

~~~
mft_
People with lots of clothing and shoes and other things absolutely might be
criticised for their behaviour, and may be called hoarders. (I have direct
personal experience of such people.)

There is definitely a mindset —at its most extreme, a pathology— associated
with hoarding behaviour.

Digital hoarding _may_ mostly avoid the negative life consequences of hoarding
—as you say, you can store a lot of data in a relatively small physical space—
but this doesn’t change the nature of the psychology behind the behaviour.
(Although I’d wonder how many digital hoarders really have a small neat modern
NAS as you imply, versus a room full of old racks?)

Interestingly, the rationalisation given in the headline (“betterment of
mankind”) isn’t so very far away (although maybe more grandiose) than the
rationalisations that physical hoarders offer to justify their behaviour,
around the perceived intrinsic value of their belongings.

~~~
pessimizer
> this doesn’t change the nature of the psychology behind the behaviour.

The "psychology" behind the behavior isn't the problem with hoarders, it's the
actual hoarding. The reason we try to change hoarders' thinking is because of
the physical consequences of their hoarding - we define this as illness
because of its distressful results, if it had no consequences it wouldn't be
illness.

edit: I think you're accidentally using circular logic to rationalize a bit of
magical thinking (person who obsessively collects garbage is ill, therefore
person who obsessively collects anything is ill.) It's not the obsession
that's the problem, it's the garbage and the distress.

~~~
Mediterraneo10
I know that some of the members of the organized data-hoarder community are
spending huge amounts of money on disks month after month, and it really
bothers their spouses. So, even if there is no physical mess being created in
their homes, their obsession is still affecting their lives and other people
around them.

------
JKCalhoun
Yes, hoarder here. Started with scanning old family photos, progressed from
there. Was psyched to find youtube-dl after I had discovered many of the
YouTube videos I had playlisted years ago, are now missing. Perhaps not very
interesting things to most: research footage testing some aspect of the X-20
Dynasoar space plane, etc. Many things gone missing I don't even know what
they were since YouTube doesn't indicate what the missing video was. So, yeah,
hoard it is. "Japanology"? Got all I could find. "How It's Made?" Yep. Live
footage of the band "The Embarrassment" sleeps on my drives as well....

~~~
netjiro
2nd youtube-dl

Feels insane to store youtube vids, but quite often they are simply gone for
one reason or other after a few years.

------
jccalhoun
I'm definitely a digital hoarder but not on the level of the people in the
subreddit. It started when I was in grad school and a web site I wanted to
reference went down before I was able to archive it and it wasn't in
archive.org and youtube wasn't a thing. So I started downloading copies of
anything I thought relevant to my research. I know I have some rare stuff that
is gone from the internet. One of these days I'll get around to posting them
on archive.org. Until them I practice 3-2-1 backup to make sure I don't lose
anything.

------
germinalphrase
My only regret about digitizing my music over the years is that I didn’t care
about the quality as much when I was younger. I’m older now - with better
equipment - and would love if everything was FLAC rather than mixed .mp3

Years ago, I was a member on a private file trading site that only exchanged
old and unusual art films. The kind of stuff that is entirely non-viable
commercially and will never be found on a streaming device. It shut down - but
I miss having that small corner of the Internet.

------
sneak
Just picked up two more 36 bay sata 4Us for the garage on eBay - only $500
each!

Shoot me an email if you have bulk old/used sata drives for cheap. I’m buying.

~~~
leonroy
Are multiple used smaller drives really more cost effective than fewer new
larger drives once you take into account cost of power, HBA, backplane and
lifetime of used drives?

I'm a data hoarder too (proof):
[https://i.imgur.com/j9jk9OP.jpg](https://i.imgur.com/j9jk9OP.jpg)

But always been curious about cost so let's do some sums...

With a used 3TB setup. Assume 12x3TB = 36TB storage. A reasonable 2U server
would be needed to support that which would include a backplane and HBA. Power
consumtpion would be 12x 5W (HD) + 10W (HBA) + 65W (server). Total is 135W.
Power cost in UK for one year at 24/7 is approx £176. Assuming used 3TB drives
give us two years of reliable operation and cost £30 each to buy (going price
on ebay a few months back) total cost to buy would be £360 of just the drives
(not including the server). Add cost of electricity for two years operation
and we get a total of £712. Average the cost per year (£712/2) and we come to
£356 per annum for the 3TB drive setup.

Now with new 12TB drives. 3x12TB consumes 3x 5W of power + 30W for the server
(no backplane or HBA needed). Total is 45W. Power cost in UK for one year at
24/7 is approx £60. Cost of drives would be 3x£300 = £900. Assume we get 5
years of life out of these new drives, plus electricity costs of £60 - over 5
years total would be £1200. Averaged the cost per year (£1200/5) and we come
to £240 per annum for the 12TB drive setup.

(I have not counted the additional cost of the 2U server for the 3TB scenario
since that would complicate the math since servers tend to last longer than
drives.)

So I'm wondering what the appeal of large servers with lots of small drives is
when back of the envelope math suggests it costs more in the long run. Unless
of course it's because you like those blinkin' lights :)
[https://i.imgur.com/4ypumBQ.jpg](https://i.imgur.com/4ypumBQ.jpg)

~~~
m463
The calculations might depend on whether the drives are powered on all the
time.

~~~
omgwtfbyobbq
That's my sense too. Unless someone has to have access to that data relatively
quickly all the time I don't see why the system couldn't be left off.

Even if the data is accessed frequently, lets say once every day or few, it
could still be left off most of the time if someone is OK with waiting a few
minutes for everything to power up.

