
Backing up a Linux system to Usenet - Tomte
https://arstechnica.com/information-technology/2017/02/bored-with-ho-hum-cloud-backups-use-usenet-yes-usenet-instead/
======
jasode
_> With access to a Usenet news server, you can simply upload your backup
there, and it will be stored redundantly in news servers all over the world.
Best of all, this approach typically costs considerably less than a cloud
backup service._

I saw no mention about "retention policies" in the article. For newsgroups
with binaries, the _other_ Usenet peer servers can choose to download only
text and ignore binaries. Or they only hold binaries for 30 days or whatever
discretionary time period they choose based on available disk space.

With AWS Glacier, Backblaze, etc the data retention would be explicitly
specified.

[1] example of limited retention period:
[http://www.giganews.com/blog/labels/retention](http://www.giganews.com/blog/labels/retention)

~~~
problems
Most paid Usenet services these days advertise their retention time of > 1000
days.

Some of the big providers in the business right now:

[https://www.easynews.com/usenet-plans.html](https://www.easynews.com/usenet-
plans.html) 2600+ days

[http://www.supernews.com/](http://www.supernews.com/) 2300+ days

[http://www.news.astraweb.com/](http://www.news.astraweb.com/) 3000+ days

Easily enough for some backups. Of course, there's no guarantee of integrity,
but it might work for some people.

~~~
jasode
_> Of course, there's no guarantee of integrity, _

Exactly. It's been a decade since I've used Usenet but I remember there were
always tons of messages complaining, _" part43.rar and part62.rar is missing
please reupload!!!"_

As another anecdote, I personally posted a huge list of AWK one-liners 20
years ago to Usenet (comp.lang.awk) and I can't even retrieve that message.
That's just pure ASCII text (probably less than 2k) and even the Google Groups
(dejanews acquisition) archives don't have it.

~~~
therealmarv
Maybe you have not read the article closely. You normally have a bunch of
.par2 files (my knowledge is also 10 years old) which can repair your missing
files/parts when the amount of loss is smaller than the size of all (non
broken) .par2 files.

~~~
quirkafleeg
> You normally have a bunch of .par2 files

vol003+04.par2 and vol031+32.par2 is missing please reupload!!!

Before any flippancy haters downvote, this is _exactly_ what happened on
Usenet. Shitty servers are shitty, whatever the file type.

~~~
calvano915
par2 data could be uploaded but also kept on local backup or traditional cloud
backup providers.

------
bb101
This has to be a joke, surely? A good example of the Tragedy of the Commons...

As ProfessorGuy succinctly commented: > Why not an article on how you can get
a bed for yourself in the local hospital so you won't have to pay rent? Hey,
if they're going to build a public institution like a bunch of suckers, they
deserve to be taken advantage of!

~~~
aw3c2
This would be a drop in the bucket compared to the terabytes of warez uploaded
daily.
[https://en.wikipedia.org/wiki/Usenet#Usenet_traffic_changes](https://en.wikipedia.org/wiki/Usenet#Usenet_traffic_changes)

~~~
WillPostForFood
While there are different, and maybe bigger ethical issues there, at least it
is using the system as designed.

------
ycmbntrthrwaway
Encrypting your backup with a passphrase and making it public is a bad idea.
It is not forward secure. Anyone can try to bruteforce it offline. You better
use a keyfile and backup it somewhere else, such as your other computers,
external drives etc. And even then, publishing your backup is a bad idea due
to possible breakthroughs in cryptography. No crypto is secure _forever_ , you
better assume that it will be broken in the next 5-10 years.

------
ashark
Related-ish-ly, I've had a notion bouncing around in my head that it should be
possible to build a fairly complete social networking interface on top of
email and ical (or similar), and some kind of contact-management standard,
such that the whole thing (or at least enough of it) could be rebuilt from
_e.g._ your (possibly dedicated to this purpose?) email account when
necessary, possibly in stages—say, a new installation of the application
builds what it can from the last 30 days in a few seconds, then displays a
populated interface while it continues digging to some arbitrary point in the
past (until it exhausts caching resources it has been allotted on a given
platform, basically).

An initial search has revealed a couple abandoned partially-complete efforts,
but that's it.

~~~
teddyh
I believe that p≡p is aiming to implement something like that in due time.
They had a talk at FOSDEM this year:
[https://fosdem.org/2017/schedule/event/pep/](https://fosdem.org/2017/schedule/event/pep/)

------
kosma
> For most users, however, you’ll probably find it costs you around $10 per
> month to maintain the Usenet service in this manner.

Which is more than a dedicated cloud backup service like Backblaze or
Crashplan. Genius.

~~~
wsha
If you read the whole article, the author mixes in caveats throughout and says
that it requires a special set of requirements for the usenet backup to be
worthwhile. At some point, he addresses your point by saying that in some
cases (possibly requiring careful coordination between all of the servers) you
could backup many different servers with the same usenet account whereas
services like Backblaze and Crashplan usually restrict the number of machines.

That said I still think the point you make is the most important critique.
Online storage/backup is such a crowded market (Google, Amazon, DropBox, Box,
Backblaze, Crashplan, Carbonite, Mozy, tarsnap, Spideroak, rsync.net,
OneDrive, etc) that if you go with something significantly cheaper than all of
the mainstream options you are probably going to get what you paid for.

------
thieving_magpie
>Bored with ho-hum cloud backups?

Well I can't say that's something I've ever thought.

------
comstock
Another idea:

Split your data into ping packet sized chucks and throw them out on the
Internet. Keep all your data bouncing around on the Internet and use that as
your storage provider.

Not really practical, but a neat idea. Saw in the excellent "Silence of the
wire":
[https://www.nostarch.com/silence.htm](https://www.nostarch.com/silence.htm)

~~~
Florin_Andrei
Point a laser at the empty sky, modulate it with your backup file.

It's gonna last _forever_.

------
benzinschleuder
Some time ago, I implemented a little tool for myself which backups folders
incrementally on usenet (deterministic message-id creation from a secret key,
append-only style with metadata, parity, encryption etc. so you only have to
remember one unique key to access all of your data, even if new data is
added).

It can mount the current state of the usenet backup with FUSE and it's
possible to browse through the files and listen to music etc. I understand
that it might not be a good idea to store all your data only on usenet, but I
thought that was an interesting concept and a fun little project to work on :)

~~~
ycmbntrthrwaway
Open source or didn't happen.

~~~
benzinschleuder
at the moment its very tailored to macOS and has plenty rough edges because
its for personal use. But I'll gladly clean it up a bit and put it on github
if enough people are interested and/or i will write-up how its "protocol"
works. Just let me know (twitter pm or sth.) who is interested.

EDIT: basically it's one deterministic stream of messages for journaling
everything, one recursive stream of folders and linked raw files.
deterministic lookup rougly like: HMAC(type|index|revision|replication,
key_for_locating) and it iterates through that

EDIT2: and one for parity of meta and raw

------
buzer
Reminds me of

"Only wimps use tape backup: real men just upload their important stuff on
ftp, and let the rest of the world mirror it ;)" \- Linus Torvalds

------
cestith
From Amazon Lightsail I can get this for $5 a month:

* 512 MB Memory

* 1 Core Processor

* 20 GB SSD Disk

* first 1 TB of outbound Transfer

That's a whole VPS I can put a git repo on and upload encrypted files to using
ssh. So I get gigabytes of storage, secure transfer, a commit log, and can do
some light other work on it. Or I could just rsync or scp them there.

GitLab is even one of their preinstall-able images, or you can pick some other
application/stack (MEAN, LAMP, Nginx, Drupal, Node.js, Redmine) or a bare OS
(Amazon Linux or Ubuntu) and initialize your own bare repo.

For the $20 mentioned in the post, I could set up four boxes in four AWS
zones. I could configure my backup scripts to do a duplication rate of only
two full copies and have roughly 40GB of stuff backed up fully duplicated in
physically disparate data centers.

This isn't even getting into elastic storage offerings or paid cold storage
which is dirt cheap by comparison. This is just simplified point-and-click
deployment of predefined images, and it's already far more reliable for the
money.

If one really wants to spend monthly to rig up their own backup solution
rather than just using Amazon Drive, Google Drive, or Dropbox or something
similar, this at least makes the stuff easy to find, redundant, and simple
enough without polluting the public forums.

If one really needs two or three year data retention offsite with multiple
copies and wants to only pay for the upload and download without paying for
storage in between, just buy outright some 32 GB USB flash drives for $8 a pop
and mail them to three friends. There, you've paid $30ish one month for
hardware, postage, and taxes and will pay one friend $1 in postage to get the
data back.

~~~
boomboomsubban
>For the $20 mentioned in the post, I could set up four boxes in four AWS
zones.

And for the $2.50 also mententioned, you could set up zero. Or you could buy a
bandwidth based plan, where $20 will get you at least 250GB of bandwidth.

~~~
Fwirt
It's actually quite possible to get an unmanaged VPS for under $2 a month if
you shop around. I'm currently renting one for use as a personal email server
for £0.002 per hour, which works out to around $1.85 a month for 1 core, 1GB
ram, 20GB storage, and 2TB bandwidth.

[https://lowendbox.com/](https://lowendbox.com/) is a good resource for
finding cheap VPS offerings to mess around with.

------
83457
Idea: An open source peer-to-peer backup service where data is encrypted and
backed up in pieces across countless systems around the globe. No one person
had your data but in some way it would be guaranteed your data would always be
available. Everyone who uses the backup service would be required to also
accept data pieces for backup.

~~~
lb1lf
It is called BitTorrent. Generate the backup file as described in the parent
article, give it a saucy name involving a (female) celebrity and let it out
into the wild. As disk space is effectively free and people tend to hoard
files, I suspect you'd have no problems recovering your file a year or two
down the line.

(I tried this way back when using WinMX (anyone remember THAT?) to see whether
it would work. It did.)

We dubbed it 'hardcore backups.'

~~~
lozf
You could even throw a little steganography in to the mix, so those who check
for the content they want, but pay little attention to file size continue to
participate in storing your data!

~~~
83457
Was thinking the exact same thing. Win-Win

------
tdkl
Seems like someone got the task to write something controversially stupid to
get traffic, because if you proposed such scenario to any real life company,
you'd be laughed out of the room instantly, disregarding "20 years of
experience".

------
dillonb
While interesting, this seems like a bad idea. You're uploading your backups,
no matter how encrypted, to a place where they will be publicly available to
download.

~~~
problems
Most cloud backup services are worse - they do no client side encryption, your
files are freely available to the service provider or anyone who can break in.

I'd be much more comfortable with this personally. Trust the math, not the
people.

~~~
therealmarv
Exactly. I rather trust well proven math more than people or infrastructure.
One famous example nowadays is Bitcoin ... nobody was able to break the
fundamental math behind it.

~~~
organsnyder
Depends on how long you want your data to be private, though. There's no
guarantee that the encryption won't be broken in a decade or three. And, even
if it's not mathematically broken, increased computing power (quantum?) could
make brute-forcing fairly trivial.

~~~
ycmbntrthrwaway
> quantum

irrelevant to symmetric crypto

~~~
aaronmdjones
Not irrelevant, it is my understanding that it can still cut the effort
required considerably.

If you are using only a 128-bit key, a quantum computer can cut the brute-
force effort required to 2^64, which is feasible today.

------
ourcat
I've often thought that Usenet binaries would be perfect for podcasts.

------
ddp
This is one of those ridiculously pointless exercises that keep me coming back
to Hacker News for more. What a waste.

[http://www.tarsnap.com/](http://www.tarsnap.com/)

------
therealmarv
Has anyone experience with that in practice? What are the data limits on this
kind of backups?

~~~
loeg
It works well enough. You can find binary providers that charge _by download
quantity_ and use that style of plan to upload for free. (And upload is
usually free -- usenet providers want more content, after all.)

Name your backups something inconspicuous and have your client generate an
.nzb file for later recovery (has the unique mail ids for each component mail
in it).

It's a little risky in that maybe providers would delete your backups if they
identify a pattern, or if you name them wrong some studio may issue a (bogus)
DMCA takedown against them.

Tarsnap is cheap enough that I wouldn't bother with any of this unless you're
already using a binary usenet account anyway.

------
storrgie
this kills the usenet

------
apricot
Also, if you ever get bored putting junk mail in your recycling bin, why not
mail it to the author of this article?

