
Backblaze: to decrypt your backup, send us your private key passphrase - noja
https://help.backblaze.com/hc/en-us/articles/217664798-Security-Question-Round-up-
======
tyingq
The title is obviously editorialized and making a statement of some sort.

But the linked page is a long FAQ that seems reasonable after a cursory read.
It's not the ideal setup for everyone, but they don't seem to be hiding
anything.

What am I missing here?

~~~
nailer
They should provide the encrypted backup to you, you decrypt it yourself.
Backblaze should never see your unencrypted data.

~~~
tyingq
They seem to be saying it's not that way because their customer base is more
aligned around convenience. So it's structured where they can't see your
data.... until/unless you need a restore.

Is there another well known service that does this completely right? (honest
ask, not a challenge)

Edit: sounds like there isn't anything in their budget price range that does
it well either. Sounds like it should be a goal to provide it, but hard to
fault them too hard.

~~~
rocqua
Tarsnap is a service known for being done right. It's build by the former
security officer of freeBSD.

It's fully aimed at power-users though. I'm sure all of my family would be
more secure on backblaze because they couldn't set up tarsnap right. For
myself though, I use tarsnap.

~~~
shawkinaw
By my math, Tarsnap is about 10x too expensive for my usage. The $5/mo I pay
for Backblaze would get me 20 GB of storage on Tarsnap and no bandwidth,
that's pretty poor.

~~~
rocqua
Back when I decided, backblaze didn't work nicely with headless linux. If that
changed I might consider changing.

Currently I only use it to back-up the non-huge parts of my home directory and
/etc. This keeps costs reasonable to me.

I know these prices are pretty close to the actual cost of storage, which is
amazon S3. I suppose backblaze can do better by handling their own storage.

~~~
cookiecaper
There are lots of options to store to most cloud backends nowadays. Check out
rclone and duplicity.

------
CodeWriter23
I don't understand why BackBlaze would decrypt on their premises rather than
offload that processing to the client computer. I mean they've shaved every
other cost in their system to the bone to achieve a low cost product for the
consumer, but the one area where the rubber meets the road, they chose to
incur the cost of decryption.

I understand the vital convenience tradeoff of storing the private key. I
might be willing to make that trade too if not for the various 'bleed
vulnerabilities known and yet to be discovered, that threaten to disclose my
passphrase to evil haXor$ when the time comes to transmit my passphrase to BB.

And when you combine those two concepts together, it really only leads me to
the conclusion that this is defective by design in order to support
surveillance requests. Even ones that can only be satisfied when the user
types in their PEM passphrase. Prove me wrong by implementing that so-called
"FINAL improvement" noted at the end of that KB entry.

~~~
tyingq
My guess is that their average customer is backing up data that's on the same
drive that the keys would be on. Such that the keys would be lost in the same
event that would have the customer wanting a restore .

There are many ways around that, none very elegant. They picked what they felt
exposed the least amount of awkwardness. Only expose a flaw in the somewhat
rare case of a restoral.

~~~
brianwski
Brian from Backblaze here.

> they've shaved every other cost in their system to the bone to achieve a low
> cost product for the consumer

In addition to making sure you have a backup you can restore from, we always
want Backblaze to be: 1) fast (low impact on your system performance), 2)
easy, 3) inexpensive. You only focused on the inexpensive part. To be super
easy, we need to be as easy as it is to sign into your Gmail account ->
username and password and that is it. Then for the more security conscious
(just like Gmail) we offer two factor authentication, and go even further than
Gmail goes by providing the ability to set a private passphrase.

> their average customer is backing up data that's on the same drive that the
> keys would be on

This is correct. We could require the customers to remember their own private
encryption key on a USB thumb drive and store it at their bank in a safety
deposit box, but that would SEVERELY decrease the number of customers
Backblaze has.

> this is defective by design in order to support surveillance requests.

Backblaze has never had a surveillance request. Plus, if you set a private
passphrase we literally could not comply because we simply could not decrypt
your data. We really, REALLY don't want to know what is in your backup and we
take great pains to not know. Seriously, it is a liability to us to know what
is in your backup. It is a liability to have the ability to decrypt your
backup.

You can also check out our team at
[https://www.backblaze.com/company/team.html](https://www.backblaze.com/company/team.html)
and ask around about us. We have been in the same 30 mile radius in Silicon
Valley for 25 years (working at Apple computer, Silicon Graphics, HP, Oracle,
GE, etc), and we have been doing online backup for ten years now. We take our
customer privacy VERY seriously, and our personal reputations are extremely
important to us. We are the good guys and we try our best to do right by our
customers.

~~~
CodeWriter23
You will be able to comply with a surveillance request WHEN a customer has to
restore. Also, you've done nothing to address the risk eavesdropping due to
'bleed style exploits while transmitting my passphrase to BB servers. The
passphrase should stay on my hardware period.

~~~
brianwski
> address the risk eavesdropping due to 'bleed style exploits while
> transmitting my passphrase to BB servers

You are correct. There is a "window of decreased security" where _IF_ there
was a zero day hack on the same day you prepared a restore -> your data might
be compromised.

> The passphrase should stay on my hardware period.

For some customers and some data, that is true. If you would PREFER to lose
your data than to have even a 0.000001% chance of that data being read by
hackers then I totally agree with you. A good example is if you will
immediately be arrested and sent to jail for the rest of your life if the
information in your files is ever discovered, then you should keep the
passphrase on your hardware (or preferably only in your head and never written
down).

But let's say the data is simply all your pictures of your dogs and your
vacations? That type of customer might value increasing the chance of getting
the data back over the ultimate security solution. Having a recoverable
password lowers security (because anybody who gains access to your email can
now access your data) but it increases the chance of data recovery. People
forget passwords sometimes.

Different customers have different preferences, and Backblaze tries to offer
several valid choices and let the customer decide which one to use.

~~~
CodeWriter23
You're really conflating issues. Yes, Store the Passphrase-protected private
key on your server. No, do not send my passphrase to your server for a
restore. Yes, send the private key to my client. No, do not decrypt my data
server side. Yes decrypt it client side after I enter my passphrase into my
client. No, don't transmit my passphrase anywhere at all.

By arguing the passphrase should be sent to BB servers, you're unnecessarily
violating two fundamental principles of security, least access and least
privilege.

------
2bluesc
Currently to restore your data you need to provide your private key:

> As you prepare a restore, you must type in your private passphrase into the
> restore server.

Seems like a pretty fundamental architecture fail here.

Later the site goes on to acknowledge this:

> We actually want to improve this to provide a password encrypted ZIP file
> for download, and then the FINAL improvement is to actually allow you to
> download the private encryption key, download the encrypted files, and
> provide the pass phrase in the privacy of your computer. We hope to add this
> functionality in the future.

~~~
cryptarch
> ZIP

Are they serious? Security conscious users a) still have to give up an
encrypted copy of their private key, ripe for cracking, and b) they are
supposed to download all their terrabytes of data in zip form, presumably over
HTTP with no resume option?

Smh...

~~~
lol768
Why does ZIP imply:

* Unencrypted HTTP

And why is it that HTTP implies no resume (Range header has existed since HTTP
1.1 if I recall correctly)?

~~~
brianwski
Brian from Backblaze here.

> Range header has existed since HTTP 1.1

Backblaze built the original zip restore functionality in 2007 and I believe
the range header was introduced after that? We FULLY support the range header
in our B2 Object Storage and we actually have support for it (now) in the ZIP
file download functionality.

However, because we built the client side custom program bzdownloader (our
restartable restore downloader) it seems to still succeed in some situations
where a web browser doesn't succeed. We can slowly retire our custom
bzdownloader application as all web browsers catch up and have restartable
downloads.

------
hackerhasid
i don't understand why they went through all this trouble of encrypting things
client-side if they're just going to store the private key on their own
servers! what actual benefit is there to this service? the fact that data is
encrypted in-transit? i assume every backup provider does that via https/etc!

~~~
rocqua
The private key can be encrypted by a password, presumably this encryption is
done client-side.

This means the data is also secure in their datacentre at rest. That is, until
you provide backblaze with your password to actually access your data.

Essentially, until you need the data that is backed-up, no-one can get to it.

~~~
jlgaddis
My SSH keys are also protected by a passphrase, but you certainly won't find
me uploading them to the Internet.

~~~
rocqua
If it uses pbkdf2 with sufficient rounds and a decent password (say 5 words
from diceware) it should be perfectly safe to upload the keys.

Better to keep them offline and keep ownership of the key as a second factor
certainly, but encrypted keys can be a fully acceptable single factor.

------
timcederman
BackBlaze's lax approach to security and unreasonable requests for what data I
should provide for them to debug a bug in their client is why I switched to
CrashPlan.

(said bug also caused me to exhaust my 1TB Comcast cap several months in a
row, BackBlaze was unresponsive and unapologetic throughout the whole process)

~~~
cryptarch
Do you happen to know if CrashPlan's encryption is any better?

AFAIK AES in CBC-mode is a no-no because it betrays which blocks of data are
identical, and that's what BlackBlaze uses.

~~~
rocqua
That is not the case. That would be ECB mode. CBC stands for cipher-block-
chaining. Where you xor the plaintext with the ciphertext of the last block
before encryption specifically to prevent identical blocks from yielding
identical ciphertext.

There are some issues with CBC regarding the fact that it can't really be
parralelized. It also lacks authentication as opposed to the newer GCM mode.

In general though, AES-256 CBC is essentially THE standard best practice
proven form of encryption.

~~~
cryptarch
Oh, thanks for the info, I mixed those up.

------
brianwski
Brian from Backblaze here, glad to answer any questions. To clarify, Backblaze
produces four different products/modes for different customers with different
needs and requirements. We want customers to choose what is appropriate for
them. One size does not fit all:

1) Online Backup ($5/month) where every file is encrypted on your laptop
BEFORE being sent to Backblaze and your backup is secured by your
username/password - where you can recover your password if you have access to
your email account. (We support two-factor auth which provides an additional
optional layer of protection.)

2) Online Backup ($5/month) where every file is encrypted on your laptop
BEFORE being sent to Backblaze and your backup is secured by your
username/password AND your private encryption key is secured by a "passphrase"
that is not recoverable in any way, shape, or form. (Two-factor auth is also
optional here.)

3) B2 Object storage (half of 1 cent/GByte/month) where you store your file
completely unencrypted, and this can be "private" (only accessible by
username/password) or "totally public accessible by knowing the URL". A good
application of this is serving up a web page to the public - you really WANT
people to see all the contents!

4) B2 Object storage (half of 1 cent/GByte/month) where Backblaze has zero
knowledge. You cannot browse your file hierarchy because Backblaze doesn't
know your filenames. You cannot preview your images. You cannot recover your
passwords. There is no other option other than downloading the encrypted blobs
and applying whatever decryption algorithm you decided on (we have no ability
to know what that is).

Ok, so I think some (many?) people in the security field think that Backblaze
should _ONLY_ offer mode #4 (and maybe #3 to serve up public websites). I
happen to disagree and I personally feel that products #1 and #2 are useful
and appropriate for some customers. But everybody is welcome to their opinion
and we want to be completely open as to what exactly is occurring and what we
are offering as a service.

Personally I think #2 is an excellent trade off of security vs convenience.
Your data is as impervious to attack as a zero knowledge system in #4 for
years upon years. Then one day your laptop is stolen or crashes and you want
your files back. You want all 4 TBytes back - so you order one of our free
(encrypted) USB hard drives to be FedEx'ed to your home with all your data. To
kick this process off FOR THE FIRST TIME EVER you tell us your passphrase (up
until this very moment it really has been zero knowledge). At this moment you
are opening a window of SLIGHTLY lowered security that slams shut after a few
hours. For those few hours of preparing your 4 TByte restore, if an undetected
hacker had compromised the one restore server in the Backblaze data center
that your job was on, that hacker could possibly get access to your files. But
then the reduced security window slams shut, we __NEVER __write your
passphrase to any disk so it has now vaporized and we do not remember it, and
if a hacker hacks into our system the following day you are STILL completely
impervious.

But again, as long as you fully understand the implications of #3 I am
COMPLETELY supportive if you instead choose #4 which is our "Zero Knowledge"
offering.

~~~
noja
I think most technical users - the ones making decisions about which backup
product makes sense for family and friends - would expect that a backup
service that offers the ability to encrypt locally would also offer the
ability to decrypt locally. Anything else is a big surprise.

When it comes to restore you have the users over a barrel - if they want their
files back, they've got to pony up their passphrase. If they don't, their data
is gone.

~~~
atYevP
Yev from Backblaze here -> we've always tried to be pretty transparent with
the way our system worked. For the vast majority of people in a crisis - they
want speed and ease, which are the things we designed the backup service
around. For folks that want a more technical option, we have 2 & 3 from
Brian's list. You can encrypt stuff yourself, use Rclone or Hashbackup (or
even the CLIs if you truly don't want to use other people's services) to
upload and download the data blobs.

~~~
noja
I've asked a lot of people about this, and none of them expected it to work
the way it does.

My parents could encrypt the stuff themselves using rclone or hashbackup, but
they have no idea how to do that. So they use Jungle Disk with client-side
encryption (and decryption) instead.

------
rocqua
BackBlazes stand on this, from the link:

Question: I Don't Trust You, but I am Expecting an Honest Answer. "If I opt to
password-protect my private key, are BackBlaze servers still able to decrypt
my data? (For instance, by sending the password to the server where my private
key is stored and decrypting the private key on the server) Or is my data only
able to be decrypted on the client?"

Answer: ABSOLUTELY NOT. Once the password is set do not forget it, there is no
way to recover it. Once protected by a private encryption key Backblaze can
only decrypt the backup files when the user supplies the password. This is
done during the restore process and the supplied password is never saved on
our systems, although never to disk, only to RAM.

What it Comes Down to is Trust, but, Unfortunately, I Just Can't Trust You.
"Having the private key outside my computer is unacceptable. The unfortunate
thing is that I can't trust you. I'm not allowed to trust you. So, I guess my
question is why do I have to? It makes most sense that I shouldn't -have- to
trust you..."

I understand. What you are describing is conceptually perfect and provably
more secure than what Backblaze provides , unfortunately it is not easy to
use. Backblaze focuses on ease of use. It is backup for people who need backup
and "pretty good security" and who aren't computer professionals. It is not a
perfect choice for all users on earth, you are unfortunately outside our
scope.

Take our default security model -> it is about as secure as a Facebook
account. It uses a username and password. You can recover (reset) your
password if you have access to your email. That mode is good for many, many
people on earth, people who have pictures of their children they don't want to
lose. Music they would rather not repurchase. And a few Quicken docs that they
would not like a malicious person to have, but they will not _DIE_ if somebody
reads them. And it's SUPER convenient to recover their password. What a great
system!

Our second level is really, really much more secure. You cannot recover the
password to access to your email account does not give you access to the
backup, and even a malicious hacker in our datacenter could not possibly
compromise your data. Now while it is more secure, it is also harder to use ->
if you forget that password you have actually lost the backup, your wedding
photos are gone forever. That's very secure, but it's not as secure as what
you describe.

We stand by our reputation as trustworthy, careful programmers who have worked
in the security field for over a decade. You can check us out on LinkedIn,
through colleagues that have worked with us, through the publicly traded
companies that have acquired our companies in the past. Here is our team page:
[https://www.backblaze.com/team.html](https://www.backblaze.com/team.html) We
live and work in Silicon Valley, we've been here for 20 years, and we plan to
keep doing this for a long, long time, and therefore we have _LOTS_ of
interest in keeping our reputations rock solid and utterly clean. Previously
we fought phishing fraud, fought email viruses, and fought spam at a company
called MailFrontier. We are totally customer focused and all around good
people, ask ANYBODY. If you come here to our offices, I'll buy you a cup of
coffee.

~~~
brianwski
Brian from Backblaze here.

I wrote the above information several years ago, and I'm still proud of the
"I'll buy you a cup of coffee" line at the end. :-)

But we need to update that web page because we have expanded our offerings
(mainly with the B2 Object Storage) and can offer customers more choices now.

------
Neliquat
A suprising oversight, but I have noticed in the heat of the moment few
customers care. Without friction, the incentive is not there, given all the
fires they likely fight 24/7\. Looks like someone just gave them incentive.
Hopefully an opprotunity to address this and look good, backblaze has always
seemed better at the (tech) public facing game than most.

~~~
cookiecaper
It's not really an oversight. There's just not a good option that normal
people will _actually use_. You could have the keyfile exist only on the
consumer's computer, but then when their computer crashes, the exact reason
they installed Backblaze, they aren't going to be able to access it. You
_could_ tell them to make a copy of the keyfile and store it in a safe deposit
box at their local bank, but that's an additional recurring cost for the box
and an additional time cost, and what if they can't get to the box when they
need to restore? etc.

This is one of those situations where security and convenience are
diametrically opposed. If Backblaze doesn't keep a copy of your keyfile and
you didn't complete steps B-D to make and keep a secure backup, you're
screwed. Most consumers are better off trusting Backblaze to keep that data
safe for them.

You could say that they should offer this as a secondary mode of operation,
but then people would _say_ "Yes, I want the most security possible! Yes!" and
then they'd sing quite a different tune when they get their backups back and
can't actually use them, and Backblaze already has enough problems with their
customer service dept.

If you want full client-side encryption, use B2 and hook up something like
duplicity to automatically difference and encrypt your files _before_ you
upload them.

