
Ask HN: Why is there no standard encryption file format? - binwiederhier
When encrypting strings or files, application developers have to make choices about what cipher&#x2F;mode to use, where to store the IV and how to represent all of that in the database or on disk. This is annoying at best, but also leads to errors.<p>I&#x27;ve searched for RFCs but I couldn&#x27;t find any standard, and not even a defacto standard. Why does this not exist? Am I missing something?<p>If we had such a format, libraries could define a simple &quot;encrypt()&quot; function that would pick today&#x27;s suggested cipher spec (e.g. AES128&#x2F;GCM), but could provide backwards compatibility for the corresponding &quot;decrypt()&quot; function -- similar to hash mechanisms supporting the crypt syntax ($2y$...) such as PHP&#x27;s password_hash().<p>I&#x27;ve defined such a format before for an open source project I worked on years ago [1], so I don&#x27;t think it would be hard to define a standard like this. Am I overlooking something here?<p>[1] scroll down to &quot;crypto file format&quot; - https:&#x2F;&#x2F;syncany.readthedocs.io&#x2F;en&#x2F;latest&#x2F;security.html#encrypting-new-files
======
badrabbit
Been down this road. GPG exists for that purpose. Although not intednded for
disk files,s/mime has cross platform support as well.

To answer your question,interoperability with other applications is neither
expected nor desired for most apps that store encrypted data in a file. Quite
often there are additional implementation details to be included (for which
AEAD supportig cipher modes account for).

Let's say I encrypt outputs of my program and store them in a file,I will most
likely serialize it,encrypt it and store it only caring about my app being
able to read and deserialize it. There is mostly no need for interoperability
for data-at-rest encryption since there are a lot of interoperable protocols
for data-in-transit encryption. I do think it would not hurt to have something
like .gpg but more srandardized and easy to implement.

~~~
binwiederhier
You bring up a good point. In most cases interoperability is not required.

However, assuming it find enough adoption, having a standard format would
likely make many things easier, like switching languages, or moving parts of
the application logic into the database layer. Think of json support in
databases these days and how it enables versatile use across applications.

Another benefit would IMHO be the possibility for a simplified API for
developers. Many engineers don't know what AEAD or block cipher modes even
are. Providing a simple encrypt()/decrypt() function that would work across
applications would be awesome.

------
verdverm
Implementation matters and the same cipher may be good in one language and
broken in another.

What happens when I cipher becomes exploitable? How would you re-encrypt with
only a single en/decrypt functions? Which cipher was originally used?

~~~
binwiederhier
Hm. I don't buy the "implementation matters" argument. If AES-128-GCM is
broken in a library or language that'd be a pretty big deal. Cipher
implementations should be interoperable, otherwise you'd have a big problem.

Your other argument is totally valid. If a cipher becomes exploitable, you'd
treat it the same way as you would with password hashes. You'd need to re-
encrypt. password_hash() is designed to hash with today's best hash.

My question was mainly about the file format though. The encrypt/decrypt
functions would be a simple perk that becomes possible if we had that.

Edit: encrypt->hash

~~~
verdverm
By broken, I don't mean incorrect, just poorly implemented and there for
exploitable. This is true of libraries in the wild.

File format is typically binary or blob. It's encrypted so you can't see or
understand it's structure, only assume.

------
binwiederhier
This is obviously not a spec, but it's as close as it gets to the API that I
would like. And it's got Daniel Bleichenbacher listed as author so it's got my
vote. I couldn't find any on disk spec though at all though which is a little
odd: [https://github.com/google/tink](https://github.com/google/tink)

Very cool project.

------
jolmg
> I've searched for RFCs but I couldn't find any standard, and not even a
> defacto standard.

How about RFC 4880 - OpenPGP Message Format[1]?

[1] [https://tools.ietf.org/html/rfc4880](https://tools.ietf.org/html/rfc4880)

~~~
insomniacity
This page explains some of the problems with the format:
[https://latacora.micro.blog/2019/07/16/the-pgp-
problem.html](https://latacora.micro.blog/2019/07/16/the-pgp-problem.html)

~~~
binwiederhier
I remember reading this a while ago. Thanks for reposting.

I did in faxt stumble upon the linked RFC, but I quickly realized that it's
quite complex for the simple thing I would have expected. As the article
nicely points out, it's trying to be the Swiss army knife but that makes
things difficult.

