
Zstandard Compression and the application/zstd Media Type - okket
https://tools.ietf.org/html/rfc8478
======
twotwotwo
I'd love this added as an HTTP Content-Encoding, like brotli recently was (and
gzip has been for a long time). You could get a bit more compression at gzip-
like speeds, or have very cheap compression using its fastest modes (hundreds
of MB/s/core; see --fast modes at
[https://github.com/facebook/zstd/releases/tag/v1.3.4](https://github.com/facebook/zstd/releases/tag/v1.3.4)).
brotli especially helped at the high end (static stuff you'll serve many times
and can spend a long time compressing); zstd can be especially helpful nearer
the low end (dynamic content you'll serve a single time).

On Twitter a designer of Brotli raised the question of whether there should be
a standardized limit on RAM required to decode in the HTTP Content-Encoding
context, since zstd supports huge window sizes (GBs). I don't feel strongly--
clients can refuse to handle responses that take too much RAM to decode,
servers have an incentive to serve up things their clients can handle, and
there tend to be sharply diminishing returns to increasing window size--but I
can imagine a limit in the MBs being a reasonable compromise between allowing
high compression and not breaking on phones with a couple hundred MB RAM.

~~~
felixhandte
The RFC touches on that issue:

> In order to protect the decoder from unreasonable memory requirements, a
> decoder is allowed to reject a compressed frame that requests a memory size
> beyond the decoder's authorized range.

> For broader compatibility, decoders are recommended to support memory sizes
> of at least 8 MB. This is only a recommendation; each decoder is free to
> support higher or lower limits, depending on local limitations.

As a corollary, and in the absence of an explicit negotiation process between
client and server, servers should probably avoid using a window larger than 8
MB.

~~~
twotwotwo
Ah, great that there's an official number. And does seem like incentives are
aligned for servers to try to do the reasonable thing.

------
jepler
So can someone help me untangle how this relates to possible Zstandard
patents?

Back some time ago, when all public facebook git repositories had a patent
"grant" document, there was some angst about what exactly this meant:
[https://github.com/facebook/zstd/issues/335](https://github.com/facebook/zstd/issues/335)
\-- this was considered "resolved" when facebook decided to distribute the
software under GPLv2, though after reading the relevant sections of GPLv2 I
can't say I follow why this was supposed to resolve any uncertainty around
potential FB patents of zstd.

Under BCP79 it would be Facebook's duty to disclose any IPR it has in zstd;
there are presently no such disclosures registered:
[https://datatracker.ietf.org/ipr/search/?draft=&rfc=8478&sub...](https://datatracker.ietf.org/ipr/search/?draft=&rfc=8478&submit=rfc&doctitle=&group=&holder=&iprtitle=&patent=)

As far as I can tell, this is good news for people who would like to adopt
zstd!

~~~
electrum
The commit that dual-licensed under BSD and GPLv2 also "removed PATENTS
clause":

[https://github.com/facebook/zstd/commit/4f73b3b55d83b082aad1...](https://github.com/facebook/zstd/commit/4f73b3b55d83b082aad180816a5d336eff33a243)

~~~
tensor
BSD doesn't have a patent grant, so doesn't this mean you are only legally
free of their patent if you use the GPL code? Also, any independent
implementation of the algorithm would be subject to patent fees?

~~~
garmaine
GPLv2 doesn't have patent clauses either (that was the reason for GPLv3). So
I'm extra confused.

