
Deploying Brotli for static content - grey-area
https://blogs.dropbox.com/tech/2017/04/deploying-brotli-for-static-content/
======
powturbo
It may be interesting to look at a web content benchmark with the best gzip
compatible compressors and peak memory usage:

[https://sites.google.com/site/powturbo/home/web-
compression](https://sites.google.com/site/powturbo/home/web-compression) Only
facts and numbers, no rumors, no speculation, no hype,...

------
d0vs
I wonder who will win in the long term between Brotli and zstd.
[http://facebook.github.io/zstd/](http://facebook.github.io/zstd/)

~~~
valarauca1
ZSTD.

It is superior to Brotli in most categories (decompression, compression
ratios, and compression speeds). The real issue with Brotli is the second
order context modeling (compression level >8). Causes you to lose ~50%
compression speed for less then a ~1% gain in ratios [1].

I've spoken to the author about this on twitter. They're planning on expanding
Brotli dictionary features and context modeling in future versions.

Overall it isn't a bad algorithm. Brotli and ZSTD are head and shoulders above
LZMA/LZMA2/XZ. Pulling off comparable compression ratios in half to a quarter
of the time [1]. They make GZip and Bzip2 look outdated (which frankly its
about time).

ZSTD really just needs a way to package dictionaries WITH archives.

[1] These are just based on personal benchmarks while building a tar clone
that supports zstd/brotli files
[https://github.com/valarauca/car](https://github.com/valarauca/car)

~~~
terrelln
What use case do you have in mind for packaging dictionaries with archives?
There is an ongoing discussion about a jump table format that could contain
dictionary locations [1].

[1]
[https://github.com/facebook/zstd/issues/395](https://github.com/facebook/zstd/issues/395)

~~~
valarauca1
For large files >1GiB a library + archive is often smaller then the archive on
its own.

~~~
terrelln
How are you compressing the data?

I would expect a dictionary to be useful if the data is broken into chunks,
and each chunk is compressed individually.

If the data is compressed as one frame, I would be very interested in an
example where the dictionary helps.

------
JyrkiAlakuijala
It is interesting that Japanese, Russian and Thai benenefit more (30 %) from
brotli, than latin languages (25 %). This is because of the utf-8 context
modeling in brotli.

~~~
SaveTheRbtz
The first draft of the article actually had that reason, but there is also a
strong correlation between the size of the dict (these dicts are almost 1Mb,
while other languages are closer to 500kb) and compression ratio improvements.
Therefore I've played it safe and attributed it to the window size.

Though for languages like Korean and Chinese (whose size is more inline with
latin languages) we see 27.5% improvement, which is most likely due to context
modeling.

Therefore I assume ratio improvement is split ~50/50 between these two. It was
easy to verify that by compressing data with `brotli --window 15` and
comparing ratios there, but I was lazy there. I'm sorry.

PS. I've also skipped NFC/NFD part of the post which is very interesting for
Korean, where NFC normalized text occupies 30% less space. It also gives
additional ratio 5% for brotli and 15% for gzip.

~~~
JyrkiAlakuijala
There is no Thai or Korean in the dict. The total size of the dict (including
all languages) is 120 kB.

~~~
SaveTheRbtz
By "dict" I meant the data we are compressing: these are basically
dictionaries for "English to X" translation.

What was saying is that there is a strong correlation between size of the data
I was compressing and compression ratio improvements over gzip.

------
nailer
You can also deploy Brotli for dynamic content: at setting 4 it's both faster
AND compresses more than gzip.

[https://certsimple.com/blog/nginx-brotli](https://certsimple.com/blog/nginx-
brotli)

------
alexbecker
I use brotli on my personal website, and usually notice 15-20% smaller files
than the gzip equivalent. It's a bit of a pain to install a brotli compressor
but then very easy to add a single build step to compress all static assets.

------
WadeJohnson
Does someone know if IE11 will get brotli support? Or will IE11 only receive
security patches?

~~~
ameliaquining
Security patches only. IE11 exists on Windows 10 only for the benefit of
enterprises that can't drop horrible legacy technologies like ActiveX. All new
feature development is on Edge.

------
vladdanilov
Seems like most of the image optimizations are undone by WordPress/CDN, saved
>60% on the xkcd-styled charts.

------
huac
Why are their graphs xkcd-style?

~~~
yathern
I've seen it suggested before to imply a bit of a margin of error in the
numbers. I personally like it a lot

~~~
Ajedi32
Haha, I don't think the previous poster was referring to the error bars.
That's not "XKCD-style", it's just standard practice in statistics:
[https://en.wikipedia.org/wiki/Error_bar](https://en.wikipedia.org/wiki/Error_bar)

I think what he was referring to was the fact that the graphs appear to be
hand drawn.

~~~
decode
> I think what he was referring to was the fact that the graphs appear to be
> hand drawn.

Yes, and so was your parent comment. Some people have suggested that the hand-
drawn appearance communicates imprecision. For example:

> The rough, seemingly hand drawn nature of the graph provides a visual hint
> as to the imprecision of the results.

[https://www.chrisstucchio.com/blog/2014/why_xkcd_style_graph...](https://www.chrisstucchio.com/blog/2014/why_xkcd_style_graphs_are_important.html)

