
Zstandard v1.4.1 - felixhandte
https://github.com/facebook/zstd/releases/tag/v1.4.1
======
felixhandte
Context: Zstandard is a compression algorithm invented by Yann Collet, with
the goal to exceed zlib's compression performance along every dimension (those
being compression ratio, compression speed, and decompression speed). Although
that bar was cleared long ago, we've continued to work to improve Zstd, and
with this release, we've made Zstd just that little bit more
hardened/better/faster/stronger.

~~~
edflsafoiewq
How does it compare in binary size?

~~~
felixhandte
On my system, the gzip binary is 100KB and my zstd binary is 1.0MB. There are
a number of reasons for this:

\- By default, the Zstd binary has a lot of additional functionality that gzip
doesn't. For example: Zstd includes benchmarking capabilities, the ability to
train compression dictionaries, support for legacy (pre-1.0) formats, etc. You
can strip all that stuff out with compile flags, and the binary gets a lot
smaller.

\- Zstandard tries to cover a much wider range of compression speeds than zlib
does. To do this, the Zstd compressor actually has eight or so different LZ
match-finding implementations under the hood (which get inlined in various
ways, resulting in >100 actual versions in the binary), and 5+ different
entropy encoding implementations.

For really size-conscious use-cases (like decompressing Zstd content in a
mobile app), you can get a minimal Zstd decompressor library down to ~30KB.

------
retrobox
Zstd is awesome. I’d love to see desktop archive software include these
algorithms. There’s a fork of 7-Zip called 7-Zip ZS that includes zstd and
some other algorithms and I found they performed really well for my use case,
proving fairly comparable compression to alternatives at a much, much faster
speed. Handy for when you want to archive a massive directory of files or a
small number of really large files.

------
pvg
Previouslies:
[https://hn.algolia.com/?query=Zstandard&sort=byPopularity&da...](https://hn.algolia.com/?query=Zstandard&sort=byPopularity&dateRange=all&type=story&storyText=false&prefix&page=0)

------
nwmcsween
I really wish there was a small simple implementation of zstd

~~~
felixhandte
Out of curiosity, why? Zstd performs as well as it does because of the
enormous effort that has been invested into it. That necessarily brings
complexity.

We do actually have a simple “educational” decoder implementation [0], but I
wouldn’t recommend its use in production. And then there are two re-
implementations in Java [1] and Go [2].

[0]
[https://github.com/facebook/zstd/tree/dev/doc/educational_de...](https://github.com/facebook/zstd/tree/dev/doc/educational_decoder)
[1]
[https://github.com/airlift/aircompressor/tree/master/src/mai...](https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/zstd)
[2]
[https://github.com/klauspost/compress/tree/master/zstd#zstd](https://github.com/klauspost/compress/tree/master/zstd#zstd)

------
dlphn___xyz
is decompression faster than gzip/zlib?

~~~
duckerude
Funnily enough, zstd can also decompress gzip-compressed data faster than gzip
can.

    
    
      $ du -h data.gz
      324M data.gz
      $ time -p unzstd -k data.gz
      real 1.43
      user 1.24
      sys 0.19
      $ time -p gunzip -k data.gz
      real 4.85
      user 2.28
      sys 0.19
    

It's even faster at decompressing its own format.

