> “bzip3 is not yet listed on the large text compression benchmark” It is now
And it comes in relatively well, in my opinion. I'm a compression amateur, but bzip3 is the first entry I recognize as a general purpose compression program.
Yes, bzip3 compresses that 1GB text file into 170 MB while zstd needs 213 MB (SI units, from my own tests with the "enwik9" file of the GP's page, and the latest releases of both programs, same commands). But the decompression (memory and speed) is a different story.
On my desktop, decompressing with zstd requires 128 MB and 1 s, while bzip3 uses 3.2 GB and 90 s.
Without `-b 511`, bzip3's ratio decreases and decompression requires about the same memory as zstd. But the decompression is still two orders of magnitude slower.
Matt added bzip3 to the benchmark seemingly after the last year thread. If the zstd benchmark, last update at max 22 in 2016, were to be updated for today’s zstd (for long-range if nothing else) after coming across this thread, what would be the most compact possible list of option sets to benchmark to reflect the value of LR et al.?
“Hi, tool author here.” A useful explanation of Burrows-Wheelers transform as used by bzip3: https://news.ycombinator.com/item?id=42902407
“bzip3 is not yet listed on the large text compression benchmark” It is now: https://mattmahoney.net/dc/text.html
(2 years ago, 176 comments) https://news.ycombinator.com/item?id=42899713
(4 years ago, 104 comments) https://news.ycombinator.com/item?id=31324439