Unless the recipient of whatever you are compressing absolutely requires gzip, you should not use gzip or pigz.
Instead you should use zstd as it compresses faster, decompresses faster, and yields smaller files. It also supports parallelism (via “-T”) which supplants the pigz use case. There literally are no trade-offs; it is better in every objective way.
> There literally are no trade-offs; it is better in every objective way.
There literally are trade-offs, you started your comment describing one of them. If you want as wide out-of-the-box support as possible, you'd go with gzip.
The Compression Streams browser API only supports gzip (+ deflate) so if you wanna compress something natively in the browser without 3rd party libraries (or slow JS implementation), gzip seems to be the only option.
People had your exact sentiments, concerns and hesitations after gzip showed up in the early 90s. Eventually they moved on from pkzip/lzh/etc. to better, modern software - some on their own owed to being reasonable people, and some being dragged along with claws in ground while screaming about "breaking support".
It was easier back then, because there were fewer people developing new compressors.
I've lost count how many compressors have been marketed as a replacement for gzip over the decades. And it's always a replacement for gzip. Every time a new compressor starts getting popular, people start promoting a new even better replacement, and gzip never gets properly replaced.
zstd finally has some potential to replace gzip, but only if people accept it's good enough and stop trying to replace it with something even better.
bzip2 and xz have had the potential to replace gzip for the vast majority of users and use cases since more than a decade - and in many cases they have.
What I'm trying to say is that the excessive focus on cutting-edge technology is holding back progress.
gzip is still the default compressor people use when they are not sure about the appropriateness of other compressors in their specific use case, and they don't have the time or energy to find out. To replace it, the a compressor must satisfy two requirements:
* It must not be substantially worse than gzip on any relevant metric. bzip2 failed this by being slow.
* It must be ubiquitous enough that the idea of installing it no longer makes sense. xz never reached this point, before people started replacing it with better compressors.
I think these arguments are a bit contrived. bzip2 made progress on compression ratio. xz (and zstd) made progress on both compression ratio and speed. Neither hovered around the idea of cutting-edge "technology" (they all fall into the same technology: general data compression algorithms) because they aren't niched oddities like e.g. paq. But I don't understand why a successor must trump gzip in both aspects. gzip certainly didn't trump all of its predecessors on both aspects, and both aspects take turns being the more important one depending on user and scenario.
By cutting edge technology, I meant the latest products that are better than the earlier ones.
I work in bioinformatics, where people typically use either gzip or domain-specific compressors. gzip is used for the reasons I mentioned. It works, it's usually good enough, and if people in another organization you've never heard of want to use your compressed files, they can do so without bothering you with support requests.
zstd would be faster and compress better, but because you can't be sure everyone else can use it, you don't even bother thinking about it. The saved computational resources are probably not worth it. On the other hand, anything that makes gzip faster is valuable, as it allows saving computational resources without taking any interoperability risks.
I didn't say the gzip replacement must be better than gzip in every aspect. I said it must not be substantially worse. bzip2 was substantially worse, because it was substantially slower.
Well, if you care about supporting compress/decompress in the browser (natively) then you pretty much have the choice of using gzip or gzip, so there is still the limitation of support, no matter where you try to drag me.
gzip has the advantage of being ubiquitous. It's pretty much guaranteed to be available on every modern Unix-alike. And is good enough for most purposes.
Zstd is getting there but I personally don't bother with it on a daily basis except in situations where both performance and compression ratio are important, like build artifact pipelines or large archives.
You're reading old comp-sci usenet discussions from 1993 and you come across this statement: "pkzip/lzh/arc/zoo have the advantage of being ubiquitous. We should not encourage the use of gzip". You chortle.
zstd had a data corruption bug until quite recently. Eventually it may supplant
gzip as the de facto standard, but it's too soon to declare it better in every objective way. Give it time.
I agree. I wanted to use Brotli for my startup because it allows creating custom dictionaries but I had to resort to gzip because Brotli was difficult to setup on my CDN.
Instead you should use zstd as it compresses faster, decompresses faster, and yields smaller files. It also supports parallelism (via “-T”) which supplants the pigz use case. There literally are no trade-offs; it is better in every objective way.
In 2023, friends don’t let friends use gzip.