
Ask HN: LZ4 was not added as an optional compressor in HTTP/2, why gzip only? - DiabloD3
So, pretty much everyone agrees that LZ4 is a superior codec for on-the-fly compression of data, and works especially well with HTML and XML and JSON.<p>Yet, we still only allow gzip for HTTP compression. What gives? I use gzip -9 + advdef -z -4 -i 1000 + nginx&#x27;s gzip_static for static resources; but with dynamic HTML, I&#x27;m stuck with gzip -1 (with anything more, compression tends to take longer than the data saved wrt a modern Internet connection (&gt;10mbps)).<p>LZ4 would perform better here and require less CPU on the remote side (not a big thing on desktops and laptops, but a huge thing on cell phones and such), so why did Google and Mozilla and the IETF working group overlook this?
======
Someone1234
In a one word answer: Mobile.

LZ4 might be better at compressing, but it has worse performance
characteristics. But don't take my word for it:

[http://catchchallenger.first-
world.info//wiki/Quick_Benchmar...](http://catchchallenger.first-
world.info//wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO#Memory_requirements_on_compression)

As you can see, LZ4 uses 30x as much memory as gzip.

~~~
DiabloD3
Memory usage isn't that big of a concern anymore, not when phones ship with 1
or more GB of memory regularly.

And that doesn't answer my question: I'm okay with gzip also being allowed,
but the whole entire reason we allow multiple compressors is so the web
browser and the web server can negotiate what features are supported.

Low memory HTTP clients, yes, can continue to use gzip. But the rest of us,
why are we not using LZ4? I mean, hell, there is no reason why HTTP 1.0 and
1.1 clients can't also support LZ4 as well: they just have to negotiate for
the feature.

~~~
gumby
> Memory usage isn't that big of a concern anymore, not when phones ship with
> 1 or more GB of memory regularly.

Memory access costs power and time (which = power too).

~~~
DiabloD3
Yes it does, but does it cost more than the CPU power used by gzip vs the
memory access cost of lz4? I suspect the CPU cost is more by at least a
magnitude (although I can't measure it, but I bet Google has frameworks for
Android to).

------
SamReidHughes
That makes more work for people to do, makes a bigger attack surface for
hackers.

Also 12MB of memory used on compression (as seen in Someone1234's link) would
be pretty bad for a web server.

Edit: Also, do phones have trouble keeping up with gzip decompression? Isn't
it layout that's the limiting factor?

~~~
DiabloD3
Possibly not bigger attack surface. lz4 is already used in a lot of
mission/security critical areas, and they have audited many lz4
implementations for security and have found and fixed bugs.

For example, the Linux kernel can use LZ4, corporate VPN solutions like
OpenVPN can use LZ4, and many other things can use or require LZ4.

