
LZFSE compression library and command line tool - ingve
https://github.com/lzfse/lzfse
======
kvark
> It targets similar compression rates at higher compression and decompression
> speed compared to deflate using zlib

I don't doubt this is true, but I'd expect a little benchmark on the README as
a proof. It would be interesting to see how LZFSE compares to LZ4
([https://cyan4973.github.io/lz4/](https://cyan4973.github.io/lz4/)).

~~~
gribbly
It's tested against a wide range of other compressors in this thread:
[http://encode.ru/threads/2221-LZFSE-New-Apple-Data-
Compressi...](http://encode.ru/threads/2221-LZFSE-New-Apple-Data-
Compression?s=031e6ce4e62b74fde592c96e91384332)

The results are kind of underwhelming.

~~~
muizelaar
[https://quixdb.github.io/squash-
benchmark/unstable/](https://quixdb.github.io/squash-benchmark/unstable/) in
particular paints a pretty good picture of the current landscape. It seems
like ZStd is usually better than LZFSE and LZ4 is pretty competitive with
LZFSE:LZVN.

~~~
yAak
I'm curious to see comparisons on a 64bit ARM cpu, since that's what Apple
likely optimized it for. (I don't expect a huge difference, but I'm admittedly
ignorant on this topic.)

~~~
eridius
I'd also be curious to see what the energy impact is of the various methods.
LZFSE wasn't just designed for speed, it was also designed for energy
efficiency.

~~~
giovannibajo1
I can't see how "power efficiency" can be different from "speed" in the
context of a CPU decompressor algorithm. Care to explain?

~~~
eridius
I don't know the details, but I don't see why power efficiency has to be the
same thing as speed. For example, various patterns of memory access or I/O may
run at the same speed but could very plausibly have different amounts of
energy consumption. Similarly, it's conceivable that different CPU
instructions that run in the same time might have different energy demands.
CPUs also have different speeds they run at which have different energy usage,
it's possible that LZFSE might be designed to avoid putting the CPU into its
highest-power mode. But as I said before, I don't know the details about what
LZFSE does, I just know that there are various reasons why energy usage is not
the same thing as CPU time.

------
s34gull
The most interesting discussion of this compression algorithm that I've found
is in a presentation from WWDC 2015
[http://devstreaming.apple.com/videos/wwdc/2015/7125ovmdf36/7...](http://devstreaming.apple.com/videos/wwdc/2015/7125ovmdf36/712/712_low_energy_high_performance_compression_and_accelerate.pdf?dl=1).
The goal is not strictly faster or higher compression ratio, but a reasonably
balanced mix of the two at lower energy consumption (since the original target
was iOS).

~~~
wmf
Note that energy efficiency is pretty proportional to performance, so it's
very likely that a faster algorithm (e.g. LZ4) would also be more efficient.

(Back when I was working on energy efficiency, people kept trying to invent
"low-power compilers", but they kept discovering that the fastest code is also
the most efficient.)

------
jakozaur
Based on internet bechmarks, zstd looks like strictly better option:
[https://github.com/Cyan4973/zstd](https://github.com/Cyan4973/zstd)

~~~
eridius
Those benchmarks don't include energy efficiency, which was a primary design
goal of LZFSE. I also don't see LZFSE on that page anyway, which makes it kind
of hard to compare.

------
salem
There was an Apple presentation that claimed lower decompression energy usage:

[https://developer.apple.com/videos/play/wwdc2015/712/?time=4...](https://developer.apple.com/videos/play/wwdc2015/712/?time=457)

On an iOS device this is a pretty hot path, since it's used to decompress the
asset catalog, including some images, so most of your apps on iOS probably
already use this algorithm.

------
gbrown_
Same link posted a few days ago.

[https://news.ycombinator.com/item?id=11922484](https://news.ycombinator.com/item?id=11922484)

Previous (but brief) discussion.

[https://news.ycombinator.com/item?id=9689471](https://news.ycombinator.com/item?id=9689471)

------
devy
Pardon me if this was already answered. Is this the official implementation of
LZFSE from Apple? The main contributor for the Github repo @ebainville is
listed in Cupertino, CA.

~~~
wmf
"Copyright (c) 2015-2016, Apple Inc."

