
Pik – a new lossy image format for the internet - tinnet
https://github.com/google/pik
======
vladdanilov
> New lossy image format gives 55 % less bytes than jpeg with the same
> butteraugli score.

It's the most serious attempt to date to make an open source replacement for
JPEG which misses a lot of advances in data/image compression. I hope it stops
the spreading of HEIF plagued by patents.

~~~
pasta
I had high hopes for FLIF. What I think is great is that the client can
control how much data is downloaded. This is great for responsive use.

Unfortunately the license is holding back implementations.

~~~
oliwarner
> Unlike some other image formats (e.g. BPG and JPEG 2000), FLIF is completely
> royalty-free and it is not known to be encumbered by software patents.

The reference implementations are also open source (dual licensed between LGPL
and Apache 2)... But even if you don't like those, you can write your own
under whatever license you like. They're just implementations of a spec.

------
TD-Linux
I noticed this project is missing quite a few features common to modern image
codecs, such as variable transform sizes, deblocking, or intra prediction. Of
course, it's a work in progress, so those things could still be to come.

Have you considered using Daala's lapped transforms? They work with variable
block sizes and are quite effective for still images.

Also, was there anything that motivated doing a custom image format rather
than applying Butteraugli to WebP?

~~~
JyrkiAlakuijala
We have tried a large variety of approaches for the integral transform, but we
were not impressed with the density/complexity/quality/decoding speed
compromise -- at high image quality. We believe that we can only add a certain
amount of complexity in this kind of format, and we try to keep our complexity
budget targeted on solving the high-quality-high-decoding-speed image
compression as well as possible.

------
ThomPete
Good thing Denmark is a small market as it means Dick in Danish.

~~~
sondr3
I'm just gonna start naming my projects by rolling my face on my keyboard,
that way I won't have to be afraid of naming it something offensive in another
language.

/s

~~~
majewsky
A naming pattern that I used for some time was to open the Wikipedia article
of the thing that my program is about, and look in the sidebar for nice-
sounding names for it in another language. It often ended up being something
Scandinavian. For example, KDE's jigsaw puzzle game is called Palapeli
(Finnish for "jigsaw puzzle") for this reason.

------
pornel
At the first glance it looks like it uses all the basic building blocks of
JPEG, but with everything upgraded to the next level.

It uses the same 8x8 DCT blocks, but with adaptive quantization. Like JPEG it
also codes DC separately, but with 8 predictors instead of 1, etc.

It's like a combination the Guetzli JPEG encoder and Lepton JPEG recompressor.

~~~
JyrkiAlakuijala
Yes, PIK is like guetzli + lepton, but decodes 2-3x faster and compresses 20 %
more.

------
jxcole
Can anyone explain if there is something particularly new or interesting about
this image format? Or do I have to read the source code?

~~~
JyrkiAlakuijala
PIK is building on guetzli and butteraugli, gives about -55 % less bytes than
JPEG for the same butteraugli score. Decoding speed is 2/3 that of JPEG.
Encoding speed is impractically slow in this version. PIK aims at delivering
very high quality photographs with minimal bandwidth and decoding (cpu and
memory) costs.

~~~
ninju
\- 55% _less_ bytes is a double negative so its 55% more :-)

------
Noctem
The readme mentions "Brunsli (lossless JPEG repacker)" as a related project,
but I can't seem to find that anywhere. Has it been released?

------
adrianN
Because Webp was such a success?

~~~
kyriakos
Not a failure either

~~~
kuschku
It’s unsupported on the browser that holds 43% of desktop marketshare in
central Europe, how is that not a failure?

~~~
trevyn
That doesn't make it a failure.

~~~
extra88
Google has failed to convince any browser maker not using their entire Blink
rendering engine (i.e. Opera) to implement support for WebP. It's been almost
seven years, not what I'd call successful.

On the server side, if you're processing images to create multiple versions
anyway (thumbnails, different sizes for mobile & desktop, etc.), throwing WebP
versions in there is simple enough and should reduce bandwidth use. It's not
worth doing if you're not fully automating image handling for other reasons.

~~~
theandrewbailey
Firefox support is coming. Looks like there's been a bit of activity on the
ticket:

[https://bugzilla.mozilla.org/show_bug.cgi?id=1294490](https://bugzilla.mozilla.org/show_bug.cgi?id=1294490)

~~~
mercer
I am not well-schooled in this area, but I find it very surprising that webP
is not supported on Firefox yet. Am I wrong in being surprised?

------
s0me0ne
Now if APNG would gather some steam, high quality animated fully alpha
transparent images would be nice. Gif's 256 color and 1 level transparency is
ok, but to do better work we need something like APNG, although not sure how
well it compresses.

~~~
Ajedi32
Pretty sure APNG is supported now in all browsers except Edge and IE. So not
quite there yet, but we're getting close.

~~~
ac29
Doesn't work in Chrome for Android either:
[https://caniuse.com/#search=apng](https://caniuse.com/#search=apng).

~~~
Ajedi32
Are you sure that's accurate? Chrome Platform Status says it's enabled by
default in Chrome 59 on Android:
[https://www.chromestatus.com/feature/6691520493125632](https://www.chromestatus.com/feature/6691520493125632)

------
cjensen
HEIF already exists as a standard [1] with comparable compression [2]. What
does Pik hope to do better?

(And please don't say "patent free" unless you are certain Pik will not
infringe on those same patents)

[1]
[https://en.wikipedia.org/wiki/High_Efficiency_Image_File_For...](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format)

[2]
[https://nokiatech.github.io/heif/technical.html](https://nokiatech.github.io/heif/technical.html)

~~~
mattnewton
Maybe "The software currently requires an AVX2 and FMA capable CPU, e.g.
Haswell." holds the secret? Some kind of experimental test bed for using CPU
vector instructions to speed up compression/decompression?

Disclaimer: I have no idea what I am talking about. For all I know possible
use of these instructions is already a feature of HEIF and that statement is
irrelevant.

~~~
cjensen
Yep. Use of AVX2 and so forth a pretty standard thing to do to get decent
performance for video/image encoding and decoding.

Worth noting: Skylake with Integrate Graphics includes an x265 encoder and
decoder. If that can't be adapted to HEIF, that's a huge win for the format.

~~~
janwas
If you're interested in this kind of thing, dc_predictor.cc might be fun to
look at. It computes the predictors in parallel (rather than the usual 'do the
same thing to 32 pixels') because here pixels depend on causal neighbor
pixels.

------
Camillo
Calling it "Google Pik" implies that it's an official Google project, but the
page explicitly says it is not. The post title should be changed to remove
Google to avoid confusion.

~~~
wmf
Or maybe _Google_ should avoid confusion and stop putting unofficial projects
under github.com/google.

~~~
londons_explore
Pet projects done with googles time and resources can't easily be published
elsewhere on github...

~~~
DonbunEf7
It might not have been done with Google's time and resources; Google likes to
strong-arm employees into agreeing that Google owns their side projects. The
`google` organization on GH is a compromise.

------
jgh
Not gonna comment on the image format itself since I'm not very familiar with
image compression, but from a code aesthetics point of view this project makes
me a bit sad. It's a half-hearted modern C++ attempt and everything is in one
directory. At least we don't have to dig it out of the Chromium repo, I guess.

------
euroclydon
Well, is it still using discrete cosign transforms like JPEG, or wavelets like
JPEG2000, or what?

~~~
JyrkiAlakuijala
Yes, it is using the discrete cosine transform like JPEG.

------
blahshaw
Is this what Google Photos uses for their unlimited "high quality" image
storage?

~~~
Scaevolus
Google photos "high quality" means downscaling to 16MP if larger and
compressing at 85% libjpeg quality level. I don't believe it uses exotic
codecs.

------
tarikozket
Does Google recognize it and show it on image results?

~~~
pfranz
"This project is in the initial research stage, please don't use it for any
purpose."

I doubt it

------
pmontra
Any patents on this?

~~~
cyphar
It is licensed under Apache 2.0, so if Google has any patents on it they could
not use them against users of this code or derivative code because of the
terms of the license. This doesn't protect you from the several other
irresolvable issues in software patents however.

------
azhenley
"This is not an official Google product."

Can anyone explain this to me? It is posted under the Google organization. Are
they just not actively supporting it?

~~~
ilyagr
It's written by a Google employee, and therefore Google owns all rights to it.
However, in all other respects it's just an open source project written and
supported in the authors' free time.

~~~
stepik777
How could Google own something written and supported in the authors' free
time?

~~~
polk
If you work at Google everything you make, even in your spare time, belongs to
them. That's how the contracts work, unfortunately.

~~~
trymas
> If you work at Google __everything you make__, even in your spare time,
> belongs to them.

Okay, google makes software and if you make software on your spare time maybe
it's reasonable that company does not want to leak IP and thus produces these
contracts.

But what about these stupid examples:

\- Building a backyard deck for your house \- Taking pictures as a hobby \-
Doing woodworking (e.g. making a chair and a table for your home office) \-
etc..

So with this contract, Goggle owns my deck, desk, chair and a picture of these
items, because I produced them on my spare time? Will horde of lawyers will
come to my house and take my deck, desk, chair and picture of those items,
because I used them while working remotely and bought tools to build them with
the salary they payed me?

Even if those examples are completely rubbish and won't stand in court for
Google, it's still absurd idea. You get payed salary for your time and work -
that's it! That's where Google's power ends. After hours you go home and it's
bullshit that they can own your personal work done on your own freetime at
your private property with private tools and your own ideas. If they want to
own everything then they should pay hourly average times 24hours , times 365
days per year as a salary, because I am giving all my time and work for them,
while I am in contract with them.

EDIT: quote from Google's Personal Projects (IARC) [1]:

> Because Google’s business interests are so wide and varied, this likely
> applies to any personal project you have. That includes new development on
> personal projects you created prior to employment at Google.

Well that's just messed up.

[1]
[https://opensource.google.com/docs/iarc/](https://opensource.google.com/docs/iarc/)

~~~
DannyBee
Errr. Decks are not intellectual property, for starters.

"Even if those examples are completely rubbish and won't stand in court for
Google, it's still absurd idea. You get payed salary for your time and work -
that's it! That's where Google's power ends. After hours you go home and it's
bullshit that they can own your personal work done on your own freetime at
your private property with private tools and your own ideas. If they want to
own everything then they should pay hourly average times 24hours , times 365
days per year as a salary, because I am giving all my time and work for them,
while I am in contract with them. "

You are sorely confused about the state of the law in the US. In the US, in
most states, it doesn't even have to be written into an employment contract.
The employer will simply own the IP, period.

~~~
Djvacto
I really don't think this is the case.

Check out the top comment from @grellas here
[https://news.ycombinator.com/item?id=2208056](https://news.ycombinator.com/item?id=2208056)

At the end, he mentions that general statements never apply to everybody, and
unique situations vary. Additionally, in California the law protects moreso
than other states.

~~~
DannyBee
You are certainly welcome to think as you like.

but grellas doesn't actually disagree with me anywhere?

FWIW: I'm also an IP attorney, and among other things, have been doing
"invention assignment" work for quite a while. I'm _very_ familiar with the
employment law situation in california, and this area in particular.

" Additionally, in California the law protects moreso than other states."

I didn't say it didn't? I said it's not as good as people seem to think

------
alpb
The title of this post is misleading. It's not an official Google product and
is not called Google Pik. This is just a project named "Pik" that happens to
be developed by Google employee(s).

~~~
sctb
Thanks, we've updated the submission headline.

~~~
tinnet
thanks for the info and the edit :)

------
tyteen4a03
I speak a bit of Danish and can't help but to giggle at the name.

(Yes, it means dick)

------
svisser
Unfortunate name if you happen to know Dutch.

~~~
colinbartlett
Oh... (NSFW)
[https://www.google.com/search?q=pik+meaning+dutch](https://www.google.com/search?q=pik+meaning+dutch)

~~~
forgot-my-pw
Good for porn images.

~~~
corobo
Well that is how new tech is often proven

------
jordache
yawn.. let us know when a major browser implements support for it...

~~~
Houshalter
With web assembly it should be possible to use new image formats even if the
browser doesn't support it. This will improve in the future when wasm gets
access to SIMD instructions and GPU computation.

------
coockciiciv
pik refer to the male sex organ.

------
Piccollo
ewwww c++

~~~
0xffff2
What would be an acceptable language in your view?

------
anigbrowl
Why?

------
amelius
We're seeing a lot of different formats lately. So to make this more generic,
I'm thinking we should move to an encoding-independent format. That is, the
file contains an encoding-id, and the decoder can be downloaded from a
standard location (a website run by a standards organization) using the
encoding-id as a reference. Of course, this can be cached.

The decoder can be in different formats, e.g. bytecode, WASM, i386, ARM,
etcetera. But of course, any binary decoders should be verified by the
standards organization before it is published.

~~~
a_t48
The file does contain an encoding-id - it's the last three or so characters in
the filename (alternatively, the first four bytes in the file data).

~~~
amelius
Yes, but the amount of encodings is pretty limited. It's better (imho) to make
the encodings free of any viewer or browser. That way, we can more easily
handle a large variety of encodings, and different versions of encodings.
Anyone could make a new encoder, and create files using them.

You can see it as a "democratization" of encodings.

~~~
janwas
The FourCC at the beginning is indeed your encoding-id. This is common in file
formats (e.g. RIFF). Out of curiosity, how is the 32-bit space insufficient?

