
HEIF – High Efficiency Image File Format - indream
https://nokiatech.github.io/heif/
======
TD-Linux
The license in the provided implementation seems to be a custom, non-
commercial, non-OSI-approved license [1]. It also includes a patent grant, but
excludes codec patents, making it even more useless (Nokia can't license other
people's HEVC patents, of course). It's based on the LGPL-licensed libde265
library.

[1]
[https://github.com/nokiatech/heif/blob/master/LICENSE.TXT](https://github.com/nokiatech/heif/blob/master/LICENSE.TXT)

~~~
bengotow
License notwithstanding, has anyone tried to create a HEIF file? There are a
few samples on this site, but even given the code they've provided on GitHub I
can't seem to create a HEIF image from another image format. Their code seems
to take an H265 HEVC bytestream as input, which is fine, but I can't find
anything that will build an HEVC bytestream for an image rather than a
video...

~~~
astrange
You can easily turn an image into a video, or into HEVC, with ffmpeg.

~~~
bengotow
Haha thanks - I didn't think it supported H265 but it looks like I just have
to build it from source
([https://trac.ffmpeg.org/wiki/Encode/H.265](https://trac.ffmpeg.org/wiki/Encode/H.265)).
Suppose that's still fairly "easily" ;-)

------
niftich
This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1],
which was essentially a lightweight wrapper around an HEVC I-frame. The format
got small amounts of attention in encoding circles, but didn't pick up
mainstream traction.

The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4
Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4,
.3gp, etc.), which theoretically gives it wide support. In this structure,
HEIF supports image sequences, which will help it compete with animated GIF,
GIFV (which is really just a short MPEG-4 Part 14 file containing usually an
H.264 video track), animated WebP (who uses these?), and WebM.

This format doesn't really blaze new ground _(EDIT: it does in the sense that
it defines a container format to express image-y constructs like still images
and sequences of images in an ISO media container, but see my other comment
that asks how this is similar but different to video [3])_ , but if this
repackaging and the resulting code donation and political clout helps it gain
traction, we still would gain a lot.

The problem, of course, is always with backwards-compatibility. WebP was
aggressively promoted by Google the same way Microsoft used to promote its
quirky Windows Media formats back in the early 2000s, but the WebP and non-
WebP camps are still largely separate. This is unfortunate, because WebP in my
opinion really isn't very good, and a backwards-compatible compressor on top
of JPEG, such as Dropbox's Lepton [2] achieves similar results. Any HEVC-based
image compressor is necessarily incompatible with JPEG; so broad buy-in would
be required from makers of software and hardware products, from operating
systems, file managers, image viewers, image editors, cameras, and the like,
to really enjoy its improved compression rates, vs. being just another
incompatible format that only functions inside controlled ecosystems.

The video codec AV1 currently in development was supposed to be a grand
alliance of disparate companies to agree on a common format for the future,
and rectify the WebM vs. non-WebM split, but continuing to promulgate
sophisticated formats based on work outside of this scope (such as MPEG- or
ITU-derived custom formats) just muddies this further.

[1] [https://bellard.org/bpg/](https://bellard.org/bpg/) [2]
[https://news.ycombinator.com/item?id=13230805](https://news.ycombinator.com/item?id=13230805)
[3]
[https://news.ycombinator.com/item?id=14490734](https://news.ycombinator.com/item?id=14490734)

~~~
wereHamster
At last for the web (which is what WebP targets mostly, right?) you don't need
broad support. We have the new <picture> element where the UA selects the most
appropriate image from a list of options. So Chrome will prefer webp, other
browsers will pick jpeg, png or whatever else they support at the moment.

~~~
jacobolus
If possible, I’d much prefer having a single format that is widely supported,
hardware accelerated, and with a state-of-the-art codec. Making content
authors create multiple different files in different formats—some with
deficient feature support—sounds like a pain in the butt for everyone.

If we can get everyone to agree on a format that handles layers, exposure
bracketing, transparency, animation, high bit depth, different color models,
both lossy and lossless compression, every common type of metadata, etc., that
would be great.

~~~
astrange
If your format handles everything it isn't a format, it just is everything.
(The one you want is TIFF though.)

~~~
jacobolus
The point is that you want support for those features (certainly the core
codecs, etc.) in the spec, so if you send someone a file you can be sure they
can read it.

TIFF is not useful if you want to use a state-of-the-art lossy codec and get
hardware-accelerated decoding on an arbitrary mobile device, or if you want to
display an animated image with transparency on a web page.

~~~
astrange
But can you actually get people to implement it if you make it that
complicated? Images are pretty hard, hardware support is /actually/ really
hard.

That's why it's best to accept whatever format is already out there and author
to it - the only thing that matters is that it actually works when it's out
there.

Actually I'm surprised there's still no good way to do animations with
transparency, but maybe nobody wants to use it?

------
pornel
It makes a lot of sense for Apple to use H.265 as a base for the format. It's
much more efficient than all the other JPEG-killers (easily beats WebP by a
wide margin). Apple already pays for the patents, and has invested in H.265
hardware and software for H.265 video.

However, the HEIF wrapper for H.265 strikes me as quite complex. It brings
baggage of ISO wrapper formats and tries to support everything ever crammed
into any image-ish format. That, in addition to patent licensing, may be
another difficulty in widespread support for the format. It will be hard to
implement robust, secure and fully-featured players.

~~~
jpap
While the full standard text and machinery can get quite complex, you can
still construct relatively simple files housing a single image, thumbnail, and
Exif that isn't that much more complex than a typical JPEG file. Compared to
JPEG, where you need to define quantization tables, Huffman tables, etc. in
marker segments, much of the codec-related complexity in HEIF is layered
within the compressed image payload (e.g. HEVC NAL units).

Besides the coding efficiency, JPEG really hit the big time because of the
readily available libjpeg cross-platform implementation. In this case, HEIF
can leverage existing implementations of the ISO Base Media File Format box
model; it builds heavily on that standard.

Nokia is doing the right thing by releasing their format handling library,
though perhaps they could loosen up their license to include commercial use.
;-) Hopefully there are some developers amongst us inspired enough to start a
new open project that, like libjpeg, brings HEIF to the masses.

------
throwasehasdwi
Nokia owns several HEVC patents, in fact I think they're currently suing apple
about it. This is a veiled attempt to get everyone to use their patented tech
so we can relive the mp3/mp4 clusterfuck. You would be a fool to use this for
anything.

What's so bad about WebP? It's not the best thing in the universe but it beats
JPEG and PNG and has a wide support base. With a shim it has native support in
most browsers anyways since WebP is a just a single frame of video. And as a
bonus you don't have to worry about someone suing you for royalties down the
road.

~~~
pornel
> What's so bad about WebP?

WebP is based on VP8, which is now obsolete. It was meant to be a H.264
competitor, a generation behind H.265 and VP9/VP10. WebP compression
efficiency is much closer to JPEG than H.265.

This format beats WebP by a margin wider than WebP was smaller than JPEG.

~~~
throwasehasdwi
Most devices still encode and decode far more H264 and JPEG than newer
generations. We shouldn't be stuck with the horse and buggy because we're
waiting for cars to be autonomous. WebP is superior to JPEG and the most
widely available alternative.

------
Adamantcheese
I wonder how it compares to FLIF, which looks like it has a formal release
finally. [http://flif.info/](http://flif.info/)

~~~
agentgt
It looks like FLIF doesn't really support lossy well (its sort of fake lossy).

 _" FLIF does not have a lossy mode, but interlaced files can be decoded
progressively so we can simply use something like dd if=lossless.flif
of=lossy.flif bs=1024 count=5"_

You can see this in the examples where BPG does well in stream loading:
[https://nokiatech.github.io/heif/technical.html](https://nokiatech.github.io/heif/technical.html)

HEIF would probably be similar to BPG in performance.

Still FLIF looks pretty darn interesting.

~~~
sprash
> It looks like FLIF doesn't really support lossy well

I don't know what you mean by well. As far as I'm concerned lossy FLIF beats
JPEG easily. Here is a comparision:

[http://flif.info/lossy-artifacts.html](http://flif.info/lossy-artifacts.html)

~~~
agentgt
Whoops looks like I pasted the wrong link.

Here is the correct link:
[http://flif.info/example.html](http://flif.info/example.html)

And yes "well" is relative and what I meant by that is that its not readily
easy for a normal user to do it.

BTW I think Flif is damn good particularly progressive decoding. Its unclear
how good HEIF progressive decoding is.

~~~
emn13
Yeah, even on that fair; jpg-unfriendly example jpeg still clearly beats flif
above 22kb. And below 22kb - sure, FLIF is _less_ bad, but it's still
unusable. Also, the examples below 22kb are a little unfair to most codecs in
practice, since you'd get far better results by downscaling first.

------
amluto
It's based on HEVC. I wonder how bad the patent situation is.

~~~
MontagFTB
It would seem the Nokia open source implementation is what's limited by the
patent rider. I don't think that applies to the format at large.

~~~
BillinghamJ
Nokia and Apple just recently resolved their differences on patents it seems

------
zokier
While the basic idea of using HVEC for still image compression is good, I'm
somewhat concerned on how complex the whole format, especially the container,
for HEIF is. It is kinda cute and from one viewpoint pragmatic to reuse
existing container format, but I'm afraid the flexibility might have
unforeseen and undesirable side-effects. And of course there is the problem of
different implementations supporting different subsets and extensions of the
format.

~~~
TD-Linux
They've already specified three different codec mappings - JPEG, AVC, and HEVC
- so to open all HEIF files, you presumably need to implement all of the
decoders.

Worse, there is a financial incentive to implement only a subset of the
decoders - HEVC is more expensive than AVC which is more expensive than JPEG.
Even things like 4:4:4 color are more expensive than 4:2:0 color for HEVC.

------
vbernat
This seems pretty similar to BPG:
[https://bellard.org/bpg/](https://bellard.org/bpg/)

~~~
masklinn
The core idea does, but according to the comparison table HEIF seems to go
quite a bit beyond BPG.

------
hoschicz
It's really sad that Apple decided to support the proprietary H.265 and this
image format instead of the royalty-free WebP and WebM.

~~~
threeseed
I think you're confused. WebM is a container not a codec.

And either way VP9 and AV1 are simply inferior codecs to H.265 in almost every
way. And as for royalty free well that's because MPEG-LA has't ever sued but
they did give a royalty free license to Google for patents infringed by VP9.
And since Google accepted it does indicate that AV1 is likely to also be
patent encumbered.

We would all like a royalty free codec but it isn't happening anytime soon.
And so failing that I would prefer to go with the codec with the best support
which by far is H.265.

~~~
Veratyr
> And either way VP9 and AV1 are simply inferior codecs to H.265 in almost
> every way.

Source?

AV1 is already beating HEVC in compression and it's not even finalized yet:
[https://youtu.be/lzPaldsmJbk?t=2416](https://youtu.be/lzPaldsmJbk?t=2416)

~~~
threeseed
VP9 versus H.265: [http://www.streamingmedia.com/Articles/Editorial/Featured-
Ar...](http://www.streamingmedia.com/Articles/Editorial/Featured-
Articles/Netflix-Finds-x265-20-More-Efficient-than-VP9-113346.aspx)

Also the link you posted is using x265 which as the article suggests was
ranked 4th out the 6 available encoders:
[http://compression.ru/video/codec_comparison/hevc_2016/MSU_H...](http://compression.ru/video/codec_comparison/hevc_2016/MSU_HEVC_comparison_2016_free.pdf)

And whilst AV1 and H.265 are pretty close in PSNR where it is inferior is in
vendor support. There are TVs for example available today that natively
support H.265 whereas AV1 is still in development. Not to mention all of the
terrestrial broadcast providers all using H.265.

~~~
Veratyr
> the link you posted is using x265 which as the article suggests was ranked
> 4th out the 6 available encoders

The other encoders there are commercial or not publicly available as far as I
can tell, so it's hard to use them as a comparison. Plus according to that
PDF, the best encoder is 0.85x x265, which is about the same as AV1 in its
present state in the video I linked. And again, AV1 isn't even finalized yet,
so I actually find it somewhat impressive that it can already compete against
these highly tuned encoders with years of development put into them.

> where it is inferior is in vendor support

Of course there's no vendor support _now_ , the codec isn't even finalized
yet. However the codec is backed by Microsoft, Google, Mozilla, Qualcomm,
Intel, AMD, ARM, Broadcom, NVIDIA, Adobe, Netflix and the BBC, i.e. all but 1
of the major browser and OS vendors, every major mobile, server and desktop
processor company and the 2 most widely used video streaming companies.

~~~
threeseed
H.265 is already in shipping hardware from Sony, LG, Samsung, Intel, AMD, ARM,
Nvidia etc. As I listed above there are at least 100+ more companies
supporting H.265 than AoM. Including the most important of all the broadcast
standard for terrestrial TV which is still hugely popular in many countries.

Also without Apple it's over. YouTube and Netflix will be forced to support
H.265 or give up on iOS/OSX users (350 million users).

~~~
Veratyr
> As I listed above there are at least 100+ more companies supporting H.265
> than AoM.

AV1 _isn 't finalized yet_. Of course HEVC is being shipped in more stuff,
it's being shipped at all. I'm not debating this, there's no debate. At
present HEVC has more support than a codec that isn't yet finished. H.264 had
more support when VP9 was being developed too, that doesn't make VP9 inferior
to H.264.

> broadcast standard for terrestrial TV which is still hugely popular in many
> countries

Broadcast TV is a different world to the web and has very different concerns.

> Also without Apple it's over. YouTube and Netflix will be forced to support
> H.265 or give up on iOS/OSX users (350 million users).

This is demonstrably not true as YouTube and Netflix both currently and
successfully utilize VP9, which Apple doesn't support. Apple users just get
the inferior H.264 codec.

The question is really whether the browser and other OS vendors are going to
start supporting HEVC and given the current patent situation, that seems
impossible, so we're going to end up in a fragmented world where for web
streaming, most users get AV1 while Apple users and anyone else who doesn't
support AV1 gets H.264.

~~~
abritinthebay
> AV1 isn't finalized yet.

You keep saying that as if it solves anything.

Released > Unreleased.

Is that a criticism of the format's _potential_? No! But it _may as well not
exist_ until it is finalized... so it can't be a competitor to a format that
_has been_.

~~~
Veratyr
> VP9 and AV1 are simply inferior codecs to H.265 in almost every way.

This was your claim.

Let me sum up the entirety of my position so we're on the same page:

\- In terms of compression, AV1 is competitive with or beats HEVC, despite not
being finalized.

\- AV1 has support from everyone but of import save Apple when it comes to the
web.

\- Apple's support is not necessary for a codec to be successful (see VP9).

\- AV1 is not finalized. There's more work to do. Particularly, I don't expect
the present encoder/decoder implementations to be competitive with the HEVC
encoders when it comes to CPU efficiency.

\- HEVC has solid adoption in some areas (the more "traditional" areas like
video cameras, TVs, broadcast) but has zero adoption when it comes to the web
and this is extremely unlikely to change due to the patent mess.

\- As members of the AoM, it's near certain that Microsoft, Google and Mozilla
will add AV1 support to their browsers, which, depending on the particular
source you look at, cover roughly 70% of the total browser market.

\- Due to AoM support and HEVC not being an option, AV1 will be the primary
successor to H.264/VP9 on the web.

\- Due to its likely widespread availability from streaming services on the
web and support from hardware vendors, AV1 will be prolific in other kinds of
hardware.

\- Due to the support of Apple and the broadcast industry, HEVC will likely
not go away anytime soon.

\- I agree that HEVC is not _currently_ a viable alternative to HEVC, it not
being finalized yet and all, however despite that, there are already aspects
in which it is already superior, namely compression and likelihood of being
supported on the web.

------
amelius
Why don't we store a url to the decoder inside images and other compressed
files for maximum flexibility? I mean, looking at WASM, sandbox technology is
sufficiently strong. And performance and availability aren't really an issue
either, because for specific cases we can fall back to what we are doing now.

~~~
masklinn
> Why don't we store a url to the decoder inside images and other compressed
> files for maximum flexibility?

Image formats are already some of the biggest attack surface of modern
systems, "executable" image formats (hello PDF) have an absolutely ghastly
track record there.

And not being able to open an image if you don't have an internet connection
sounds dreadful.

~~~
amelius
But, as I already implied, if we can safely run arbitrary "binary" code of the
internet (see WASM), why can't we run arbitrary code of a decoder?

~~~
jerf
The things that make it safe to run that code will significantly negatively
impact image decoding speed right now. That's not necessarily a fundamental
problem with the universe, but it's a true statement at the moment.

And that's accepting that WASM is safe, which I do not axiomatically accept.
History suggests that I am very safe in claiming that most implementations
will end up with some catastrophic-level security vulnerabilities in them
before all's said and done.

~~~
Dylan16807
What's your threshold for significance on image decoding speed? I bet making
it 4x slower would be negligible in the vast majority of cases.

The computational power you need for image decoding is extremely narrow and
easy to make safe. You need some mathematical operations and some loops. You
don't need any APIs or data structures. Mask off all the pointers and you can
have have a provably safe interpeter/compiler that runs pretty fast.

~~~
astrange
Remember that 4x slower means (at least) 4x worse battery drain on a phone. In
the modern internet there basically is never any excuse to waste resources.

~~~
Dylan16807
But it's also on a phone where saving data transfer can give you huge battery
benefits. And it's not intentional waste; having this fallback doesn't stop
browsers from adding native decoders.

------
tehabe
So HEIF is for HEVC what WebP is for VP9.

~~~
niftich
Roughly, BPG [1] is for HEVC as what WebP is for VP8 or VP9, in the sense that
you take an I-frame out of a video format and stuff it into an established,
classic image container.

Compared to this, HEIF is a different way of packaging HEVC frames; namely
into ISOBMFF (MPEG-4 Part 12), the most basic level of MPEG container. This
comes with benefits [2] that come along with using that particular container,
but also drawbacks, because with some of the advanced features of HEIF you're
essentially describing a sequence of frames -- which is almost like video --
but you're doing it in a way incompatible with the way you'd describe video.
I'd be curious if the two "representations" are losslessly convertible, for
example.

[1] [https://bellard.org/bpg/](https://bellard.org/bpg/) [2]
[https://nokiatech.github.io/heif/technical.html](https://nokiatech.github.io/heif/technical.html)

~~~
jpap
> I'd be curious if the two "representations" are losslessly convertible, for
> example.

They are: you could just extract the HEVC NAL units, and re-write them into a
MP4 or QuickTime container, making sure to properly place the codec
configuration box, etc.

HEIF also goes beyond a sequence of frames, in that it can describe alpha
planes, depth maps, tiling, etc. In that case there might not be an analog
with a standard video. If you really wanted to decompose a HEIF container, you
might choose to extract the raw media into elementary streams (for HEVC or
AVC; or if you're using the JPEG codec, just plain JPEG files) adjacent to any
metadata like Exif, etc. This is essentially what Nokia's conformance files
are [1].

[1]
[https://github.com/nokiatech/heif_conformance/tree/master/bi...](https://github.com/nokiatech/heif_conformance/tree/master/bitstreams)

------
intoverflow2
"Have you got a 'heef' file of that photo please?"

Could have been branded better if I'm honest. I know engineers generally don't
care about stuff like this but some of us will have to say this term many
times a day if takes off.

------
PM_ME_YOUR_NDA
Can somebody please ELI5. Assuming I have iOS 11 on my iPhone and I take a
bunch of pictures. Will I have to convert the pictures to JPG/JPEG/etc. myself
when I decide to import them to my Mac/Windows machine?

------
namingwaysway
What's the support for popular image process libs?

------
fao_
What benefits does this have over PNG, farbfeld, etc. ?

~~~
masklinn
PNG is a lossless format, it's pretty much pointless for photos. IDK about
farbfelt but it seems like a TIFF competitor more than anything else.

This is an (significant) improvement over JPEG.

~~~
fao_
> a lossless format [is] pretty much pointless for photos

Why is that? Every edit you make to that photo means that the photo will
degrade in quality. I always thought JPEG artifacts and compression lossage
was a bad thing for photos, not a good thing?

~~~
masklinn
> I always thought JPEG artifacts and compression lossage was a bad thing for
> photos, not a good thing?

Sure JPEG artefacts are bad, but that can be mitigated (stop compressing so
much). Your photos being an order of magnitude larger is worse, and inherent
to the way PNG works.

------
vim_wannabe
If Apple started to support natively this in Safari, which one would succeed:
HEIF or WebP?

~~~
floatboth
Just look at Chrome (and everything Chromium based) vs Safari market share…

[https://caniuse.com/#feat=webp](https://caniuse.com/#feat=webp)

------
pcgamestime
Agree

------
frik
iOS11 now takes photos in HEIF instead of JPEG by default. I think that's why
it's posted here.

~~~
0x0
Wonder how this will work with random websites' photo upload features. Will
all other browsers break because they are serving HEVC user-uploaded content?
Or will iOS transcode to JPEG on upload (triggering double lossy encoding?)

~~~
roywiggins
How many websites don't already re-encode most of the JPEGs they're handed?
Facebook does- Flickr does, though sometimes you can request the original file
specifically. Imgur probably does.

~~~
dom0
Imgur very much does, with a visible loss of quality.

~~~
i336_
This can be disabled in account settings though.

------
williamle8300
If I could give unsolicited advice: focus on graceful progressive
enhancements. Galleries, tiles, etc should be left to JavaScript.

