
Introducing Fresco: a new image library for Android - vierja
https://code.facebook.com/posts/366199913563917/introducing-fresco-a-new-image-library-for-android/
======
seanwilson
I wrote a painting app with layer support a while ago for Android that had to
use similar tricks (the app is coincidentally called Fresco:
[https://play.google.com/store/apps/details?id=org.seanw.fres...](https://play.google.com/store/apps/details?id=org.seanw.fresco.lite)).

Each image layer (like you would have in GIMP or Photoshop) takes a couple of
MB to store in Java and given the roughly 16MB Java app memory limit you
didn't have much memory left to play with after creating 5 layers. I found
that apps that used C and the NDK had no such limit; you could allocate huge
amounts of memory compared to Java without any issues.

To take advantage of this, I wrote code that would let me allocate C memory,
copy a Java bitmap into it then free the Java memory. To make all the app
features work without crashes due to Java hitting the memory limit, I would
shuffle bitmap memory between Java and C as needed, while having to implement
several algorithms in C that I would rather have done in Java. For example,
the undo/redo feature needed to store bitmaps of what changed when edits were
made so this was mostly kept in C. One tricky issue was I couldn't find a way
to get the Java APIs to render image data that was stored in memory used by C;
you had to copy the data back into Java first.

I found the above frustrating in that if you're writing an image manipulation
app, it means there is a massive incentive to avoid the native Java UI
libraries and write your app entirely in C as you don't have the same memory
limitations for some reason.

~~~
realrocker
"you could allocate huge amounts of memory compared to Java without any
issues". No sir. It still comes under the process memory account. It works in
practice because coincidentally other processes were not using that memory.

~~~
seanwilson
Well, I meant that if you hit the Java memory limit your app was guaranteed to
crash but C allowed you to use an order of magnitude more memory without
noticeable consequences when your app was in the foreground. Your app was
probably more likely to be closed when it went in the background but that
wasn't an issue.

------
hngiszmo
The mandatory link to Facebook's stance on patents that make their "open
source" a no-go for many:

[https://github.com/facebook/fresco/blob/master/PATENTS](https://github.com/facebook/fresco/blob/master/PATENTS)

~~~
rtpg
so what's the layman description of this? Don't sue us and we won't sue you?
That seems like a pretty good way of applying MAD to the patent space

~~~
Sanddancer
It goes a step beyond and says not to even think about challenging one of
their patents. So if facebook patents something stupid, and you call bullshit,
you can't use their software.

------
tyronen
I'm an engineer on the project. Would be happy to take questions.

~~~
realrocker
Hey. Quote,

"Our breakthrough came when we realized we didn't have to do that. If we
called lockPixels without a matching unlockPixels, we created an image that
lived safely off the Java heap and yet never slowed down the UI thread. A few
lines of C++ code, and we were home free."

That doesn't sound complete. It's off the Java heap, true but doesn't mean
it's either safe or won't slow down the UI or is even home free. Allocation in
ashmem will still count towards the PSS[1] of the app. The image data won't be
unpinned right until it throws an OutofMemory exception. And if you have
handled the exception, the UI thread will wait around for the unpin/unlock of
data. This would be ok if you could make an educated guess of how much ashmem
can be allocated to your app. You can't make that guess easily as it depends a
lot on OEM implementation of Gralloc/HWComposer and the GPU. But since you are
Facebook, :), you can probably test this on all device/gpu variants. And when
you do please publish the results. And while you are doing that, please test
this too:
[http://developer.android.com/reference/android/os/MemoryFile...](http://developer.android.com/reference/android/os/MemoryFile.html)

[1][https://developer.android.com/tools/debugging/debugging-
memo...](https://developer.android.com/tools/debugging/debugging-
memory.html#ViewingAllocations)

Thoughts?

Edit: Incorrect link

~~~
georgemcbay
As someone who has written a lot of code (including driver level) in the
DirectX realm, that line (about not unlocking surfaces) gave me a minor
aneurysm though I admittedly do not know enough about the Android ashmem
internals to know if the reaction is actually warranted or not on the Android
platform.

~~~
realrocker
Well allocating on ashmen does reduce the UI lag since its just a memory pool
and espaces GC reprimand but it's definitely not safe. Such operations are
better handled at Gralloc/HWComposer Layer where you have a better handle on
what's going on. Someone's gotta unlock that surface!

------
vierja
Better:
[https://github.com/facebook/fresco](https://github.com/facebook/fresco)
[http://frescolib.org/](http://frescolib.org/)

~~~
TD-Linux
... and the good ol' Facebook PATENTS file.

------
tiler
I wonder if this work has any link to what Carmack and the Oculus Rift crew
are doing with Android?

