Hacker News new | past | comments | ask | show | jobs | submit login
The Care and Feeding of the Android GPU (satine.org)
57 points by buymorechuck on Jan 3, 2011 | hide | past | favorite | 55 comments

This isn't the first thing I've seen that Google devs really just didn't get. I think it's from the rapid expansion that Google has done. They get people working on things that aren't really their area of expertise because they wanted to try something in their 20% time.

In this case, it's obvious that we've got server people working on mobile code. Future hardware will resolve these inefficiencies? UGH. That's definitely someone from the server world speaking.

But the thing is, nobody actually cares about this. I have used an Android phone since the beginning. Touch responsiveness is not something I have ever noticed. Any delay in rendering web pages is due to the network.

I think people that write articles about touch responsiveness or animation latency are suffering from the "benchmarking hello world" problem. Yeah, you could speed that stuff up. But 99% of the time, the phone is waiting for data on a socket, which is not going to get fast any time soon.

People care, they just don't know what to call it. People who use iPhones try out an Android device and say things like "it's just not as polished", or just notice that it doesn't feel right. That "not feeling right" is because when you scroll on an iPhone it really feels like there's a piece of paper under the screen that your physically moving around with the tip of you finger. With Android it just feels like your finger is telling a computer what to do, which is in turn doing a reasonably good job of doing it.

Twelve years ago or so, I was reformatting my home desktop (then a Linux machine) and I decided to try to change things up a little, so I installed FreeBSD to try it out. I figured that there wouldn't be any superficial differences, but I knew that there were architectural ones that I'd run into eventually and maybe I could learn a little more about how different OSes worked.

One thing I did notice right away was that the console felt different. It felt like it was maybe more responsive somehow, and not just when it was scrolling a wall of text. It even felt different when I was typing. Somehow, typing at a console (not in X, not via SSH) felt better in FreeBSD than it did in Linux. I couldn't tell you why, and I didn't really understand it myself. It wasn't something I could measure, or even describe, but it left me feeling like FreeBSD was more than 'just like Linux'.

I only used it for a day or so before I realized that I'd screwed up the partitioning and wiped out my Windows partition, and so had to re-reformat, but it left a profound impression.

I feel like iOS is the same way. All the Android phones I've tried (including a friend's Nexus One) have been, theoretically, the same as an iPhone in terms of features and capabilities and even hardware, but even before I got into using them in depth there was just something that felt a little off about tapping on icons or scrolling lists, and I could never put my finger on it, so to speak. Perhaps it's just design decisions or usage patterns that I'd internalized already, but I've always felt like it was something more. Maybe the scrolling behaviour is something I picked up on unconsciously, because while I don't remember it being slow or jerky, I do remember it didn't feel right.

Perhaps the numbers-oriented engineers at Google can't quantify it and so don't prioritize it, or maybe they just don't think it's worth wasting time on, but I think consumers, in the end, will notice. The real question is how many will actually care, and how many will shrug it off because of the benefits (perceived or real) that Android provides them over the iPhone?

Wow, I totally had the same experience with FreeBSD about 15 years ago. I have no idea if it persists on modern hardware, though.

Does anyone have a good explanation for this?

At that time the FreeBSDs scheduler was much better than Linux'. I had the same feeling about FreeBSDs responsiveness.

I had a similar perception with Solaris on SPARC. It was slower overall but more responsive when interacting on the shell. Sometimes I had high load (10 or more) and did not notice when using a shell in a terminal.

I think Linux got much better since then. I don't feel the difference on current FreeBSD, Solaris or Linux installations.

I'd suggest "not feeling right" has more to do with experience with a different platform, just as random decisions about mouse acceleration or font hinting or what side of the window to put close buttons can completely befuddle switchers.

You specifically state "People who use iPhones" and many others seem to think that the problem is only apparent if you compare it side-by-side. Which suggests to me that it's a fairly marginal concern, particularly compared with other Android flaws that the same people could be working on.

I switched from iPhone (to one of the lowest-end Android phones on the market) and didn't notice this particular issue. I did have general confusion at where things were and how to do things, but by the same token two weeks later my iPhone felt like it was missing a back button.

With subtle effects like this it's easy to sound like audiophiles talking about mp3 encoding. I'd suggest that if you're expecting the hoi polloi to pick up on these subtleties (which may well be imagined or exagerrated in some cases) and carry this fight for you then, like the audiophiles, you're going to be disappointed.

"when you scroll on an iPhone it really feels like there's a piece of paper under the screen that your physically moving around with the tip of you finger"

Thats one of the best descriptions of iOs scrolling I have ever read. It's exactly right.

People do care. They care about price too ;-) Android is heading to US$99 or even cheaper devices (in which I am not sure whether GPU is an option).

Perhaps, but for me, it's absolutely about usability. Whenever I see new Android phones or builds, the first thing I try out is how responsive the scrolling or zooming is. Often times, I'm waiting for Android to respond to my touch, other times my touches are missed completely. These issues cause unpredictability for me and make the phone harder to use.

As phones got faster, the situation improved, until a larger screen was added, which brought things back down again as the CPU burns to keep up. The milliseconds latency here and the stuttering FPS there decreases the usability significantly in my opinion.

I think that there is a group of users who don't notice these latency and animation issues. But I do, and I think many users still unconsciously attribute their dislike for Android to these issues. (example: my brother in law, used several Android Froyo phones extensively before settling on a BlackBerry because it felt "faster".)

I have never noticed any latency. I blame the manufacturer-specific Android "improvements", as I use plain AOSP and don't have any trouble.

My EVO 4G gets about 24 hours battery life, with moderate usage during the day. (But again, this is because I don't use HTC's build. With their build, the battery life is barely long enough to boot the phone.)

I've used vanilla Froyo on the Nexus One. It's dog slow. I don't think this problem is isolated to OEM-specific Android modifications (though they certainly don't help).

Responsiveness and framerate-wise, even a simple cursory examination will tell you: WP7 and iOS run laps around poor old Android. I got to play with a Nexus S recently, and the responsiveness is greatly improved over Froyo on N1... but is still noticeably less smooth than either Apple or MS's platforms.

It's an odd day when MS's UX is more polished than somebody else's...

One area where Android's lacklustre performance really hurts usability is diving into menus. iPhone came up with the novel idea of sliding menus - a simple animation that communicates a lot of context to the user (A belongs under B triggers modal window C, all based solely on the direction of the swipe animation). On Froyo+N1 this animation is jerky enough, frequently enough, that this relationship can easily be missed, resulting in more user confusion and worse UI comprehension overall.

Google needs to realize that we are in the year 2011, not 1995. "UI design" is no longer about placing the right buttons in the right places, and animation is often now the defining line between an easily comprehensible user experience and an obtuse one. IMHO graphics/animation performance is the #1 obstacle in Android's way, and the main advantage everyone else seems to have on it (and it's a huge advantage).

When I use a 2.2 phone, my impression is one of annoyance. I don't know if my touches will be detected, and even if they are, I don't know how long I'll have to wait for the phone to respond. Heck, I enjoy using WP7 phones more than Android because of the performance, despite the fact that WP7 is significantly less featureful than Android! Navigating WP7 is a pleasure, and it never feels like the phone stands in your way. I have yet to get this feeling from Android.

I turn animations off. Useless eye candy that just wastes the battery.

Also, is it really fair to compare the N1, a phone that's over a year old, to brand-new WP7 phones?

How about we compare the N1 (released Jan 5th, 2010) with the first gen iPhone (released June 29th, 2007)? Even the 2007 iPhone had smooth scrolling/compositing/animation and was always responsive to touch gestures. I'm pretty sure you can't claim a phone made three years before the N1 had an unfair technology advantage, right?

It's interesting how you railed on people earlier for mentioning that responsiveness and animations weren't great, and yet you have animations turned off.

I have a Droid Incredible. (Similar specs to the Nexus One) I don't have animations off. I have 5 home screens and 3 active home screen widgets (Beautiful Widgets and 2 different Jorte Calendar widgets.)

I don't have any problems with responsiveness or battery life.

I have a DI as well (with an undervolt/overclock kernel) and while the stutters are quite short, they're definitely there. As an easy example, scrub left/right quickly using the optical joystick on the stock home screen. If you're looking for it or paying attention, it's not that difficult to notice 2-3 frame pauses elsewhere but those are intermittent so I assume they're GC pauses. It doesn't hinder my enjoyment of the platform, but when you're looking for something to criticize it's an easy target.

As for battery life, it depends entirely on where you live. I experimented on it for months before finally figuring out that well over 50% of the power draw on the system is the data network (mobile network option when you hold down power). If you have it on when you have really weak/no reception, the phone will burn through the full battery in about 4 hours and it's pretty simple to lose 20% battery in 15 minutes. If you always have a decent connection, your battery life will be fine, if you happen to live in a place where you have holes in your coverage (e.g. NYC urban canyons), battery life is terrible.

After realizing that, I set up a set of Locale rules to shut down the network when I'm on the wireless network at home/work and to shut it down when I lose signal. Combined with the undervolted kernel, I normally get 48 hours on a charge for my normal use (~350MB/mo).

Yes, it is really fair. Both the N1 and the WP7 phones have the same screen resolution and the same first gen snapdragon SoC.

For perspective, the first-gen iPhone and iPod touch are perfectly responsive, even though they're ancient in gadget years. When there are Android phones which feel as slow as crud despite featuring the same (or better) hardware as an iPhone 3GS, you have a big problem.

Google has done a lot of cool stuff for developers in the last few iterations of Android. The NDK has evolved into something quite decent, definitely good enough for writing games now. But responsiveness is the major user experience issue that they've completely failed to address.

Well, no wonder you "don't notice touch responsiveness."

This is getting circular... let's not keep this up.

Samsung cared, and they spend X million dollars building new devices that made use of the hardware they had onboard because they cared. As a result they can run a browser at 720p smoothly on their phone.

Innovation doesnt come from asking people what they want. It comes from finding new markets and making new means.

Any delay in rendering web pages is due to the network.

many many webapps do not use the network, and any latency experienced comes from the page's own processing, the browser & the os.

The first thing I've noticed about Android was the overall slugishness of UI animation. And the guy who was showing it to me was amazed by the smoothness of my iPhone 3G UI. BUT YES OF COURSE NOBODY CARES.

If you actually look really closely, a lot of what iOS does with animations are about giving the illusion of responsiveness and smoothness.

I'm not saying things aren't incredibly responsive, but some of the animation techniques are used brilliantly to disguise lag.

Three examples:

1. Mobile Safari doesn't wait to render what it hasn't downloaded/formated before responding to touch gestures. You swipe and the page scrolls. It might display a checkerboard before it catches up with rendering, but the swipe command takes precendent.

2. Those wizzy transitions when double-tapping the home screen, or returning to the spring-board. They start immediately and actually mask what's going on behind the scenes with regard to processing a request. They're the ultimate illusionist progress-bar. And they need to be tuned finely to present the illusion or else it becomes obvious that they're masking tactics.

When people complain about the performance of new iOS versions on older devices such as the 3G, what they're really noticing is the illusion has been broken as Apple is optimising and fine-tuning the illusions for the latest hardware.

3. Those screengrabbed snapshots of apps that have been completely closed and display when an app is restarted. If the load time is short enough, the illusion of instantly re-entering an app is maintained, even if all you're seeing is a stale image of the app your closed earlier.

I'm not sure if Android employ any of these tricks, but they demonstrate the focus and undocumented nature of Apple's drive to create the illusion of perceived GUI 'smoothness'.

Well, of course. And if you think about it further, all user-facing software is, in fact, an illusion. Smoke and mirrors all the time. Apple just pushes the illusionist work even further than what most software developers (who are themselves illusionists, whether they realize it or not) consider as being "real". But it works, users prefer that.

Not only that, but I'm convinced that users are actually more efficient when perceived responsiveness is added: for instance, most users will not start thinking about what to do next on an app until they are seeing the app interface; presenting then with a non-interacting screenshot as soon as possible actually makes the overall interaction more efficient. But the ultimate is what the guys making PCalc discovered ( http://www.dragthing.com/blog/2009/07/how-to-make-your-iphon... ): you can put a non-interactive screengrab in such a way that even though it is a dumb image, touches will actually be recorded and have their effect as soon as the actual interface is set up; users can actually interact very soon. Now that's badass.

(however, most animation stuff doesn't fall in the category of perceived responsiveness, as most often, for instance for transitions, the end result must be ready before the animation begins; also, "undocumented nature"? Apple's obsession with perceived responsiveness is well-known).

Thanks for that link... I'm a designer not a developer, but I find articles like that fascinating.

(You are right about the other things you objected to in my comment.)

Whilst I agree with your main point, I have to nitpick your aside about running iOS on older devices. When I tried iOS 4.0 on my 3G (and the reason I'm now running 3.1.3 again) the UI would frequently completely freeze for several seconds. That can't be explained away as poorly-tuned animations.

One of the reasons I'd switched to Opera over Mozilla/Firebird/Firefox was because it remained consistently responsive. A page would be freaking out dog slow doing heaven knows what... in firefox, the browser would be frozen, in Opera, I could continue interfacing and issuing gestures and keyboard commands. Even on my old Fujitsu P1120 - a dog slow 800MHz Transmeta Crusoe beast with 223 MB ram - Opera, while not visually responsive, was at least queuing my operations, and would respond as fast as it was able. There didnt seem to be any event queue in firefox; if the browser was freaking out, keyboard commands &c would be outright dropped.

if you're dealing with lots of data from the Media Providers right on the phone, and you've worked for months on optimizing how to retrieve, cache, massage and display this data on lists, and you're still not getting a decent frame rate when scrolling through List Views you care about posts like these.

Never ocurred to me that GPU would help. I also come from a server programming background and hearing that GPU acceleration on the UI could help is like breathing fresh air.

There's several other more complex things you can deal with if you're writing apps that bring some actual value to the user, other than doing stupid web clients on android. It's not all waiting for data on a socket.

i'd throw ANDROID-1147 and ANDROID-1376 in the Android technical management "absolutely really just doesn't get it" bucket. otoh, the issue Charles Ying of Satine.org rages aobut, ANDROID-6914, seems much more questionable to me because the issue itself is really complex.

the two tickets, "Support for external keyboard, mouse" and "Browser doesn't render SVG images" both strike me as absolutely essential to making smartphones competitive with PC's. people have been happily attaching fold up keyboards to Palm Pilots for a decade now, SVG is perhaps the only sane route forwards for resolution independent apps and animation, and Apple has found religion on both these topics-- yet Android is obstinately behind.

otoh composited gpu acceleration is not nearly the huge obvious win C. Ying boasts it as. the gpu is more power efficient than the cpu for intensive applications, but a smartphones natural state is not intensive use. mr guy of android points out a number of complexities-- opengl drivers only supporting a single context, opengl acceleration proving less speedy than software implementations. it places huge emphasis on driver support, which any embedded wonk will tell you is a scary prospect.

although gpu acceleration is becoming better supported, keep in mind that the smartphone market expands, both up and down, with broadcom & others putting out chips designed for sub $75 smartphones. android is definitely not winning the race for the top atm, but performance has always been a rather lonely market-- that said, unsmooth experiences dont fly. software graphics focus insures a base level of performance for everyone, and strikes me as the right place to start to insure a speedy experience not subject to external interrupts.

the current silkiness situation is a little sad. it'll get better. more relevant to me is Android not playing in the ecosystem-- external peripherials, UPnP/DLNA, SVG-- these are the real things that will make or break not Android, but the open digital web, and Android is failing, badly, even in compare with non-web-enthusiasts Apple.

I'm talking out of my hat here a bit because I haven't actually worked with Android yet but it sounds to me that these problems might be related to the choice of Java as the core programming environment. I doubt that was a decision made by somebody working on a 20% project.

Although Java has its problems, I think the core of Android's issue is a bad utilization of computing resources. One can only hope that Google will eventually fix this problem.

Palm's WebOS put much of its core logic in JavaScript, which had a slower implementation (at the time) than Dalvik VM does. However, Palm carefully made sure to take advantage of the GPU to achieve many of the effects in the UI.

Have you guys tried MIUI on Froyo ? It gives me 60FPS for scrolling, panning and almost everything on a Nexus One: http://www.youtube.com/watch?v=Or-9T44Bt7M It has got NO GPU acceleration. That PWNs it

Anyone who thinks Android's animation is "fast enough" is either using a faster device than I have ever tried, or has never tried to compare, say, scrolling lists versus the iPhone.

I'm not saying the GPU is the answer, and maybe 2.3 is the answer, but ignoring the problem is not.

(Edit: So why does a GC pause stop animation, and not just by taking up CPU time, since moving to the GPU would help in that case? Is the animation code itself written in Java and waiting on the GC? If so, I bet rewriting it in C would be a massive help. Also, yuck.)

Gingerbread has a concurrent GC, so animation should not stop completely. However, on a single core system I wouldn't expect this to have a huge impact. But this is just my personal guess.

Or use a GC that doesn't need to stop the world. (Or at least only does so for very small intervals.)

Apparently touch responsiveness is improved in 2.3. However this article doesn't really explain what's wrong with it. I've been doing latency benchmarks for touch recently, and the highest latency I've seen for touch delivery is 10ms. Now 10ms is pretty terrible for what should be a very straightforward thing, but on the other hand it's only 2/3 of a frame, at 60 FPS.

The argument in favour of a GPU-accellerated GUI which is not mentioned here is battery life. I haven't been testing this side of things, but just because a CPU can do a 60 FPS GUI doesn't mean it should. Deferring it to the GPU would certainly reduce power consumption while navigating a UI. Whether this is non-negligible in terms of overall battery life, I'm not sure.

You're seeing sub-10 mSec response times on projected capacitive screens? Care to share what you're analyzing? Cypress PSoC needs a good 20-30 to get a lock.

That's interesting. I'm not measuring touches, I'm injecting events into the Linux kernel by writing to /dev/input/event? and measuring how long it takes for them to get to the application. That's because I am working on software optimisation, so I'm interested in the overhead introduced to events processing by the framework rather than the hardware.

User: I think this feels laggy.

Developer: No you don't. I have the benchmarks to prove it.

The argument in favour of a GPU-accellerated GUI which is not mentioned here is battery life. I haven't been testing this side of things, but just because a CPU can do a 60 FPS GUI doesn't mean it should. Deferring it to the GPU would certainly reduce power consumption while navigating a UI.

Both Charles Ying of Satine and wzdd seem sympathetic to this notion that the gpu is a power saving device. Surely it is when doing complex work. On the other hand, most smartphone usage consists of relatively simple screens and basic if any transitions. If it's only going to take 3ms of CPU work to run a 150ms animation, by all means, figure out how to keep the OS from interrupting the high priority graphics task and make the CPU do it. Particularly if the GPU would have to be on across the entire 150ms. If not, what is the cost of copying the graphical content between the GPU and the CPU? How much CPU does it take to initialize and set up the GPU for this extremely simple task you ask of it? Is the CPU going to be able to sleep while the GPU is running the animation, and if not what kind of context switching and control costs are going to be imposed on the CPU to manage the graphics state?

We've barely begun to see multi-core cell phone GPU's, but the point of the GPU is basically to run lots of work at a nice low clock rate where voltages can be dropped. If you dont have a lot of work to do, there's really no sense waking up the GPU and having it's extremely dumb poorly-branching cores chug away at figuring out, say, the SVG animations that a semi-competent ARM core could knock off right quick. You're just wasting power turning on the GPU.

Smoothness and polish, I suspect, are much more a questions of resource allocation than CPU capability, particularly when you've got a sub 640x480 surface and a 1GHz core.

Are those 10ms with a realistic app/usage scenario ? I wouldn't think users would complain about 10ms, isn't that below the threshold of perception ?

A stutter 2/3 of a frame long means there's a 2/3 chance that you'll draw a frame that doesn't match the current finger position, followed by a frame that catches up more than it should have had to.

The ~40ms input lag shown by most desktop IPS LCD monitors is noticeable even though it falls below the reaction time. On a touch display, it should be even easier to notice input lag, because you can see your finger moving and the screen around it not keeping up. 10ms sensor lag may be below the threshold of perception for circumstances like this, but probably not by much. If you add even a few milliseconds of processing lag after your app receives the touch event, you'll be behind by a frame.

I work with digital audio/MIDI and a latency of 15-20 ms between pressing a key and hearing the sound is easily audible making it near impossible to play expressive music. 10ms can be felt and is 'annoying', under that is acceptable. I usually keep latency down to 5ms, at most 7ms, only letting it increase, e.g. during mixing when responsiveness is not utterly essential.

Actually, I wrote about battery life at the very end of the article. I already discussed battery life in a previous post. I certainly feel that battery life is one of the most important issues in mobile, hence, "efficiency"

As for touch responsiveness, I don't have enough current knowledge of Android's situation and touch hardware in general to write on that topic.

This may be related to overall linux interactivity scheduling. There has been a running debate on the Linux kernel mailing list about this for years. The Linux scheduler is more efficient for server-workloads and compiles, but it is not as good for interactivity.

Very recent kernels have a new scheduler which supports a new scheduling algorithm which people are anicdotally reporting better interactivity with. Perhaps this will help the situation.

I think the android bug linked in the article is much more informative than the article itself.

Basically: 1) Android phones can already animate the screen @ 60 fps in software in most cases. 2) When they can't, it's not because the CPU is too slow, it's because there is other stuff happening (GC, etc.). A GPU can't fix that.

The GPU can offload most of the graphics intensive tasks from the CPU, utilizing the GPU instead of letting it sit idle. freeing the CPU to focus on GC tasks. In addition, most of the animation work done with Dalvik code can be moved into a dedicated thread / task with highly optimized code backing it.

I think the author nails it. Without GPU acceleration Android really is a notch below iOS in terms of UX. In my opinion, even WebOS and Windows 7 have much smoother scrolling and touch response than Android. But if Google wakes up and adds GPU acceleration and better eye-candy, watch out Apple.

Purely by chance, I was trying out a galaxy tab at a store today. I loaded up the Financial Times web site, which took an age because the store wifi was crap and tried scrolling the web page and it stuttered quite a bit at first then smoothed up, zooming as juddery as well. Did the same thing on an ipad - slow loading about the same, but the instant the page was rendered it scrolled and zoomed perfectly.

Also the widget for editing the web page URL on the Tab was a horrible usability car crash. What's up with that? Disclosure - iPhone user, but Android curious.

What is the main performance difference between Vista SP1 and Windows 7? Hardware accelerated UI.

Android's VM can improve perf-wise - see http://blogs.oracle.com/javaseembedded/2010/11/how_does_andr...

It doesn't matter what the solution is, but Google need to prioritise UI responsiveness above all else. Until that happens, the UI on Android will suck.

Applications are open for YC Summer 2023

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact