
Android’s 10 Millisecond Problem explained - lmedinas
http://superpowered.com/androidaudiopathlatency/
======
S_A_P
Reading about this takes me back to the late 90s. I had just gotten serious
about recording music on my computer and purchased an 866mhz dell pc. I was a
using windows 98 and ASIO was still mostly a Steinberg only development and I
couldn't afford cubase. I got a hand me down version of cakewalk pro audio 8.
I remember the latency of that set up was about 35ms and thinking this is
pretty good. There was some degree of having to start playing just slightly
early so you could stay on time. With guitar and keys this was doable but I'm
sure it would have been maddening with drums. I think it was when fl studio 3
came out that I had the first "daw" that supported ASIO. I remember being
blown away at how much tighter everything sounded from that point on.

For those interested in why iOS is better than android, a good summary would
just be to say Core Audio was designed from the ground up to be a low level
low latency audio API. There are fewer layers in it vs android. When designing
OS X Apple knew they had a large professional media market so that got
priority. I am interested to see the results of these guys efforts and have
wondered when an ASIO equivalent would pop up for android.

~~~
daddykotex
Is it possible that their Audio API is much simpler due to the fact that they
control all of the implementations (Hardware interface) of iOS?

~~~
ianlevesque
With Android the variety of customized android builds on various manufacturers
does make this more challenging than it needs to be. As the article points
out, many vendors simply break realtime audio by using slow drivers and slow
paths.

However, as someone who has dealt with Android, iOS, Windows Phone, and BB
audio APIs, Android's audio API really is just amateur hour compared to
Apple's, even to the present day. In older versions it was simply unusable for
anything more complicated than playing a sound effect or fixed-length mp3
file. Today it functions for more use cases but still abstracts concepts (like
codecs and container parsing) in flawed and inconvenient places.

Windows Phone and BB are still even worse though.

~~~
ElijahLynn
Do you think Ubuntu for Phones (or whatever it's called) will suffer from this
too? Since Ubuntu is using a portion of Android.

~~~
ianlevesque
I'm not that familiar with Ubuntu Phone, but from a look at the docs I am
concerned that it may have some of the same media issues (or at least the same
variations between vendors). According to the architectural overview in the
porting guide[1] their Qt-based API sits on top of some abstractions of their
own, which then sit on top of stagefright, OMX, and Android's HAL (and thus
Android's drivers). Not reassuring.

I'd personally be much more at ease if I saw ALSA sitting directly under Qt,
but I can understand why they'd want to leverage the huge Android hardware
ecosystem.

1\. [https://developer.ubuntu.com/en/start/ubuntu-for-
devices/por...](https://developer.ubuntu.com/en/start/ubuntu-for-
devices/porting-new-device/)

------
code_duck
Interesting to read of the details behind this issue. This has been a serious
issue for me - it's actually why I own an iPhone.

My first modern touch device was an iPod touch 4. I downloaded Garage Band
and, as a long time milt instrumentalist and composer, loved it. I was amazed
by how well the touch instruments worked and how easily I could record riffs
and flesh out small snippets of songs. It ran almost flawlessly on the 4th gen
Touch.

Next, I decided to buy an Android phone - a Motorola Droid 2. I was surprised
to find that despite the power advantage over the iPod, none of he music apps
I tried were usable. The drum programs, for instance, we're so lacy and
unpredictable as to be worse than useless. Hit a drum, and you may hear it
seemingly instantly, maybe 1/8 a second later, maybe half a second, possibly
never. Meanwhile the tiny iPod could play Garage Band instruments so well one
could use it for live performance.

I upgraded my phone twice, first to a Droid X, then a Galaxy S3... Each time
was disappointed that the improved specs gave no improvement in the terrible
audio performance.

Currently I have an iPhone 6 so I can use Garage Band. Kudos to apple for
doing this right - it's the best app I've ever used.

~~~
astral303
Thanks for a musician's perspective!

------
kllrnohj
Google has a whole section on this here (if you want more technicals and less
sales-pitch):

[http://source.android.com/devices/audio/latency.html](http://source.android.com/devices/audio/latency.html)

including a bunch of measurements of Nexus devices here (even going as far
back as the Nexus One on Gingerbread):

[http://source.android.com/devices/audio/latency_measurements...](http://source.android.com/devices/audio/latency_measurements.html)

~~~
tim333
And here are some measurements for iOS as well (Superpowered Mobile Audio
Latency Test App for Android and iOS.)

The iOS devices come out at 6-18ms, Androids at 17-860ms. The faster Androids
have Samsung's Professional Audio SDK.

[http://superpowered.com/latency/](http://superpowered.com/latency/)

------
ynniv
TL;DR the Linux layer (ALSA) and the Java layer (Audio Flinger) use widely
compatible but high latency techniques, whereas Apple designed their API's and
hardware such that these layers can be optimized to almost nothing. (From the
article: [http://superpowered.com/wp-
content/uploads/2015/04/Android-A...](http://superpowered.com/wp-
content/uploads/2015/04/Android-Audio-Path-Latency-Superpowered-
Audio700px.gif))

~~~
cnvogel
ALSA is not the problem, it works very fine for the low-latency case, I can
reliably run soundcards with light processing load at 96 kHz @ 64
frames/period (2 _0.7ms -- > 1.4 ms latency) on a quad core i5, e.g. for
running a reverb effect, but most of the time I only record and will settle
for 1024 frames/period or so. (2_10ms --> 20ms). The period size, just for
completeness, is the number of samples recorded on each block that is
forwarded to the audio processing application.

If whatever audio framework you use doesn't allow to run processing with a
input-to-output delay (latency) of two times the period size, it's broken
(probably the case for Audio Flinger at Android, don't know much about it).

    
    
        ➜  ~  jackd -d alsa -p 64 -r 96000
        jackdmp 1.9.10
        Copyright 2001-2005 Paul Davis and others.
        Copyright 2004-2014 Grame.
        (...)
        creating alsa driver hw:0|hw:0|64|2|96000|0|0|nomon|swmeter|-|32bit
        configuring for 96000Hz, period = 64 frames (0.7 ms), buffer = 2 periods
        ALSA: final selected sample format for capture: 32bit integer little-endian
        ALSA: use 2 periods for capture
        ALSA: final selected sample format for playback: 32bit integer little-endian
        ALSA: use 2 periods for playback
    

(this is on my laptop, just for illustration purposes)

~~~
jblow
But ... why is there a period size? Isn't that a broken design that can only
introduce latency? What is wrong with "however much audio data is ready when
the application asks, send it"?

~~~
cnvogel
Well... it's like all audio chipsets work nowadays. Only some DSPs will be
able to efficiently handle single-frame data processing, but they have the
help of dedicated address generators and lightweight interrupts synchronized
to the digital interface.

If you write "process however much audio data is ready", then you already
imply that your CPU will _not_ be up to speed to process 48000
interrupts/second reliably and you need some buffering.

And if you have to assume that sometimes you'll miss 100 samples (which, then,
you'll process en-block), this means that to work reliably, you'll have to
start at least 100 samples early so that you don't miss the deadline of the
DAC, because the DAC _will_ , with intractably output one sample every 48000th
of a second. This already implies some kind of periodic processing of blocks,
doesn't it?

(and yes, such a scheme will theoretically allow you to half the latency from
something to 2 _period-size to 1_ period-size + the time for processing)

Third, a lot of the algorithms for processing audio can be implemented much
more efficiently if you have a known block size and don't have to calculate
your filters or convolutions with constantly changing number of samples for
every step.

Also efficiency of processing will decrease (reloading the cache after each
interrupt when switching from processing plugin to processing plugin), so the
time spent on calculating per frame will go up if your period size gets
smaller. At one point you'll need exactly one "period size" to calculate one
period size worth of samples: That's the maximum your machine can handle, and
at that point you'll have a latency of your "period size"*2, which is exactly
the same as running with a fixed period-size ;-). And as you can choose the
period-size rather freely (maybe completely arbitrary, maybe 2^n, depends on
the chipset/hardware) there's no disadvantage left.

~~~
jblow
As someone who has actually done a good amount of soft-real-time audio
programming, I can tell that you probably haven't. Everything you are saying
about CPU speeds is made-up nonsense. Look into how these things are done on
systems where folks actually care about latency (for example, commercial audio
hardware, game consoles, etc).

~~~
jblow
I understand that people are downvoting this because it is just a negative
comment, or something. But, I felt it was VERY important to call out
information that is clearly false. Someone who doesn't know about audio
programming might read the above post and think "hey that sounds plausible, I
learned something today" when in fact they were deeply misled. Registering
dissent is important and I tried not to be rude about it. I did go on to give
a sketch of reasons in the thread below (but it is a complex issue with a lot
of details; exact situations differ on every platform; etc, etc.)

~~~
cnvogel
I didn't downvote you and was genuinely interested in why you were considering
my information to be incorrect. And I now realize it's because I've always
worked with systems where processing is always strongly synced to the central
frame/sample/... clock. Also I read your initial comment as "why don't we use
'process every single sample' to reduce latency at all costs" which is -as you
wrote- clearly a bad idea. Sorry for misrepresenting that.

------
lnikkila
If you’re interested in this topic, you should listen to episode 20 of the
Android Developers Backstage podcast. [1]

Raph Levien talks about audio latency and how they started working on
minimising it in Lollipop. He mentions that it’s still an ongoing process.

The relevant part starts at 35 minutes into the episode.

[1]:
[http://androidbackstage.blogspot.com/2015/01/episode-20-font...](http://androidbackstage.blogspot.com/2015/01/episode-20-fonts-
and-audio.html)

~~~
mackey
Also this:

Related presentation at Google I/O 2013.
[https://developers.google.com/events/io/2013/sessions/325993...](https://developers.google.com/events/io/2013/sessions/325993827)

This was probably the best presentation I saw during Google I/O.

------
dottrap
This is a great description of the lowest level problem. But for those
wondering why iOS doesn't have the problem or why only 10ms is such a
difference, the answer is that this is only the beginning of the problem of
Android audio latency.

There are still additional problems that add latency that span the entire
Android stack from the actual hardware and drivers, to the kernel and
scheduler, to the Android implementation of their audio APIs.

Anybody who has tried to do any serious audio on Android knows the infamous
Bug 3434:

[https://code.google.com/p/android/issues/detail?id=3434](https://code.google.com/p/android/issues/detail?id=3434)

Google I/O 2013 did a pretty good talk on the problem and shows how there are
problems across the entire stack. Glenn Kasten pretty much carries the brunt
of all the audio problems with Android. I find it telling that he had to
handcraft his own latency measurement device using an oscilloscope and LED
because there were no actual tools built into the OS to help them analyze
performance.

[https://www.youtube.com/watch?v=d3kfEeMZ65c](https://www.youtube.com/watch?v=d3kfEeMZ65c)

Audio has been terrible since Android's inception. It has improved a little
over time, but unfortunately, 7 years later is is still pretty much
unacceptable for any serious work.

~~~
rasz_pl
Whole Androids latency has been terrible since its inception,
screen/input/audio.

Android is famous for $600 2GHz phones that drop frames when doing simple
animations :(.

------
troymc
There is something I don't understand; maybe someone here can explain:

Sound travels at about 340 m/s (in a typical room). That means it travels
about 3.4 metres in 10 milliseconds. Therefore another way to get a 10
millisecond problem is to stand 3.4 metres from the orchestra.

Most people sit farther than 3.4 metres from the orchestra, yet they don't
complain about a lag between when the violin bow moves and the sound is heard.
Why not?

(The speed of light is so fast that we can assume it's effectively infinite
for the purposes of this argument.)

~~~
Synaesthesia
I'm a musician, 10ms latency would be fine, however as they note, most Android
apps have 100ms latency, or 200ms round-trip latency. That is definitely not
usable.

~~~
izacus
Huh, those tests show it has 35-50, not 200ms O.o

~~~
on_and_off
I am too lazy to try to find the exact numbers and versions but back in the
Android 1.x, 2.x days, the latency was in the 200 ms ballpark. So things have
improved a lot since then (even though there is still a lot of ground to
cover)

~~~
abraham
[https://source.android.com/devices/audio/latency_measurement...](https://source.android.com/devices/audio/latency_measurements.html)

------
rglullis
Isn't this kind of a known issue in Linux land? As much as it has improved,
latency has always been the bane of audio applications in the Linux kernel. I
remember in the days of kernel 2.2 that even XMMS would stop playing any music
if I started using more than one or two applications.

Recently I got one of those cheap USB interfaces to connect my guitar. I spent
some good 4 hours changing the kernel to "low latency" one provided by ubuntu,
then trying to setup aRTs to run, then trying to make pulseaudio work with it,
then figuring out how to keep both aRTs and pulseaudio dependent applications
happy. In the end I got most of it working and I could run some kind of guitar
effects application, but the next minute I realized that the volume control
media keys stopped working. It was enough for me to throw the usb interface in
the drawer and give up on linux for audio applications on workstations: buying
a 50€ effect pedal was cheaper than all the time devoted to it.

~~~
digi_owl
>I remember in the days of kernel 2.2 that even XMMS would stop playing any
music if I started using more than one or two applications.

Back then the sound subsystem didn't do any mixing or similar, so if some
program grabbed /dev/snd, everyone else had to wait.

As for low latency sound work on Linux today, Jack is what you want rather
than pulseaudio. Frankly Pulseaudio is a massive detour when it comes to Linux
audio.

~~~
BFay
Pulseaudio has gotten a really bad rep, and I think there was at time where it
was legitimately awful, but I think it's better than it was.

On mainstream distros with pulseaudio like Fedora or Ubuntu, audio just works,
when you don't have low-latency requirements.

When you do have low-latency requirements, things are a bit tricky. You do
pretty much need to get a low-latency or realtime kernel, and you definitely
want to use Jack or ALSA. Maybe this setup is a little more frustrating than
Windows, where you may just need to install one driver like ASIO4ALL.

But it's flexible, and you can actually get Pulseaudio and Jack working pretty
well together after installing the pulseaudio-module-jack. You can turn on
jack when you need it, turn it off when you don't, and all of the pulseaudio
stuff will get routed through it so that you don't lose sound from your other
applications. If you want, you can route audio from pulseaudio into whatever
other audio applications you're using - sometimes I like taking Youtube videos
and routing the audio through weird effects in Pure Data.

I toggle jack on and off with this little script, works pretty well for me
right now:
[https://gist.github.com/YottaSecond/f0a1b515f95b2e791755](https://gist.github.com/YottaSecond/f0a1b515f95b2e791755)

~~~
aidenn0
Ubuntu is the reason PA got a bad rap, and PA is an example of why I don't use
Ubuntu.

When Ubuntu adopted PA, the readme file still described it as "the sound
server that breaks your audio"

It was the most mature thing that had the features that Ubuntu wanted, so they
adopted it despite the fact that it was clearly not yet ready for prime-time.

That all being said, PA is not the choice if you want to do DAW style stuff;
it tends to prefer lower cpu utilization to lower-latency.

------
vlaskovits
Hi everyone -- Gabor and I wrote the piece -- let us know if you have any
questions we can help answer for you.

~~~
madez
What are the concrete steps you are taking to tackle the issue?

Idea: A latency ranking for devices would put pressure on manufactures.

Will you work together with the Linux community so we all benefit from it?

~~~
vlaskovits
Hi there -- latency ranking you ask?

As Westley from The Princess Bride might reply, "As you wish"

[http://superpowered.com/latency](http://superpowered.com/latency) :)

~~~
madez
Awesome, thank you!

It's remarkable how apple cares about certain quality aspects of their
devices. They would be interesting options if they wouldn't jail and lock you
down :/ However, I wonder why newer and supposedly faster devices like the
iPhone 6 have higher latency. I'd think to surpass the latter generation would
be the goal for each successor.

Also, it seems Samsung is taking the challenge seriously.

------
sandGorgon
interestingly, a 2012 article[1] by Arun Raghavan mentions:

 _On the Galaxy Nexus, for example, the best latency I can get appears to be
176 ms. This is pretty high for certain types of applications, particularly
ones that generate tones based on user input. With PulseAudio, where we
dynamically adjust buffering based on what clients request, I was able to
drive down the total buffering to approximately 20 ms (too much lower, and we
started getting dropouts). There is likely room for improvement here, and it
is something on my todo list, but even out-of-the-box, we’re doing quite
well._

Let's get systemd on android and then see !

[1] [http://arunraghavan.net/2012/01/pulseaudio-vs-
audioflinger-f...](http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-
fight/)

------
Polyphonie
One of the reason for the audio latency issue on Android is the complex power
management circuitry (PMIC) implemented on most of not all Qualcomm SoCs.
That's why to anyone with experience in regards to using Android device in
audios production, most would agree that the older 2011 Galaxy Nexus is still
the Android device to use. Unlike the the recent Snapdragon based Nexus
device, the GNex is based on TI OMAP4460 (dual-core ARM Cortex A9). It might
not be a coincidence the Apple kept the number of cores on its A series SoCs
to a minimum (only moving to 3 with the A8X).

[http://createdigitalmusic.com/2013/05/why-mobile-low-
latency...](http://createdigitalmusic.com/2013/05/why-mobile-low-latency-is-
hard-explained-by-google-galaxy-nexus-still-android-of-choice/)

------
nness
Can someone explain in layman's what the issue actually is, because the
article, neither its comments here or on Reddit, seem to dumb it down for non
audiophiles?

~~~
analog31
Audio processing on Android devices is Slow. Even seemingly small delays (10
milliseconds) is enough to make apps seem clunky and erratic. This puts the
Android platform at a disadvantage for developers whose product ideas involve
delivering high quality audio performance.

The embedded graph with the U shaped figure pretty much sums up the author's
search for the components in the software that are the culprits.

~~~
valdiorn
> Even seemingly small delays (10 milliseconds) is enough to make apps seem
> clunky and erratic

Not just that, if you try to play guitar with a 10ms latency from input to
output, you'll find that it's impossible to keep rhythm at all. It's the same
effect as a speech jammer:
[http://www.stutterbox.co.uk/](http://www.stutterbox.co.uk/)

And I don't think many people outside the the tech audio community realize how
many people have ditched their guitar amplifiers and are playing guitar purely
through the iOS devices. It's a great use case for tablets, but is completely
infeasible to do on Android at the moment.

~~~
vkjv
> And I don't think many people outside the the tech audio community realize
> how many people have ditched their guitar amplifiers and are playing guitar
> purely through the iOS devices.

I'm super interested in this question because I honestly don't know. My
assumption is that among serious guitar players (defined as individuals who
play in groups or in front of people at least monthly) the number is almost
nil.

I'm sure the number of casual guitarists (those who rarely play, but
technically own one), this number is quite high.

I'm unwilling to even concede the tubes in my amp let alone the amp itself...

------
cfstras
Going completely off-topic: Why has everyone recently decided to use ultra-
thin fonts everywhere? On my system (24" FullHD, Win8.1, Chrome 42 and Ubuntu
14.04, Chrome 43) the text is thinner than one pixel and thus unreadable below
130% zoom. Sure, I can open up DevTools and fix the font-weight, but
seriously?

~~~
emn13
My theory on this is that Windows 8 hyped this.

It's particularly problematic when it's being copied in mac circles because
OSX has a biased font-smoothing algorithm that adds font weight (it's a design
flaw). In other words, to get something to look thin on a mac, especially a
non-retina mac, it needs to be _very_ thin, sometimes less than a pixel thin.
How other systems display borderline visible strokes varies depending on the
system and the details of the font.

~~~
cfstras
Oh, so I can tell whether the designer uses OSX by the fact that I can't read
the fonts. That's nice.

------
Lewton
This squares pretty well with my personal experience. Pretty much every music
geek I know chose an iPhone over an Android because of the music apps

~~~
buro9
I'm a music geek, I just chose to carry a dedicated music player.

~~~
Someone
The issue isn't about having your phone play music, it's about you playing
music on your phone, in some cases with friends who play on their own phones.

~~~
buro9
Ah, for that I have a workstation. But as my guitar playing is very poor it
doesn't help a great deal.

~~~
dvfjsdhgfv
It seems you're talking about two different things still. If you have an iOS
device, you can connect an electric guitar to it via a dedicated (but
relatively inexpensive) interface and have it sound like a reasonably amp plus
some effect processors. It's really great for mobile musicians. I don't even
mention Garageband and other apps which basically allow you to compose quite
decent music on the go.

------
mox1
I spent a __lot __of hours fighting with the low level Android libraries while
writing a Spotify App for Android back in late 2013. There is very little
documentation on the web (especially for streaming Audio). Getting this
working was a serious headache.

If anyone is interested, I pulled the OpenSL ES parts out and posted them to
github.

[https://gist.github.com/mox1/3516112bc52c54aabb98](https://gist.github.com/mox1/3516112bc52c54aabb98)

------
guelo
Up to the ALSA driver step these delays would be the same on any Linux system.
Do Linux desktop systems experience these types of delays? I have experienced
these types of delays trying to setup a Windows box as karaoke machine. In
fact, I've never seen a DJ use anything but a Mac. That leaves the question,
how does Apple do it?

~~~
krakensden
ALSA isn't the problem- JACK runs on top of ALSA, and people comfortably run
well under 10ms. The problem is higher up in the stack. Even switching to the
much-maligned pulseaudio could be a huge improvement:
[http://arunraghavan.net/2012/01/pulseaudio-vs-
audioflinger-f...](http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-
fight/)

------
stinos
So if I get it correctly the problem is twofold: there is some extra
intermediate processing _and_ the buffer size and sampling rate are fixed to
256 samples and 48kHz respectively? And which of these two does Superpowered
fix? Or both? And what would be the lowest possible latency on for example the
Nexus 9?

~~~
cnvogel
The lowest possible input-to-output latency of an audio workstation is always
two times the period size (samples/buffer) (plus a few microseconds of
internal delays in the ADCs, controllers, ...)

The sound chips operate on integral periods or a fixed number of samples, and
when you get a period worth of audio data from your ADC, the DAC will already
have started putting out the first samples of the next period. Hence, you
prepare your audio samples for the second-to-next period.

Assume you have audio processing code roughly looking like this:

    
    
        while (1) {
            poll(); /* some API function waiting for the "next period" */
            read(soundcard, block_of_samples);  /* or let DMA do it */
            process_samples(block_of_samples);
            write(soundcard, block_of_samples); /* or let DMA do it */
        }
    

Let's try some ASCII art:

    
    
          v- audio samples going into your soundcard
           _     _     _     _     _     _     _     _     _     _     _     _
          /1\   /2\   /3\   /4\   /5\   /6\   /7\   /8\   /9\   /0\   /1\   /2\ ...
             \_/   \_/   \_/v  \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \
                            |
          \________________/\________________/\________________/\________________/
           Period 1          Period 2          Period 3          Period 4
                            |
                            |
                            [*] here, Period 1 has been DMA'ed from the soundcard
                                to an mmapped buffer of your audio application
        
                              <~~~~~~~~~~~>  here processing of your audio takes place
                            
                                           [*] this is the latest point at which processing
                                             | must complete so that there will be data for
                                             | the soundcard to output. DMA will start
                                             | from the buffer to the soundcard DAC.
                                             |
           _     _     _     _     _     _   v _     _     _     _     _     _
          / \   / \   / \   / \   / \   / \   /1\   /2\   /3\   /4\   /5\   /6\ ...
             \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \
        
          \________________/\________________/\________________/\________________/
        
                                               ^- processed audio samples will come out
                                                  of your laptop's speakers...
    

Obviously, your machine has to be fast enough to do the whole real-time audio
computation in a little less than the time between two interrupts, that's the
period size. And it must reliably be able to do this, because if it misses the
time to have a block of samples ready for the DAC. If it misses that goal, a
"underrun" will take place, and the audio application will have to
resynchronize, possibly causing some clicking, intermittent audio, ...

~~~
stinos
+1 for this fine explanation. However I'm well aware of how this works and my
actual question was really what part of the problem in achieving this ideal
scheme Superpowered is solving. Or are they just targetting <10mSec latency?
Are they getting rid of intermediate layers? Or are they just lowering
buffersize? Or both?

------
ohitsdom
Is 10ms really that big of a deal? I'm an amateur musician so have some
experience playing in bands, but I have a hard time believing 10ms would feel
off when playing with others.

~~~
marquis
9ms is the industry-recognised time that it takes before the brain notices
that sync is out. If you have 2 woodblocks clapping and they are more than 9ms
out of sync, you will hear one as an echo instead of 2 blocks at the same
time. So if you are a band wanting to play remotely and you have 100ms network
latency plus all the other latencies of the software layer, it's not going to
be possible. Similarly if you are trying to drum in time with a track: more
than 10ms latency from when you tap your finger to when you hear the audio,
you'll have a hard time staying in sync.

~~~
hyperpallium
IIRC it's a similar latency figure for VR headtracking (15ms), and also for
touch screens (10ms).

------
DigitalSea
This is the one downside with using Android I have experienced. I came from an
iPhone 4 and as an amateur musician I used it to produce music, record ideas
on the go and whatnot. When I moved to a Samsung Galaxy S4 (my first Android
phone) I was mortified. I grown to love a couple of drumming applications on
iOS, when I moved to Android there were a few sub-par applications and they
all had ridiculous latency, to the point they were not usable and I had to
bust out my keyboard at home to record drums (no drum ideas on the go
anymore). You hit a snare in a drumming app and it felt like hundreds of
milliseconds of lag, horrible.

It was so bad to the point where I went out and purchased an iPad just so I
could have that freedom of recording ideas when I am not at home. The iPad and
iPhone offered what sounded like basically no latency at all, I never
understood why Android devices struggled (but I speculated and assumed it was
how the audio was being processed). I considered moving back to an iPhone, but
I love the freedom that Android affords me and the competition, so I stuck it
out and kept using the iPad.

I recently purchased a Samsung Galaxy S6 Edge and while I still notice some
slight latency, it is usable again. I can finally jot down ideas when I am
away from home using my phone again. Research seems to yield some improvements
that Samsung themselves have made to their hardware and software, not to
mention the professional audio driver that allows the use of a third party
audio interface (great feature by the way). Google needs to make this a
priority, because believe it or not a lot of people use their tablets and
phones to produce music. We need to fix this.

Music production might not seem like a big deal to Google, but Apple
definitely gets it and have from the beginning. The one aspect I miss about
owning an Apple device, not enough to make me switch back but definitely a
good feature of iOS devices.

------
afandian
Anyone with any insight about how Apple does this?

~~~
miahi
Probably by custom and known hardware. Let's say it takes 1000 working hours
to produce a low latency driver/firmware; Apple only needs to do that for a
few devices, but every Android manufacturer has different audio chips and most
of them only care that audio works, not that it works well - so they do not
invest that time for each device type. Having known hardware also works for
the application developers, because they can optimize their code for the
specific device (knowing that you always have a 40ms latency is way better
than knowing that the latency is between 35 and 300ms).

Also, Apple is much more motivated to get the audio path right, having started
with the iPods and selling a lot of music.

~~~
madez
The latency problems arise with ALSA (mostly hardware-independent) and
AudioFlinger (totally hardware-independent).

~~~
kllrnohj
That's technically correct but an extremely inaccurate summary.

The _reason_ ALSA and AudioFlinger add latency is to _hide_ hardware-dependent
differences as well as kernel-caused scheduling issues & policy decisions.

To achieve low-latency you need real-time scheduling, something Linux has with
SCHED_FIFO but it's a bit kludgy, and getting the policy right on that is
tricky (obviously you don't want a random app to be able to set a thread to
SCHED_FIFO and preempt the entire system). So you have to restrict the CPU
budge of a SCHED_FIFO thread, and you have to only allow apps to have a single
SCHED_FIFO thread. But how much CPU time you give it needs to depend on the
CPU's performance in combination with the audio buffer size that the
underlying audio chip needs (and those chips also have different sample rates,
is it 44.1khz or 48khz or etc...).

tl;dr: this is insanely hardware-dependent.

------
kchoudhu
Any word on how Windows Phone does on the latency metric?

~~~
ygra
There is this thread: [https://social.msdn.microsoft.com/Forums/en-
US/14f96b68-60e3...](https://social.msdn.microsoft.com/Forums/en-
US/14f96b68-60e3-47a8-8d96-9cf6fe2c347a/lowlatency-audio-on-windows-
phone-8-hint-forget-it-for-now?forum=wpdevelop)

and it seems to be that anything below 140 ms latency seems impossible on
Windows Phone.

------
jongraehl
Compare to a nice grand piano: 1 meter of air ~3ms. (hammer travel after you
bottom out the key - another 10ms? A MIDI keyboard doesn't know the velocity
of the note (or send the note at all) until key travel is mostly done too.)

I have only 5 ms of ASIO buffering using my PC but I don't actually know how
much actual key->sound latency; I do know that using headphones it's only
slightly less immediate than a nice grand.

I think the low MIDI bit rate (31kbaud) also adds a little latency on chords.

It would be nice if keyboards in the future immediately sent a lower
latency+precision keypress-initiated notice (before full key travel) so disk-
based samplers can make ready for that note (and make any initial attack sound
that's appropriate).

------
guscost
Great explanation, thanks for taking the time to write this up. A while back I
figured that there had to be complicated structural reasons for the lack of
progress on the notorious "issue 3434", and I decided to go with native iOS
for mobile audio projects rather than wait. Seems it will be quite a difficult
problem to solve (although perhaps a library that is not totally backwards-
compatible would be easier to optimize). But kudos for taking a crack at it,
and good luck. I'm interested to hear how it goes.

------
Terretta
> _“Consumers ... have a strong desire to buy such apps on Android, as shown
> by revenue data on iOS...”_

That's like saying, “Consumers have a strong desire to buy gourmet steaks from
McDonald's, as shown by revenue data from Ruth's Chris.”

No, McDonald's serves billions of meals by understanding its own market, not
by catering to diners at Ruth's Chris. And naturally, comparing top sellers at
each will give very different lists.

~~~
DanielStraight
This is not a good analogy. McDonald's and Ruth's Chris are not direct
competitors, they are targeting completely different markets. Android and iOS
are in direct competition.

If we have to use a food analogy, I would propose Starbucks and a competing
coffee shop. Almost all the major players in app development make
corresponding Android and iOS versions of their apps. Imagine if Twitter or
Snapchat only had an iOS or only had an Android app. Similarly, customers
expect to have certain things available at all coffee shops, regardless the
brand behind them: cappuccinos, flavored syrups, alternative milk choices,
etc. So a better analogy would be one coffee shop chain not supplying their
stores with espresso machines, or flavored syrups, or alternate milk
choices... or maybe just not letting them have refrigerators at all to keep
milk cold.

The prevalence of orders for highly-sweetened, cold, milk-based drinks at
Starbucks almost certainly means that customers would order the same thing at
similarly positioned coffee shops if it was available, and indeed most coffee
shops offer such drinks now.

There is no reason to think that somehow Android users are in such a different
market that they would have no interest in these apps on Android even though
in almost every other case platform parity is expected from large players in
the app market.

~~~
patrickaljord
> This is not a good analogy.

I'd say it's a bad analogy because most people who don't live in the US have
no idea what Ruth's Chris is.

~~~
megaman22
Nor do most people who do live in the US, I would imagine... I infer that it's
a high-end steak joint, but I've never heard of it.

~~~
csixty4
I haven't heard that name since leaving the Midwest.

~~~
worklogin
They have locations in DC, San Francisco and South Texas, so they're around.

~~~
amyjess
North Texas, too.

------
lallysingh
This is a pet peeve of mine. Hackers, when measuring time for software
performance, please use something smaller than milliseconds. 0ms is a Dirty
Lie!

~~~
mrob
In this case the performance measure in question is whether the latency is
perceptible to humans or not. Milliseconds is the appropriate unit. Nobody is
going to notice differences of less than 1ms.

~~~
madez
Accumulation.

If x is the unit of measure for the end-result and we have n components that
add together, then we need at least x/n as unit of measure for the performance
of each component. x/(2*n) is more reasonable to not deviate from the target
performance more than one x after rounding in the worst-case.

~~~
AndrewDucker
Yes. But if some of your stages have delays of tens of milliseconds then
there's no point knowing how big the delays are in the stages that are lower
than 1 millisecond.

Once they're all below 1ms it's worth increasing the resolution.

------
elcct
This is actually a known bug hanging there since 2009:

[https://code.google.com/p/android/issues/detail?id=3434](https://code.google.com/p/android/issues/detail?id=3434)

And very good video from Google about it:

[http://www.youtube.com/watch?v=d3kfEeMZ65c](http://www.youtube.com/watch?v=d3kfEeMZ65c)

------
Pxtl
I think the article may be ascribing too much technical reasoning to why iOS
has a better community of music apps - remember that Apple is also has Garage
Band and dominates the online music sales business. It's fair to say that
music is a core part of Apple's brand, and so the platform has a much greater
draw for people who prioritize music software.

~~~
s73v3r
It's possible for Apple to have stuff like Garage Band because they have the
ability to do low latency audio.

~~~
Pxtl
Garage Band (and Apple's position as the kings of music) predates iOS and
Android. That's my point. Even if Android had good low-latency audio, I doubt
that Android would still see the kind of audio-community we see with iOS
because Apple is _the_ brand for that sort of thing and has been that since
the fruity-iMac-days.

------
mianos
Does anyone actually believe the reason " the Google Play store, the Music
category is not even a top five revenue producing app category." is due to
audio latency? I am not disagreeing with the fact that there is a lot of audio
latency (no worse than Windows 8) but maybe it's time to explain cause vs
correlation? Interesting article all the same.

~~~
serve_yay
And so your preferred hypothesis is, what, everyone just forgot to write nice
professional audio and music tools for Android?

~~~
szantog
They did not forgot to write nice professional audio and music tools for
Android. They can't, because of audio latency and dropouts.

------
vlaskovits
If our site goes down (again), the webcache can be found here:

[http://webcache.googleusercontent.com/search?q=cache:zvFOaWH...](http://webcache.googleusercontent.com/search?q=cache:zvFOaWHye-
IJ:superpowered.com/androidaudiopathlatency/+&cd=1&hl=en&ct=clnk&gl=us&client=safari)

------
72deluxe
One piece of software that does do very well on Android is Caustic3.
Admittedly, I am likely creating loops instead of live playing but the
experience is a good one.

Even better is that you can run the OSX/Windows version for £0.00 and copy
your projects over to work on a "real" machine after tinkering on your phone
throughout the day.

------
LoneWolf
Developing with the NDK will not solve this problem? Does it still have to go
through alsa and audioflinger?

~~~
madez
Even if you write an application in C for a GNU/Linux distribution you have to
use whatever sound Linux-subsystem is deployed, unless you have root
permissions and want to mess concurrently with the audio stack (you don't).

The problem is not the fact that they use ALSA and AudioFlinger. ALSA and
AudioFlinger just use at this moment too much time. This could be improved by
decreasing the period size.

------
yeureka
I am writing a game that implements an audio sequencer and it runs great on
iOS and OSX. But I had to write a karaoke app for Android recently and found
an amazing 100ms latency there which makes me think I won't be able to port my
game to Android... Wish Google fixed this.

------
towelguy
It's interesting that their business bases on Google not fixing this problem.

What happens when they fix it?

~~~
macintux
People have been assuming Apple's success depends on one or two minor features
that someone else will copy and destroy their business for, well, ever.

------
lostgame
The audio thing is the number one reason I will never use the Android
platform.

At this point, iOS devs have so many numbers of years ahead of Android devs in
the music department, and the simplicity of porting existing Mac OS compatible
audio stuff makes it incomparable.

Metaphor:

Two people are going for a race, over the same distance. One starts 5 _years_
ahead of the other.

At that point, is anyone even watching the race anymore?

Sorry, despite it's absolutely God-awful flaws such as no user-accessible
filesystem, no ability to do a basic task such as download an MP3, without
GarageBand (and it's awesome ability to open files I sketch out on the go
right in Logic on my Mac...), without iElectribe, and the list goes on, I
absolutely just can't even use the platform, and I have no reason to go back 5
years technologically.

Sorry, Android, you already lost, here is a user you can never have.

~~~
JoeAltmaier
Not really good metaphors. Its not necessary to cover the intervening ground,
to catch up technologically. Fix latency, app developers port and voila -
Android looks pretty good again.

~~~
lostgame
Did you not get the '5 year head start' bit?

Doesn't matter how fast developers create applications that are amazing, iOS
has already had all these years to create an already fantastic subset of these
applications.

Doesn't matter how 'good' it looks - for the use case of music, android has
simply lost.

~~~
JoeAltmaier
That was actually my point. There's no need for Android development to cover
that ground - it can just start where we are now, with cool apps that can just
be ported over. Once the Android audio latency issue is addressed.

App developers LOVE to port, its another sale with small effort.

------
supertruth
I feel like I learned something about the problem. I was expecting to also
learn about the solution but instead got hit with sales pitch at the end
there.

How does Superpowered get around ALSA and the Audio Flinger?

------
acgourley
While the specific problems differ, in general iOS is far easier to work with
when it comes to video as well as audio. Android video is a never ending
headache for us.

------
nsajko
What downsides would there be to just halving the period size?

~~~
CamperBob2
The heart of the problem is simply the limited timing and computational
resources in a multitasking non-realtime OS.

The smaller the audio buffers are, the more prone they'll be to starvation.
Only if the application can be guaranteed to receive interrupt service and/or
thread timeslices at a 100 Hz rate or better is it possible to achieve audio
latency of 10 milliseconds. That's a difficult thing to guarantee in a modern
consumer OS. It can be done, but it won't happen by accident, only by design.

The penalty for failing to service your 10-ms buffer is a dropout that sounds
much worse than slightly higher latency, so there's an incentive to use larger
buffers than necessary at every link in the signal chain. From the point of
view of the OS vendor, musicians might complain about latency, but everyone
will complain about dropouts.

------
mrwilliamchang
Audio lag is definitely not the worst of Android's lagginess problems.

------
amelius
I hope they can fix this for Linux in general, and not just for Android.

~~~
tveita
Properly configured JACK should already achieve single-digit millisecond
latency on Linux.

------
Finster
Does this cause audio de-sync issues on Android?

------
MrBra
Happy to see this side is getting attention!

------
MrBra
it looks it's currently down?

------
bobby5467
titties

------
kabdib
I'd heard over the years that working with isochronous systems was difficult.
I'd done a number of real-time systems before, and written OS schedulers and
NTP-like systems and so forth. A little audio work should be a walk in the
park, right? A little manly-man programming from the wrist and we move on to
_real_ problems. So I walked into an audio project thinking that "Oh, this
latency and synchronization stuff, how bad could it really be?"

Bad.

Nailing down buffer-bloat and sources of latency and jitter in an isochronous
system took several months, the last few weeks of which were 80-100 hour weeks
just before ship. Several times we thought we'd fixed the issues, only to find
that our tests had been inadequate, or that some new component broke what we
had built.

I remember nearly being in tears when I finally realized what the clock root
of the audio system actually was, and that it wasn't what people had been
using. From there everything fell into place.

Don't pull the number of buffers you have out of thin air ("Oh, six is enough,
maybe twelve, don't want to run out in _my_ layer of the system, after
all...") Don't assume your local underflow or overflow recovery strategy
actually works in the whole system. Don't assume your underlying "real time"
hypervisor won't screw you by halting the whole god damned system for 20ms
while it mucks around with TLBs and physical pages and then resumes you
saying, "Have fun putting all the pieces of your pipeline back together,
toodle-oo!" Put debug taps and light-weight logging everywhere and attach them
to tests. And know what your clock root is, or you will be sunk without a
trace.

Isoch is hard.

~~~
gone35
"A little manly-man programming from the wrist and we move on to real
problems."

See... it's the little things like this. I am pretty sure that was not your
intention, but please do know that turns of phrases like that hurt a little,
and exclude a little.

To see what I mean, s/man/jew/ or s/man/white/ or some other category and see
how it reads.

~~~
kabdib
Well, if I wanted to offend, I would have. It is frankly no challenge to
deliberately say something un-PC and widely offensive. It is also apparently
little more challenging to utter something more subtle and more narrowly
offensive.

I look at "manly man" depictions of, well, manly men (for instance, watch
Kevin Kline's performance in _A Fish Called Wanda_ ) as parody. Monty Python
did it. Mark Twain probably did it. Do they offend people? Sure. Do people not
get the joke? Oh yeah. Did that stop the artists in question? Not really. Am I
comparing myself to great artists? I'm not worthy, but I study at the feet of
masters.

So, in my writing (and I've done a bit of it; look at my profile and my blog)
I tend not to give a flying unmentionable about who I piss off. Now, HN is
different because it's not my ball game here, but hey, if everyone wrote so as
not to offend then the world would be a dull place. I've opposed what I
believe is the actual evil -- Political Correctness -- for decades, and I'm
not going to stop now.

I like to take language out and give it a violent shake. Have fun with it, see
what it can do. I'm not going to change the world by substituting something
benign and utterly unoffensive for "manly-man," so fuck it. That's what down-
votes are for. I'm not even sure what I'd put there instead, quite frankly.
Your suggestions don't preserve the meaning at all. Kevin Kline would be sad.

I regret that you were offended.

Now, can we Godwin this stupid thread and get it over with? Someone toss in
the grenade, okay? in 10, 9, 8 . . .

~~~
staunch
BOOM

------
faragon
Android has many problems: \- Poor event handling. \- Braindead UI. \-
Braindead VM approach: WTF JIT on _mobile_ devices??!!!

Google: fuck fix the UI, fuck fix the VM bullshit, fuck fix bloating
everywhere. NDK is not the solution: the problem is in the architecture.

