
Measuring keyboard-to-photon latency with a light sensor - chmaynard
https://thume.ca/2020/05/20/making-a-latency-tester/
======
hasperdi
The terminal latency test more or less confirms previous tests done
([https://danluu.com/term-latency/](https://danluu.com/term-latency/)).

Alacritty is still claiming that it is the fastest terminal around, even
though that claim has been proven false several times.

~~~
trishume
Alacritty makes claims about throughput, which is different than latency, and
then uses the word "fast" which can be confused to mean either or both.
Alacritty's benchmarks are based on how much text it can process being output
to the terminal per second, which it is much better at than most terminals.
However separately I think it might be true that kitty also has better
throughput, while having better latency and many more features, this is just
based on talking to a friend though I haven't tested this.

~~~
chmln
Sadly, alacritty feels like such a waste of effort. The input lag is pretty
bad, and text processing performance is a non-issue with most terminals. I've
used xterm for years and never felt it was too slow at that.

Latency makes much more difference in how fast a terminal feels. This is where
kitty does a fantastic job, even letting me trade some CPU cycles for really
low latency with its options. I can strongly recommend it to anyone who values
low latency.

~~~
ubercow13
alacritty still has the best font handling/rendering from my comparisons,
though kitty's latency is very noticeably better

~~~
aslpavel
alacritty is actually pretty bad at font rendering quality, at least on macos,
it looks worst in comparison to all terminal i've tried
[https://github.com/alacritty/alacritty/issues/486](https://github.com/alacritty/alacritty/issues/486)

~~~
ubercow13
Oh interesting. On Linux it's clearly better than kitty:
[https://i.imgur.com/fzAPhlN.png](https://i.imgur.com/fzAPhlN.png)

Last time I checked it also had the best unicode handling too, in terms of
reliably falling back to some other font for missing codepoints and getting
the display of oversized characters right.

~~~
chaorace
It's not immediately obvious which terminal is which. I __feel __like the text
is sharper and, as a result, more readable in the right window, not sure if
that agrees or disagrees with your thoughts

~~~
ubercow13
The right one is kitty. Look how the italic characters have uneven spacing,
eg. the 'l' in 'bold', and some of them are chopped off, for example 'M'. Look
how the percent sign on the bottom looks like it's been stretched and the
circles aren't the same.

The right hand one is sharper but I have my freetype tuned to look like the
left one, which looks the same as every other application on my PC. kitty is
doing something completely different and incorrect.

------
KarlKemp
I'm not really sensitive about this issue, but recently had an experience that
has raised my interest: after making some regretful mistake, I had to use an
an UEFI shell to shuffle some boot settings around (the one that comes with
the Clover bootload).

And typing was..noticeably more "immediate". It was bewildering, sort of like
wearing prescription glasses for the first time.

I like all the extra stuff editors and the OS do these days and don't wish to
turn the time back, and I doubt it's realistic to achieve both. But maybe
someone like Apple, being fully integrated, should take a shot at it. It'd be
the sort of polish that makes using a computer enjoyable.

~~~
trishume
In this case what you were experiencing might have been the lack of
compositor, which adds a 16ms delay, or just using a slow editor. The
theoretically best possible average latency under a compositor model at the
top of the screen is 16+(16/2)=24ms which isn't that far from the 30ms I
observe at best on macOS. Cutting the compositor would still halve average
latency.

Unfortunately Apple shows no interest in getting rid of that delay, while
Microsoft is moving towards sometimes using hardware overlays to avoid that
(see raphlinus's comment about DirectFlip), and Linux lets you disable it at
the cost of tons of artifacts. Apple only recently tried to add compositor
bypass for fullscreen Metal apps but the way they did it is super opaque and I
have never succeeded at getting it to trigger and they have no example program
that they say definitely does it.

~~~
TheTon
A single fullscreen window where the content view is either a Metal or GL
layer, and nothing else in the layer hierarchy should be sufficient.

Using Instruments’s Metal and Display instruments you can see the drawables
and the vsync they present on. Bring up the app switcher with cmd-tab to
trigger the compositor and you can see the difference in Instruments.

One weird thing is, you can configure a 2 or 3 drawables swapchain on the
Metal layer. Using 2 doesn’t work for me when the compositor bypass enables -
it always locks to 30Hz, even when doing nothing but drawing a solid color.
When the compositor is enabled, the 2 buffer swapchain runs at full speed,
presumably because it’s effectively triple buffered. But, in the 3 buffer
case, in fullscreen, you can see that you will just use 2 buffers provided you
can render fast enough, so it seems like there’s no latency cost to the 3
drawable config in the ideal case.

~~~
trishume
Yah it was over a year ago so my memory is hazy but as far as I know I tried
to do exactly that. I watched the WWDC talk and made sure I met all the
conditions they say, and tried to use Instruments and Is It Snappy to verify
it and I didn't seem to be triggering compositor bypass and had no way of
figuring out why not. What I really want is a single open source demo app on
Github where someone has said "I tested and this triggers Metal compositor
bypass and I confirmed with Instruments and/or Is It Snappy"

------
pbhjpbhj
Presumably pro esports teams are measuring latency of their rigs to get the
fastest gear where ms count?

If typical gamers reaction time is O(300ms) and a keyboard, change can save
40ms, or a screen can add 80ms (extreme cases) seems like they'd be wise to be
testing and verifying things.

Anyone links to details of how esports teams are doing things here?

I'm curious about the light-detection - do screens have a very tight
synchronisation between different colours. Could the sensor be matched in some
way with the specifics of the screen?

------
lukevp
This is really interesting. I use vs code on Windows on a 144 hz gaming
monitor with an nvidia gpu and gsync, and it “feels” like it has the lowest
input latency of any editor I use (out of notepad, notepad++, and Visual
Studio).

I’m curious what impact hardware acceleration and gsync would have on this. I
would guess Electron is using a hardware accelerated surface in my scenario,
but I don’t imagine gsync is doing much on the desktop since it is
synchronized to the entire OS and not a specific application.

~~~
ubercow13
Comparing 'Chrome html input' to VS Code in the article, it seems like Chrome
is capable of rendering typed text with low latency. This seems to imply that
the high input latency in VS Code can't be due to low level GPU and rendering
pipeline issues brought about by electron, and must instead be due to
processing VS Code is itself doing every time you type (syntax highlighting,
semantic things, layout?). If this is true it seems unlikely monitor refresh
rate or gsync could help.

~~~
londons_explore
Chrome has effectively a 1 frame latency. First the event fires for a
keypress, and later the compositor decides it's time for a new frame, which
will include the effects of that keypress.

------
mfalce
Hey, it reminds me of what we did at my former research lab. It was a Computer
Human Research team, and we were focusing on ways to measure the interaction
lag when using a mouse, a keyboard or a touch device:
[http://mjolnir.lille.inria.fr/turbotouch/lagmeter/](http://mjolnir.lille.inria.fr/turbotouch/lagmeter/)

I participated in writing 2 papers. In the first one, we used an optical mouse
on a screen displaying a special texture to measure the end-to-end latency. In
the second one, we used an external Arduino (quite similar to what the author
did here) connected to a photodiode.

For what I remember, the main factor on the latency is the screen, use a gamer
screen to improve the delay (CRT screens are told to be quite fast also, but
we hadn't access to one to test :'( ).

------
mncharity
> You can see that some combination of the light sensor and the monitor take
> nearly 100ms to fully transition.

That's the sensor. It's not a wonderful match for this job.

The linked[1] light sensor module looks like a CdS photoresistor. In its
Amazon Q&A, the seller claims "The response time: Rise 30ms Decline 30ms",
under unspecified conditions.

Better to use a photo _transistor_ like [2], with a rise time of _micro_
seconds. 4 orders of magnitude faster. It's easy - one resistor, and an analog
input on the Teensy.

Deja vu. For 3D UI prototyping on my laptop, I diy'ed LCD shutter glasses
driven by an arduino with a photosensor reading a white/black square rendered
flashing in a screen corner. IIRC, I actually tried a photoresistor first, but
the plotted screen response curve times weren't plausible. Only then did I
look for a datasheet, and had an "okaaay, _that 's_ not the right thing"
moment. :/

[1]
[https://www.amazon.com/gp/product/B01N1FKS4L/](https://www.amazon.com/gp/product/B01N1FKS4L/)
[2]
[https://www.adafruit.com/product/2831](https://www.adafruit.com/product/2831)

------
gorgoiler
Oh dear.

100-133ms for my keyboard to display USB hub, to laptop, and back to display.

66-83ms for my laptops own keyboard and display.

Something about my computer has always felt a bit off. For a long time I
thought it was a commonly reported vim + tmux bug but it never felt like it
sped up very much.

Now I just want a 386 in VGA mode again. I’m sure things were better in the
_good old days_ (said with some sense of irony.)

(Measured with 5 samples on each.)

------
swingbyte
HI, We applied for a patent back in 1996 for a system that measured simulator
to human latency through various means. The basic concept is to instrument the
simulator image generator to activate a trigger i.e. a section of the display
that changes in response to human input. This was monitored by a light sensor
on the screen and a circuit that monitored that area of the video signal going
into the display device. A timer circuit started counting when the button was
pushed and stopped when the change to the display was detected. We also
automated out the human to collect mass data on the simulator latency. This
worked very well and gave us both the delay to video and to display times from
a defined input through the complete simulator pipeline.

------
brokenmachine
An LDR like he is using has a typical response time of up to 18ms. Better to
use a phototransistor which will respond in the order of μs. [1] is $0.95.

[http://lednique.com/opto-isolators-2/light-dependent-
resisto...](http://lednique.com/opto-isolators-2/light-dependent-resistor-
ldr/)
[https://www.adafruit.com/product/2831](https://www.adafruit.com/product/2831)

------
snarfy
If you want low latency you really need to use USB3/USB-c. USB3 doesn't use
polling like USB2. USB2 latency is measured in milliseconds. USB3 latency is
measured in microseconds.

~~~
trishume
If you use 1000hz polling like my latency tester and keyboard do the latency
becomes less than 1ms which is pretty small compared to the other factors. See
[https://michael.stapelberg.ch/posts/2018-04-17-kinx-
latency-...](https://michael.stapelberg.ch/posts/2018-04-17-kinx-latency-
measurement/)

I'd optimize lots of other things like trying to avoid the compositor, higher
refresh rate monitors and lower travel keys before I'd look at USB3.

------
zokier
Huh, the results seem surprisingly uniform. The difference between different
compared editors/terminals is basically one frame, and the standard deviations
don't seem remarkable either. Maybe the biggest observation here is that the
buffer state can significantly impact the latency in some cases (the added
10ms on vscode "aa" test), but even that was a sidenote here.

It would definitely be interesting to have more software and hardware tested,
I would imagine there is lot more variance out there.

------
CyberDildonics
I think it would be worthwhile to test gvim (which is known to have low
latency), windows running at a higher refresh rate and a bios shell. I think
the USB keyboard input standard itself is not great for latency unfortunately,
but it would be good to know where the lower bound is.

~~~
trishume
I unfortunately don't have a native Linux install currently, just a VM, and my
Windows NUC can only connect to my Z27.

There's some good info here on minimal USB latency, it's less than 1ms and I
don't think mine is an issue although the macOS event processing stack might
introduce some latency: [https://michael.stapelberg.ch/posts/2018-04-17-kinx-
latency-...](https://michael.stapelberg.ch/posts/2018-04-17-kinx-latency-
measurement/)

------
conistonwater
What's going on with those two editors that the input latency is two frames on
average? I would have expected it to be somewhere around 1 frame just because
the editors aren't doing any extra work like auto-completion. Two whole frames
sounds pretty bad.

~~~
raphlinus
A large part of that is the extra frame of latency added by the compositor.
One experiment that I'd very much like to do is try to get Windows to promote
a window to using hardware overlays (this is sometimes called "DirectFlip" but
the public documentation on it is very sparse) and see how much that helps.

------
jart
h/t Tristan Hume
[https://gist.github.com/jart/49cfc20d21f619c2547016be3038e89...](https://gist.github.com/jart/49cfc20d21f619c2547016be3038e89d)

------
solarkraft
Are there common approaches for measuring touchpad lag?

~~~
trishume
I wouldn't say anybody measuring input latency at all is common, but if I were
doing it I'd use [https://isitsnappy.com/](https://isitsnappy.com/) or some
other high speed camera frame counting and check the time between when your
finger starts moving and the screen changes.

------
ilrwbwrkhv
sublime text again proving to be the fastest

