
Linus Torvalds: Linux Scheduler Not to Blame for Google Stadia Port Issues - jrepinc
https://www.tomshardware.com/news/linus-torvalds-linux-scheduler-not-to-blame-for-google-stadia-port-issues
======
atq2119
Originally discussed here:
[https://news.ycombinator.com/item?id=21959692](https://news.ycombinator.com/item?id=21959692)

------
cryptonector
Linus is right. Don't use spinlocks in user-land. The scheduler can't know
you're spinning on a lock, so it can't help you. Don't do dumb things --
spinning on a lock in user-land is dumb. Mutexes are as fast as spinlocks in
the uncontended case, so use mutexes.

~~~
shadowgovt
Linus is right (for the kinds of code Linus tends to think about). There are
good reasons videogame engines tend to use spinlocks, and if the takeaway is
"Linux pessimizes for that use case," the implication is "Linux pessimizes for
high-performance games atop its kernel."

~~~
the8472
No, linux pessimizes for that case _if you do not cooperate with the
scheduler_.

You can implement spinlocks in userspace under specific circumstances. You
will need to mark your threads as realtime threads[0] and have a fallback to
futex if the fast path (spinning) doesn't work out. And even then you need to
benchmark on multiple machines and with realistic workloads (not
microbenchmarks) to see whether it's actually worth it for your application.
It's complex, linus goes into more detail here[1]

[0] [http://man7.org/linux/man-
pages/man3/pthread_spin_init.3.htm...](http://man7.org/linux/man-
pages/man3/pthread_spin_init.3.html#NOTES) [1]
[https://www.realworldtech.com/forum/?threadid=189711&curpost...](https://www.realworldtech.com/forum/?threadid=189711&curpostid=189755)

~~~
quotemstr
If you have a fallback to futex, then what you have isn't a spinlock, but an
adaptive mutex (which spins, then blocks). Lots of operating systems have
those already and you don't need to build your own.

~~~
the8472
If you really want to use a pure spinlock you'll either have to accept that
worst-case behavior being worse than a mutex or you have to jump through a lot
of hoops (pinning threads, isolating cores, reassigning interrupts) to make
sure you don't get descheduled. Because the scheduler won't know anything
about priority inversions or even waiting threads, they all look runnable.

------
sschueller
"do not use xxx, unless you actually know what you're doing. And be aware that
the likelihood that you know what you are doing is basically nil."

I got to use that quote at some point. Pure gold.

~~~
rooam-dev
Imho the 2nd part applies to those that use the quote too.

~~~
metalliqaz
if you read Linus' post, he says as much immediately after. He says he
includes himself in that "you", and notes that the kernel is a result of
decades of development by many people working to fix issues that pop up
implementing the hard stuff.

------
falcolas
As someone who has played Doom 2016 (and a bevy of other games from Steam) on
Ubuntu at 4k 60fps, I strongly doubt that it's Linux.

Wine/proton and the accompanying libraries are pretty f'ing amazing.

~~~
gnulinux
They're very amazing, it feels truly magical, but in my experience, they're
very fragile. Back in ~2014/2015 I was able to play Skyrim on my trash cheap
laptop in Archlinux (don't remember the settings and fps, but the game ran
"flawlessly" for me, yet I need to admit I'm not a gamer so I can't tell the
difference between 30fps 60 fps etc).

I tried the same in 2018 just for fun and I wasn't able to run Skyrim, there
was always some minor issues that made the game unplayable, like mouse not
working, crashing on certain screens etc. I essentially kept trying older
versions of Wine, and eventually found some version that made Skyrim playable
(still not as flawless as it felt in 2015, but playable). Idk if it was my
mistake, or some archlinux lib weirdness or something else, but I had to try
dozens of wine versions to find something that worked. This is fine for me,
since I wasn't intending to play Skyrim, I was just trying to run it (for the
"WOW" effect), but I imagine people who actually need Windows software might
have been frustrated by this.

~~~
falcolas
FWIW, I've found that using the Steam launcher eliminated any compatibility
issues which may have cropped up. I'm not a linux _desktop_ power user, and
didn't need to be.

> I'm not a gamer so I can't tell the difference between 30fps 60 fps etc

It's like driving a car with a rough idle vs. a smooth idle. You can get used
to driving a car with a rough idle, and not particularly care. But once you've
driven both, you can identify which is which pretty easily, and the rough idle
starts to bug you when you know it can be smoother.

~~~
Wowfunhappy
I might make a different comparison:

 _Everyone_ who has played more than one video game knows that some just
_feel_ better than others. In one, your character is fluid and nimble, and in
another, slow and clunky.

This is the result of many things, but a large portion comes from input lag
and frame rate. More advanced users aren't necessarily more perceptive,
they're just better at pinpointing what they're reacting to. (And in some
cases, conscious of how an experience could be improved.)

This is similar to the study of any type of art. Whereas a casual observer
might look at a painting and say "meh, I don't like it", someone who has
studied art pedagogically can describe precisely what isn't working for them.

------
littlestymaar
The original comment from Linus is much more insightful than the “its's
complete garbage” quote that was picked for the article:
[https://www.realworldtech.com/forum/?threadid=189711&curpost...](https://www.realworldtech.com/forum/?threadid=189711&curpostid=189723)

------
brenden2
Stadia is just a bad idea and a bad product. Consumer internet access is
nowhere near capable enough for it to make sense.

~~~
justwalt
But the issue isn’t bandwidth, really, is it? I would think the ping would be
the killing factor. Do they have a plan for that? Server farms no more than
20ms away from anywhere in the US?

~~~
cptskippy
>> Consumer internet access is nowhere near capable enough for it to make
sense.

> But the issue isn’t bandwidth, really, is it?

No, bandwidth isn't the issue, I believe the op was referring to Latency.

I have 75mbps DSL and the Latency is garbage with 29ms being best case. I get
frequent periods of 90+ms latency.

~~~
tudelo
These seem like pretty common latency numbers for online gaming. I guess it
feels worse when there isn't the movement prediction and other techniques to
make the latency feel better since the game isn't running locally.

~~~
cooljacob204
Well the issue is that you have input lag on your controller inputs. When I
move my crosshair it takes a long to actually show the crosshair moving on
screen.

In normal online gaming you do not have that issue. When I move my crosshair
in cs:go it moves instantly.

~~~
tudelo
Right, yeah that's kind of what I was trying to suggest. Honestly, I don't
really see how this can work in the general case all that well given exactly
what you are saying. I have seen some examples of fighting games being played
over stadia, but many fighting games rely on frame-perfect input (and purists
in communities like melee dont even use certain monitors due to this...)

------
CoolGuySteve
I think the more interesting question here is why don't Stadia games use
isocpu and sched_setaffinity to ensure their games have the best CPU cache
performance and lowest latencies possible.

They control the whole box, it should be more like a console than a regular
PC.

~~~
oliwarner
I'd guess they stack more than one game instance on a single server. The
instances need to play fair for resources.

~~~
angry_octet
There are fairly standard real time scheduling techniques to give whatever
time slice you want to each process. Usually you architect it to have a small
hard real time and another process for soft real time (e.g. paging data) and
you use a priority queue for world updates (intersection tests, gun/missile
flyout, updating other player positions). The update thread gets a minimum
time quantum and then yields, round robin sharing.

------
haydn3
What Linus is really saying is that the way Skarupke measured the spinlock
being 'bad' is wrong. Skarupke tried to measure it in C, but you can never be
sure if you're not being scheduled by anything else.

Skarupke inaccurately defined this as 'mutex vs. spinlock' and an ongoing
debate surrounding this as they are comparable, when in no way are they
comparable as in userland you simply _cannot_ use spinlocks.

The Windows scheduler apparently _doesn 't_ do this, and _handles_ spinlocks
better; whatever that means. But the way it works is that spinlocks should
work poorly if you run them poorly. Full manual-control of when running a
spinlock in userland, you _will_ be scheduled by other things, and you should
know that.

That's what Linux is really about. The ability to do something and not have
the system "better-ize" it like Windows does by doing some black magic and
running an _unknown_ process alongside your spinlock to make it run better.

A spinlock run in userland SHOULD be scheduled by other things, as Linus says.
As Linus implemented in the kernel.

------
0xff00ffee
My knowledge of linux scheduling has gone up 10x thanks to this debate!

------
trishankdatadog
BTW, Stadia is incredible, but the lack of high-quality big-name games is
disappointing...

------
deith
Are userland spinlocks the reason most games use 100% of one or several cores
even when they are minimised and doing nothing?

~~~
magicalhippo
Kinda. Normal Windows applications use a system call to wait for new messages
to arrive, and then react to those. This is very similar to waiting for a
mutex, and it allows the kernel to put the thread to sleep until a message
comes along.

Games often integrates the message loop into their game loop. In their loop,
they poll to see if any new message has arrived, and if so handles it. In any
case it does it's game thing.

If there's not much to do, because the game is paused when it's minimized for
example, this spins much like a spinlock spins. If, in addition, it has some
other threads it's synchronizing using spinlocks then yes, it can cause it to
burn several cores worth doing nothing.

Better games handles the loop differently while the game is minimized, for
example doing the default wait-for-message variant.

