
No nuances, just buggy code (was: related to Spinlock implementation) - shaklee3
https://www.realworldtech.com/forum/?threadid=189711&curpostid=189723
======
kstenerud
It's important to read Malte's response to Linus:

[https://www.realworldtech.com/forum/?threadid=189711&curpost...](https://www.realworldtech.com/forum/?threadid=189711&curpostid=189747)

Followed by Linus response to Malte:

[https://www.realworldtech.com/forum/?threadid=189711&curpost...](https://www.realworldtech.com/forum/?threadid=189711&curpostid=189752)

The long and the short of it:

\- It's nearly impossible to make meaningful benchmarks for this because
reality will almost always be different and more complicated

\- Calling sched_yield() is almost always a mistake

\- Any scheduling optimization is probably a waste of time because either your
running env will change, or the kernel will change, or someone else's
optimizations will clash with yours.

\- Even with decades of research into locking, everyone (including the
experts) continually gets it wrong.

\- If you're not running a specialized embedded setup where you control all of
the running processes and other such variables, just use sleep locks and call
it a day.

~~~
sillysaurusx
Why is calling sched_yield() a mistake? What's the correct way to yield CPU
without calling sleep?

~~~
colonwqbang
It's explained in the linked post.

Why don't you want to call sleep?

~~~
gpderetta
Beacuse it would act as as an even worse sched yield.

~~~
mritun
Calling sleep(1) instead of thread yield is faster - see the twitter thread
for a proof.

~~~
de_watcher
Still worse: it's a confusing code, and it's also confusing that it's faster.

------
asdfasgasdgasdg
Ok, so how you would write this (at least the first section, explaining the
error) constructively could be much shorter.

"The author of that post has a common misconception, which is that spin locks
are a fast drop in replacement for uncontended mutexes. They aren't. You can
only use them well when you know you won't be descheduled, like we do in the
kernel. In a generic userspace program, you don't have that knowledge. The
possibility of your time slice ending skews all the time measurements, for
example."

Boom. Roughly the same message. Most people will get the same amount of
edification from this as Linus' original letter. You could go in to more
detail if you want but it's not really necessary. Smart people will figure it
out if you just point out the general shape of the error. No need for calling
code "garbage."

~~~
buzzkillington
>Boom. Roughly the same message. Most people will get the same amount of
edification from this as Linus' original letter. You could go in to more
detail if you want but it's not really necessary. Smart people will figure it
out if you just point out the general shape of the error. No need for calling
code "garbage."

People should be shamed for being ignorant. It stops ignorance from spreading.

~~~
dochtman
I guess we should shame you for being ignorant about human beings and the
value of being kind. There are plenty of ways to point out ignorance without
being unkind (and the grandfather comment is a nice attempt).

~~~
buzzkillington
You just did, and got to pretend you didn't.

------
OskarS
This all sounds very good, and I grant that Linus knows what he's talking
about, but didn't this whole thing start with a bunch of game developers that
noticed massive stalls when porting games to Google Stadia? Stalls that,
presumably, didn't happen on other platforms (Windows and PlayStation, using a
variant of FreeBSD).

If these kinds of spinlocks are so fundamentally misguided, why do they only
cause problems on Linux?

~~~
baybal2
Ironically things went upside down with linux scheduling over the last 15
years.

All major OSes had PhD CS people doing their schedulers "efficient" at around
2000, and be able to sustain 98%+ MP loads. Linux was an odd case of a very
naive, and "inefficient" scheduler that left CPU 30-20% idle unless you really
knew how to hack around that. Linux however came with near no stalls, and
latency spikes.

Things have changed dramatically now. Linux got attention of the entire
industry on it, and years of holy wars about scheduling.

You can now hit 95%+ under multithreaded load easily, but even on the desktop
you can now see how a block in IO hog app can visibly freeze your system for
few seconds.

Now, the trend has reversed again, and now everybody is praising conservative
scheduling that doesn't give out too much CPU :-/

~~~
HankB99
> but even on the desktop you can now see how a block in IO hog app can
> visibly freeze your system for few seconds.

Or worse. I ran into an issue when copying an image from an NVME drive to a
USB drive using `cat` (e.g. `cat somefile.img >/dev/sdX`) where mouse
performance (also a USB device) became so sluggish that the desktop was
unusable until the copy was complete.

This wasn't the only instance where background I/O brought my desktop to it's
knees. In another instance it was copying a large file across the network
(Gigabit Ethernet) to a file server. The network was saturated and the desktop
was unusable. I think that the remote system was the destination but my
recollection could be in error. The network interface is a PCI device and
there was no USB device involved in this transfer.

This is on Debian Stable and using the Gnome desktop. The non-standard portion
of the system is that it runs off a ZFS filesystem. I don;t know if that
matters.

I'd like to know where to look and how to diagnose this issue if/when it crops
up again. Interrupt storm? USB driver issue? Some other issue resulting in
some resource starvation? (In neither case did overall CPU load appear high.)

~~~
labawi
I'd guess the apps were waiting for I/O, not CPU.

Don't know the reason, but lots of unwritten I/O on a (slow) drive can block
processes waiting for I/O on an unrelated drive.

By default, dirty buffers are a predetermined portion of RAM, so lots of RAM
and slow I/O can make for long pauses. I usually tweak sysctls on desktops to
limit dirty buffers. E.g.

    
    
      vm.dirty_background_bytes = 67108864
      vm.dirty_bytes = 134217728

~~~
baybal2
You will not get enough CPU time given to a task if it does, say, 10% IO, and
90% CPU if there is an IO hog running in parallel. You either IOnice
everything to oblivion, or bare with this.

IO blocks are almost always CPU blocks too on anything that does even minimal
IO. And the more simultaneous IO, the more CPU overhead there is.

Try running compiling something big with -J 32 or more.

~~~
labawi
When system is IO busy, processes block (more) waiting for IO completion. IO
blocked processes wouldn't know what to do with a CPU, so they won't have it
assigned. Are you saying something different?

Decreasing dirty memory decreases flush times, decreasing process write/flush
times, decreasing process/UI hangs/pauses. Presumably it also lowers
throughput in some cases, but it's better than having UI hangs.

------
mntmoss
Reinterpreted in my own words:

Usercode spinlocking is not legible to the kernel. Therefore anytime you
attempt it you are in the situation of fighting the scheduler, and given that
even the kernel devs are constantly challenged by the problem of making
appropriate scheduling tradeoffs, you will lose.

------
ww520
Damn, Linus really knows his stuff, getting to the heart of the matter right
the way - measuring user mode locking time is futile, as the OS scheduler can
preemptively put the locking thread to sleep for arbitrary time. The OP ended
up measuring how quickly the scheduler woke up the locking thread, thus
measuring the wrong thing.

~~~
Flicki
You think the OP doesn't know this? Common. Linus was making straw men
arguments as he seems to commonly do.

The measurements still show Windows trouncing Linux in reponsiveness

~~~
sudosysgen
Yes, because the benchmark was using spinlocks incorrectly.

There are many places where the Linux scheduler is more responsive, even in
gaming, exactly because of what makes spinlocks inefficient. The Ryzen
scheduler debacle comes to mind.

------
dang
Related and recent:

[https://news.ycombinator.com/item?id=21919988](https://news.ycombinator.com/item?id=21919988)

[https://news.ycombinator.com/item?id=21955234](https://news.ycombinator.com/item?id=21955234)

~~~
cbarrick
That first post is what Linus is responding to.

The second post was really educational for me.

------
rixed
I am under the impression that this long, vocal post, consisting of many
assertions and no measurements, would have been judged less favorably if it
were not from Linus.

I used to agree with many of Linus rants, but not this time though.

The gist of the reply, as I understand it, is that the benchmark is measuring
random spikes while the spinning thread is scheduled away. But the benchmark
clearly stated that they are using one thread per core and that the host was
otherwise idle.

It could be that, still, some random task pops up once in a while, replacing
the very thread that has the lock. But then, rather than calling the whole
benchmark garbage I think it'd be more fair to just ask for it being rerun
with, say, two free cores. Or use auditd to detect and invalidate those runs.

Linus also calling everything random sounds annoyingly slopy: varrying the
scheduler or the OS led to significantly different measurements. Maybe the
explanation for those differences are wrong but the measurements are certainly
not random.

~~~
baq
The final conclusion after throwing the blog author into the dumpster (which
I’d be more polite about, but that’s Linus for you) is that unless you spent
your whole career researching and implementing locks, you should use something
that was created by somebody who did. Which is some damn good advice if you
ask me (disclaimer: I didn’t research or implement any production-grade
locking primitives.)

~~~
rixed
I don't know why you felt necessary to rephrase the final conclusion (I've
read it - notice, by the way, that it was also the conclusion of the original
benchmark) nor to introduce Torvalds' style (I mentioned I'm familiar with
it).

Anyway, I do not think that a forum that calls itself "hacker news" should
adhere to that dogma that any topic claimed by an authority to be complex
should never be re-implemented on first principles. And I doubt the young
Linus would have paid much attention to that kind of advice when he started
developing a new operating system.

Personally, the conclusion I take from that thread is simpler:

On a situation when N threads are runnable on N cores with N-1 calling
sched_yield(), one must not expect the Nth thread never to be scheduled out
for a whole slice on Linux (for some or all of the reasons listed by Linus,
none of which seems to address that exact case unfortunately).

~~~
nelgaard
Why would you expect otherwise from sched_yield?

man sched_yield(2) says: == sched_yield() is intended for use with real-time
scheduling policies (i.e., SCHED_FIFO or SCHED_RR). Use of sched_yield() with
nondetermin‐ istic scheduling policies such as SCHED_OTHER is unspecified and
very likely means your application design is broken. ==

------
ww520
A simple reasoning on why spinlock is slower than mutex in a heavily contested
situation.

A thread (S1) attempting to acquire a spinlock will busy looping to hog all of
its time slice, preventing other threads (S2) from utilizing the CPU to free
the lock.

    
    
        S1: [**busy spinning**][..waiting........][**lock & run*****]
        S2: [..waiting........][**run & unlock***][..waiting........]
    

A thread (M1) attempting to acquire a mutex will sleep to give up its time
slice, letting other threads (M2) to utilize the CPU and thus have a chance to
free the lock. The given up time slice can be very substantial, leading to
much better performance.

    
    
        M1: [sleep][..waiting........][**lock & run*****]
        M2: [.....][**run & unlock***][..waiting........]
    
    

If one really really wants to use spinlock, the way to use it correctly is to
boost the priority of the thread holding the lock to make sure it got priority
to run over all else, so that it can finish its work quickly and release the
lock.

    
    
        S1: lower priority  [..waiting........][**lock & run*****] higher priority
        S2: higher priority [**run & unlock***] lower priority [..waiting........]
    

I assume the benchmark and the user-mode spinlock doesn't do that, and thus
has the measured problem. OS/kernel would have a better idea to raise and
lower priority with spinlock than user-mode code.

~~~
imtringued
> If one really really wants to use spinlock

You will also have to make sure that your application threads are pinned to
specific cores and that the number of threads is smaller than the number of
cores because you want to keep one thread free to run non critical code.

Spinlocks are like full plate armor. Very effective but incredibly cumbersome
to use.

------
alexdowad
Just searching the web to see if anyone has published a list of Linus
Torvalds' best technical posts -- those where he shares his extensive
knowledge of system programming, CPU architecture, etc. With all the years he
has been posting on newsgroups, there should be a lot more "gold" out there.

Unfortunately, it seems that most people who write about Linus on the web are
more interested in his worst posts -- where he has slung insults and profanity
with no valuable technical insights.

~~~
wging
[https://yarchive.net/comp/index.html](https://yarchive.net/comp/index.html)

~~~
shaklee3
Thanks! That's a great link for more than just Torvalds.

------
weinzierl
> Pretty much every time we picked an unfair - but fast - locking model in the
> kernel, we ended up regretting it eventually, and had to add fairness.

This is an interesting quote from Linus. I always thought that fairness was
for the desktop and Linux had latency issues sometimes because it historically
put the _" big-iron"_ first. Now this quote makes it seem that fairness is a
good thing for the server as well and the preceding paragraphs hint that
supposed latency issues are just a natural consequence of the trade off
between throughput and latency.

------
dpc_pw
I highly recommend reading the whole thread, especially all Torvald's
responses.

~~~
unixhero
Maybe I'm not adding much as contribution here; but I am of the point of view
that Torvald's responses are always worth reading through.

------
micimize
It's kind of disturbing to me that so many people in the comments are
defending Linus's tone here. He writes like petulant child. Its wildly
unprofessional, and only makes it harder to follow what follow what he's
trying to say. I'm really glad I'm not actively involved in any community he's
a part of.

~~~
jturpin
There's always a surge in people who defend and even mimic Linus's tone
whenever a post like this gets popular. People love being that smart guy who
gets to feels entitled to act obnoxiously because they're right - at least
about one small thing. Reminds me of _that_ group in High School.

It sounds like there was an expectation that Linux should support something
that it doesn't, and whether it _should_ support it is up for debate. Totally
agree about Linus's involvement in any communities.

I assume part of it is that he feels personally attacked when something about
Linux is challenged.

~~~
SXX
> It sounds like there was an expectation that Linux should support something
> that it doesn't, and whether it _should_ support it is up for debate.

You really didn't paid attention to what being discussed. Someone started
discussion on how bad Linux kernel is based on incorrect data generated by
badly written code.

------
sfgweilr4f
summary: Don't roll your own (spin) locking unless you _really_ want to go
down the rabbit hole. Linus thinks locking is a PITA and there be dragons. You
have been warned.

~~~
taneq
Synchronization primitives are another thing along with dates and names and
postcodes and unicode text editing: "Stuff you should avoid writing yourself
if at all possible because they are incredibly fiddly to get absolutely
right."

~~~
skunkworker
Also add crypto algorithms to that. I know of too many people that thought, a
SHA-512 is fine for passwords. /facepalm

~~~
imtringued
I don't understand this complaint. Cryptographic hashing gets obsoleted all
the time. No matter which algorithm you choose you're just buying time until a
better algorithm is needed. One day we will facepalm because people didn't use
quantum resistant algorithms.

~~~
skunkworker
There's a major difference thinking it's okay to store all of the user's
passwords on your site with a SHA-512 of the password vs using
bcrypt/scrypt/pbkdf2 etc with a high work factor. One is just ignorant and the
other is following modern practices and continually updating the work factor
to address advances in computing power.

When was bcrypt or pbkdf2 made obsolete as a algorithm?

------
dhsysusbsjsi
Can anybody explain the advantages or otherwise of Apple’s libdispatch for
locking? I understand there’s Linux support now also. Is this something you
would use outside of Apple ecosystem?

[https://developer.apple.com/documentation/dispatch/dispatch_...](https://developer.apple.com/documentation/dispatch/dispatch_semaphore?language=objc)

~~~
asveikau
Quoting from the link:

> A dispatch semaphore is an efficient implementation of a traditional
> counting semaphore. Dispatch semaphores call down to the kernel only when
> the calling thread needs to be blocked. If the calling semaphore does not
> need to block, no kernel call is made.

This sounds like the same optimization that futex(2) makes for pthread mutex
on Linux. So while a naive semaphore might enter the kernel 100% of the time,
this one does so only when there is actual waiting to do.

It would seem to me that sem_wait() does similar on glibc so Linux probably
performs similarly with the standard POSIX semaphore:

>
> [https://github.com/bminor/glibc/blob/master/nptl/sem_wait.c](https://github.com/bminor/glibc/blob/master/nptl/sem_wait.c)

mirrored from here:

>
> [https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=blob;f=n...](https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=blob;f=nptl/sem_wait.c;h=171716fdbce243d64a9da46acdc4dc77d81921b4;hb=HEAD)

~~~
lilyball
Don't use dispatch semaphores where mutexes (or dispatch queues) would
suffice. Apple platforms have a QOS mechanism that donates priority from
higher-QOS threads to lower ones when they're waiting on locks that the
higher-QOS thread owns, but with semaphores the system doesn't know what
thread "owns" the semaphore and so it can't donate priority. Mutexes (and
dispatch queues), including os_unfair_lock, will donate priority accordingly.

Which is to say, dispatch semaphores shouldn't be used if you're initializing
them with a value of 1, you should only use them if you have an actual pool of
resources you're protecting.

~~~
gpderetta
Semaphores are for signaling (sames a condition variables, events) while
mutexes are for mutual exclusion. Technically you can also use semaphores for
mutual exclusion (a mutex can be thought as a binary semaphore) but you really
shouldn't.

~~~
lilyball
Right, but libdispatch doesn't have a mutex. It has semaphores and queues. So
if you're trying to use libdispatch and you don't want the closure-based
aspect of queues, you might be tempted to use a semaphore instead. Don't do
that, use os_unfair_lock or pthread_mutex (or a higher-level construct like
NSLock) instead.

~~~
asveikau
I was hazy on remembering details for my comment above, but I now realize
fully that I do have experience with semaphores on Darwin. It went like this:

First I tried POSIX semaphores. They seemed poorly done on Darwin. [Maybe they
needed sem_open() rather than sem_init()? But unnamed ones are way more
useful!]

Then I tried Mach semaphores.

Then I saw a recommendation for libdispatch semaphores to avoid too many
syscalls.

Given that Linux does the syscall-avoiding trick by default for standard-
conforming semaphores, it seems to me like on Darwin you can't go too far
wrong always using libdispatch. But I guess you miss that priority inversion
workaround you were talking about.

~~~
lilyball
On Apple platforms the simple set of guidelines would look something like:

* Can you use dispatch_queue? Great, use it.

* If not, can you use pthread_mutex? Note that this can be configured to be unfair if desired. If so, use it.

* If not (e.g. you need actual semaphore behavior), use dispatch_semaphore.

os_unfair_lock is more of an optimization when speed is a must. It's a
replacement for the old OSSpinLock, but AFAICT it basically acts like a normal
mutex under contention, so I'm not actually sure why pthread_mutex would be
preferred over it, except the documentation says to prefer pthread or
libdispatch as they're "higher level" primitives.

Also of interest is dispatch_group_t, which is basically an inverted semaphore
(it counts active tasks, and lets you wait until all tasks have completed).
Under the hood it's actually a semaphore that's initialized with LONG_MAX as
the value and some custom logic (including atomics) around waiting.

~~~
asveikau
Yeah, obviously if I needed a mutex I would use a mutex, I don't have that
particular confusion and would expect pthread mutex to be well worn on the
platform. (And I noticed Darwin has some helpful pthread_mutex_attr values.) I
am talking only about the case where you really want a semaphore.

------
dilyevsky
I remember someone at google proposed new syscall to voluntarily give up time
slice from one process to another which they intended to use for co-routines
in c/c++ (which they used internationally and boy was it awesome) but it was
not accepted. Sure would be nice for this case as well.

------
rcurry
I know it's a typo, but still - I'm adding "schenario" to my list of good
words to use when describing screwy code.

~~~
JauntyHatAngle
Isn't it a joke contraction about schedule + scenario?

~~~
rcurry
You’re right, that makes it even better.

------
hart_russell
Some people might call his rhetoric abusive. I think it’s well deserved though
love.

------
eric234223
Where is the link to the origin of this discussion ? From the
realworldtech.com looks this is the first post
[https://www.realworldtech.com/forum/?threadid=189711&curpost...](https://www.realworldtech.com/forum/?threadid=189711&curpostid=189711)
. But that post is already an reply to something else's post.

~~~
derimagia
[https://www.realworldtech.com/forum/?threadid=189711](https://www.realworldtech.com/forum/?threadid=189711)
is the original post (what you linked). The quote is from
[https://probablydance.com/2019/12/30/measuring-mutexes-
spinl...](https://probablydance.com/2019/12/30/measuring-mutexes-spinlocks-
and-how-bad-the-linux-scheduler-really-is/)

~~~
eric234223
Is this your interpretation or you know it for sure. Sorry for being
skeptical. I am having hard time believing Linus Torvalds would have an
account on realworldtech.com. I have never heard of this site before. The only
place i have seen Linus reply is in the kernel mailing lists. Also is this
email id torvalds.delete@this.linux-foundation.org ? How do we even know the
reply is from linus.

------
bsaul
Sidenote : just had a random thought reading linus talking about not fighting
the scheduler:

are there common theoretical problems between a garbage collector / memory
allocator (aka : automatic handling of memory resource) and a scheduler (aka :
automatic handling of cpu resource) ?

Could there be some kind of super abstraction solving common issues to both
problem in a more mathematical approach ?

------
fierarul
What I'm reading from here:

> Or at least be aware of it, and have some virtualization-aware
> paravirtualized spinlock so that you can tell the hypervisor that "hey,
> don't do that to me right now, I'm in a critical region".

is that your kernel could be hypervisor aware?

Could you compile a kernel that flags itself as being in a critical region all
the time? Would that give you a whole CPU indefinitely?

~~~
aasasd
> _is that your kernel could be hypervisor aware?_

I mean, paravirtualization is precisely when a kernel communicates with the
hypervisor. Dunno however how scheduling factors into it—I was under the
impression that VMs are treated much like userland programs. I.e. would
similarly be scheduled off sooner or later.

------
hansdieter1337
Maybe Linus should update the docs:
[https://www.kernel.org/doc/htmldocs/kernel-
locking/locks.htm...](https://www.kernel.org/doc/htmldocs/kernel-
locking/locks.html) I think it would be useful to add a warning there.

------
Dylan16807
So, ignoring whether the code is good or not, what's the correct way to do
this benchmark? Is there any way measure the timers atomically, or to prevent
your code from getting preempted in a tiny section, or have the kernel notify
you if it gets preempted? Maybe something can be hacked in with transactional
memory infrastructure?

~~~
imtringued
Realtime scheduling. It only makes sense for a tiny process/thread that is
carefully written to avoid the bad effects of no preemption.

~~~
Dylan16807
This benchmark runs 32 software threads on a machine with 8 CPU threads.
Getting preempted a lot is a major part of what's being measured. But samples
that got preempted at exactly the wrong spot need to be filtered out, which is
tricky.

------
imtringued
Once your spinlock has exhausted it's timeslice it has lost. The point of
spinlocks is sub-microsecond latency but when you're way past that you can
only lose if you're not telling the OS that your lock is still spinning.

------
29athrowaway
Compared to other occassions, Linus showed some restrain this time.

~~~
JackFr
By Linus standards, positively collegial.

~~~
ncmncm
He didn't say "idiotic", not even once. Getting old.

~~~
29athrowaway
Some gems:

[https://lkml.org/lkml/2008/7/15/296](https://lkml.org/lkml/2008/7/15/296)

[https://lkml.org/lkml/2012/12/23/75](https://lkml.org/lkml/2012/12/23/75)

[https://lkml.org/lkml/2015/6/23/657](https://lkml.org/lkml/2015/6/23/657)

------
unilynx
Would it then be useful to have a lock that only spins if the current
remaining timeslice looks long enough?

~~~
thenewnewguy
I don't belive the kernel exposes timeslice information to userspace in linux.

Edit: Also, you'd have to predict how long a spinlock was going to spin for in
advance.

------
g051051
Note to self: do not disrespect the Linux scheduler. This reminds me of how
Sam Kinison dealt with hecklers. He'd stop, look at the heckler, and say "The
last thing you want is my complete and undivided f __*ing attention ". And
then eviscerate them.

------
Traster
Nice to see, once again, Linus driving reasonable people out of the open
source community. I'm so glad I don't have to work with people like that. No,
technical knowledge does not qualify someone to act like an asshole.

------
ChrisMarshallNY
Ah, I see that Linus hasn't strayed too far from his inner curmudgeon...

Even though he can be pithy, he's usually hitting the nail squarely on the
head.

I don't play at this level of things, but it's a fascinating conversation.
Even if we don't write low-level code, it doesn't hurt at all to understand
what's going on behind the curtain.

------
crustycoder
The sorts of problems inherent in userspace spinlocks are even called out in
the Linux manpages, shame the article's author didn't bother to look:

[http://man7.org/linux/man-
pages/man3/pthread_spin_init.3.htm...](http://man7.org/linux/man-
pages/man3/pthread_spin_init.3.html#NOTES)

Spin locks should be employed in conjunction with real-time scheduling
policies (SCHED_FIFO, or possibly SCHED_RR). Use of spin locks with
nondeterministic scheduling policies such as SCHED_OTHER probably indicates a
design mistake. The problem is that if a thread operating under such a policy
is scheduled off the CPU while it holds a spin lock, then other threads will
waste time spinning on the lock until the lock holder is once more rescheduled
and releases the lock.

If threads create a deadlock situation while employing spin locks, those
threads will spin forever consuming CPU time.

User-space spin locks are not applicable as a general locking solution. They
are, by definition, prone to priority inversion and unbounded spin times. A
programmer using spin locks must be exceptionally careful not only in the
code, but also in terms of system configuration, thread placement, and
priority assignment.

~~~
isodude
There is a lot to unfold in the manpages. See lsof.

A lesson for us all to actually look at the documentation.

But I find that it's easy to miss to read them for some reason, we rather read
one SO response instead (that eventually lead straight back to the manpage).
Which is interesting in itself. Must we invent the wheel all the time rather
than using the shipped documentation? Or am I alone in falling in that trap?

~~~
Avamander
> Must we invent the wheel all the time rather than using the shipped
> documentation?

I think it's a matter of accessibility, it is a reasonable assumption that
someone writing code wants to consult the entire internet thus if the relevant
manpages are not indexed by search engines then they won't find the
documentation. Doing minimal SEO and publishing sitemaps is quite important.
I've seen way too many institutions wondering the same question "Why are
people reinventing the wheel?" when the answer is that simply having a website
is not enough to be reachable.

~~~
crustycoder

      $ man -k spin
      pthread_spin_destroy (3) - initialize or destroy a spin lock
      pthread_spin_init (3) - initialize or destroy a spin lock
      pthread_spin_lock (3) - lock and unlock a spin lock
      pthread_spin_trylock (3) - lock and unlock a spin lock
      pthread_spin_unlock (3) - lock and unlock a spin lock
      TAP::Parser::Scheduler::Spinner (3perl) - A no-op job.
    

Sigh...

~~~
Avamander
Sigh... you really don't understand how much preliminary knowledge and
prerequisites it takes to "simply" issue that command. With this answer you
demonstrate exactly the mindset why so much information is behind some obscure
knowledge and pointlessly so. "I know it's there means others know it's there"
sigh...

Not to mention that ~95% of the Earth's population doesn't speak English as
the first language, how stuff is named in English is __not __obvious. Bad
search engines, like what `man -k` has, are nearly unusable if you don 't know
the exact term. You'd laugh when people would recommend to you to look up
words using a paper dictionary or write papers using paper encyclopedias but
`man -k` is pretty much that.

~~~
isodude
To be fair, this is one of the toughest lessons for all.

It makes total sense to those who know it by heart, but at the same time it
makes no sense to those who honestly don't. The line between is pretty hard.

Also, this is what makes me believe there is room for more discoverability-
commands in Linux-land CLI. Especially for diagnostics. Since those developing
it have a hard time seeing the same from a beginner point of view.

------
crustycoder
I think Linus was _incredibly_ polite and helpful, I certainly wouldn't have
been anything like as tolerant. I had barely got into the original article
before muttering "What about the scheduler & preemption, you idiot" to myself.
Anyone with any sort of OS background has all the points that Linus made
drummed in them from the get-go, it's a shame the author apparently didn't do
any even basic background reading. All the stuff Linus patiently explained has
been well-understood for literally decades. Old farts myself often bemoan the
fact that fewer and fewer people seem to understand how computers actually
work. It's bad enough that the author didn't understand the consequences of
what he was doing, what's really depressing is that so many other people
thought he actually had a point :-(

~~~
mindentropy
Not sure why you have been downvoted but even I too was gnashing my teeth.
Also introducing unwanted complexity of cache coherence etc and linking
irrelevant concepts to support his argument. Sigh!

These are all well understood by RTOS firmware and Linux driver people. The
author, if he has not understood the problem or what is happening should at
least use the mailing lists, read on documentation etc to understand how
things work rather than write a blog post blaming the kernel people assuming
they have not thought enough while designing.

~~~
crustycoder
Personally I'm not that surprised, criticism of any type is no longer PC. I
think you nailed it with "well understood" \- exactly so, anyone with an
interest in this area knows about this stuff.

It would have been fine if the post was along the lines of "I've observed XXX,
there's clearly something strange going on" but as you say there was a
misinformed leap to blame the kernel devs. That's not OK as bearing in mind
the exposure the post got, there's a real danger that the misinformation it
contains will become "accepted knowledge".

