Hacker News new | past | comments | ask | show | jobs | submit login
Evolution of the x86 context switch in Linux (2018) (maizure.org)
192 points by chowyuncat on Jan 24, 2019 | hide | past | favorite | 34 comments



Can someone say why is using the TSS still mandatory with software-based task switching? Is this a requirement imposed by x86?

In looking at the OS dev wiki I see the following:

>"The TSS is primarily suited for hardware multitasking, where each individual process has its own TSS. In Software multitasking, one or two TSS's are also generally used, as they allow for entering Ring 0 code after an interrupt."

Would you not be able to enter Ring0 after an interrupt with a TSS entry? Is this why it is still required?


The interrupt stack pointer comes from the TSS. Without that you're still running on the untrusted user stack with no way of bootstrapping a kernel context without corrupting the user state.


Oh right, without a mapping its a "chicken and egg" situation. Cheers.


I recall reading a paper comparing Linux and Solaris context switch times in ~98 and Linux was 10-100X faster. Solaris did something incredibly slow and safe.


Real context switches on Solaris were very slow which is why they had LWP. But Linux process context switches were also faster than Sun's LWP switches.


Enjoyed this article, anybody know the significance of adding the do..while(0) loop within the macro starting Linux 1.3?

Was curious if it guards against some C pre-processor issues.

  /** include/asm-i386/system.h */
  #define switch_to(tsk) do {
    [...]
  } while (0)


Good answer here: https://stackoverflow.com/questions/257418/do-while-0-what-i...


Just to add more context, this is a very common cpp (c pre-processor) idiom. You'll find it in most non-trivial C projects somewhere.


Abbreviating the C Preprocessor as cpp is very confusing imho.


It's a common abbreviation older than C++. You used to even be able to run the c pre processor on arbitrary non-C files by using the cpp command.


You still can.


Tell that to CPPFLAGS.


It enables you to invoke the macro as if it were an expression statement consisting of a function call, regardless of where it appears.

For details, see http://c-faq.com/cpp/multistmt.html:


In particular, a single statement. I'm sure the link covers it, but:

    if (foo)
      MULTI_LINE_MACRO;
Breaks without some wrapper like if (1) { A; B; } or do { A; B; } while (0):

    if (foo)
      A;
      B;  // oops, unconditional (e.g., "goto fail")


It also gives you a nice scope to keep local variables in, but there are other ways to accomplish that too.


Yeah, that's a good point too.


An addition to "Linux 2.2 (1999)" is: introduced meltdown vulnerability. That was the then-unknown cost of software context switching.

Later, with Red Hat's 4g4g kernels that Linus rejected, the problem would go away for people who installed Red Hat's version of the OS on systems with many gigabytes of memory.


Can you elaborate? How does relying on pure TSS for context switching prevent meltdown?

What were the 4g4g kernels? Might you have any literature and/or on those?


Separate address space for kernel and user. Hardware will use TSS to switch address space as needed for syscalls.


Huh? Hardware TSS has nothing to do with separating address spaces. It’s just a trick for switching the address space, and it’s really quite slow on modern CPUs. In theory, a kernel could use hardware task switching to switch address spaces on user/kernel transitions by forcing a hardware task switch when this happens, but the performance impacts would be considerably worse than KPTI.

What does seem to mitigate Meltdown on some CPUs is enabling segment limits for user code. This does nothing for 64-bit code, though.

edit: not to mention that there are no hardware context switches on 64-bit kernels. AMD removed support entirely in 64-bit mode. The TSS still exists, but it’s just an awkward dumping ground for a couple of data structures.


Do you the reason why Linus rejected this idea?


The main reason seemed to be the relatively bad performance of the hardware task switch (loading segment registers and the page table base) that would be required for any system call.

Of course, that turns out to be the fix for meltdown, unless you have the process-context identifiers (PCID) available on Haswell chips and newer. The meltdown fix for older CPUs, such as the Pentium III and Intel Core, is roughly the same as the 4g4g kernel changes.

BTW, the 4g4g kernels were created for a different reason. The kernel needed more virtual address space for itself, and thus couldn't share with user code. This was for a time when people were trying to run 32-bit kernels on systems with 32 gigabytes of RAM.


>"Of course, that turns out to be the fix for meltdown, unless you have the process-context identifiers (PCID) available on Haswell chips and newer."

Using TSS based switching is incompatible with PCIDs? Or is it incompatible with separate address spaces for user space and kernel space?

PCIDs are process ID tags on cache lines correct?


TSS switching is incompatible with running in 64-bit mode. It is very slow. Doing most of the same actions (reload segment registers and the page table base) in software is also very slow. Prior to the meltdown issue, Linux system calls had avoided most reloads of the segment registers and page table base.

PCIDs are incompatible with older hardware. They are modestly slow. I think the PCID state includes the TLB.

That pretty much means the kernel must support both methods. The PCIDs is used when possible. When the hardware doesn't support PCIDs, Linux must instead reload segment registers and the page table base, either step-by-step in software or via a TSS switch.

BTW, I had to implement x86 hardware task switching for an x86 emulator. The complexity is insane. See my "Who is Hiring?" post if that sounds fun for you.


I think you may be a bit confused here. PCID is a feature that lets the kernel avoid a TLB when CR3 is written. With or without PCID, CR3 gets written. The segment registers have nothing whatsoever to do with this on a 64-bit system.


The explanation is a bit awkward because I'm trying to describe numerous situations. (pre-2.2 kernels, post-meldown kernels, in-between kernels, 32-bit builds, 64-bit builds, using PCID hardware, not using PCID hardware...) It looks like 7 of the 12 possibilities are valid, more or less.

Some segment registers are reloaded on a 64-bit system. That includes CS, DS, SS, and GS. The 32-bit systems must additionally reload ES. All segment registers are loaded for Linux 2.0 and older, via hardware task switching.

CR3 does not get written for system calls when running on a normal Linux from version 2.2 until the meltdown workaround hit.


Nice article!

If you want to have lots of fun, you could look at switch_mm() on a modern kernel :)


This is really well done, bravo.


Very thorough. Nice job.


anyone knows what Ingo Molnar is doing these days?


A ton of stuff related to eBPF, last I saw.


I really like how good the article looks when printed. I enjoy reading long, in-depth articles much more when I can read them in print. Unfortunately many blog posts need a lot of tweaking until I can get an acceptable print result. This one looks good enough without any trickery.

Thanks to the author, for caring about the paper people. : )


out of curiosity, since it changes how the page prints and I haven't experimented much: have you tried printing while in the browser's "reading" mode? Or does that tend to be worse?


Not OP,but if it looks good on "reading" mode it prints nicely. Plus you can adjust font size and print width to conform to your taste.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: