
OpenBSD was right to disable hyperthreading [video] - rodrigo975
https://www.youtube.com/watch?v=jI3YE3Jlgw8
======
dleslie
History will show that Theo was right in a manner similar to how Stalman was
right: technically correct analysis and eerily accurate predictions, but
lacking in sufficient charisma to create more than a small following.

To some extent, you might say they are like Cassandra; speaking the truth but
not believed or listened to.

~~~
knocte
> but lacking in sufficient charisma to create more than a small following.

IMO it's not that Linus has more charisma than Theo, it's simply the network
effect of one project over the other.

~~~
dleslie
It wasn't Linus; it was RedHat, SuSE, and the rest of the early commercial
Linux endeavours.

~~~
knocte
Right, this is what I said.

------
robmusial
For those that can't watch the video GregKH says that OpenBSD was right to
disable hyper-threading earlier than Linux in response to Spectre and Meltdown
and now Linux disables it too.

He also caveats it by saying they were right for "a little bit of the wrong
reasons" but at least in this clip doesn't expand on what he meant by that or
what those wrong reasons were or why they were wrong.

~~~
dooglius
Why should the kernel be making this decision at all, rather than leaving it
up to the distro or a command-line boot argument?

EDIT: can't see the video so if it is just a default rather than a forced
disablement that's fine.

~~~
admax88q
Because sane defaults are important.

~~~
Crinus
Isn't this a "sane" default only in specific contexts though? (VMs). For a
desktop PC that almost always runs a single heavy task (games, rendering,
video encoding, etc) hyperthreading can be a day and night difference.

~~~
sharpneli
One must consider the potential damages that can happen when the default is
wrong, as it’s nigh certain that people are going to be careless and not
change it.

If the desktop PC has wrong default the performance is bad. Still functional
though.

If in case of VM the default is wrong we will read another headline about how
N million customers of $company got their personal data leaked.

~~~
Crinus
I think we'll read such headlines regardless of that setting :-P.

I'm actually wondering if there should be some sort of premade "profiles" when
it comes to default settings. Debian for example is used in a lot of contexts
so perhaps it'd make sense to have a way to ask you what sort of usage you'll
do when installing and provide different defaults based on that (not just at
the initial installation time but also when installing some package, the
default settings would be based on the profile you chose).

~~~
beatgammit
It's pretty easy to change a default, so it's completely reasonable for a
gaming distro (say, SteamOS) to have different defaults than a server distro
(say, CentOS). Most Linux distributions don't ship the kernel in it's default
configuration anyway, so this would really only affect those who use the
kernel directly from the source.

------
danarmak
Two of the three (current) top-level replies compare BSDs to Linux in general,
but that really has nothing to do with whether you disable HT. Using Linux
should not have stopped anyone from listening to Theo and disabling HT months
ago. Your security authorities don't have to be your kernel developers.

~~~
josteink
> Using Linux should not have stopped anyone from listening to Theo and
> disabling HT months ago.

Iirc OpenBSD actively disabled it for you, making it the default.

Nobody else did that.

~~~
baybal2
In fact, cache p3 timing attack was known as "theoretically possible" back in
nineties, but was booed out of the conversation by "big name security
analysts"

~~~
h2odragon
"I told you so" gets less satisfying each time you get to say it, yaknow? (*
commiseration, no criticism)

------
ajsnigrutin
A wee bit offtopic, but if we look at the VW/dieselgate, and the aftermath of
it all, and the class-actions, returns, refunds, etc, and hyundai/kia lies
about gas milage and people getting refunds for gas...

...when is something like this going to happen to intel?

We've bought CPUs with excpectations of promised performance (like people did
with emission expectations and gas milage expectations), they messed up, and
we get lower speed, and now no hyperthreading and still no refunds? If i
bought a 60" TV and the picture was only 50" with a black border around, i'd
return it immediately... why isn't there some action regarding CPUs?

~~~
umvi
If I sell you a lock, and then 10 years later someone finds a vulnerability
with the lock I sold you, should I refund you? That seems absurd. You are
basically saying the product has to be perfect and the architects have to be
able to see the future. Even if your hardware is formally verified, people can
do physical attacks like listening to high frequency chirps of your cpu and
using _that_ to break security. Do you still deserve a refund?

There is no such thing as perfect security. It is a cat and mouse game that
will continue until the end of time, requiring ever greater resources.
Therefore... all software and hardware should be free because all software and
hardware is defective?

~~~
3JPLW
Downvoted for the straw man — Intel is currently selling processors with,
e.g., 8 cores and 16 threads without any asterisks or caveats. That's in their
official marketing materials. Currently.

[https://a.sellpoint.net/a/Qo3wL1no.jpg](https://a.sellpoint.net/a/Qo3wL1no.jpg)
(via NewEgg)

[https://www.intel.com/content/www/us/en/products/processors/...](https://www.intel.com/content/www/us/en/products/processors/core/i9-processors.html)

~~~
rrss
And it's true...

Has Intel ever said "we guarantee that hyperthreads are entirely isolated from
one another?"

~~~
3JPLW
One quick google later:

[https://www.intel.com/content/www/us/en/architecture-and-
tec...](https://www.intel.com/content/www/us/en/architecture-and-
technology/hyper-threading/hyper-threading-technology.html)

> By combining one of these Intel® processors and chipsets with an operating
> system and BIOS supporting Intel® HT Technology, you can:

> * Run demanding applications simultaneously while maintaining system
> responsiveness

> * _Keep systems protected_ , efficient, and manageable while minimizing
> impact on productivity

~~~
rrss
I have no idea what "keep systems protected" means (and probably neither did
the person who wrote that).

That statement is a long way from an actual guarantee that there is no way for
one logical thread to extract information about another.

------
3xblah
Full interview:
[https://www.youtube.com/watch?v=sDrRvrh16ws](https://www.youtube.com/watch?v=sDrRvrh16ws)

One of the things the Linux kernel developer says in the interview is that
researchers are going through Intel patents to find "security bugs".

He says this is "fun". Twice. He seems pretty nonchalant about these issues.
Like it is great to fix them, but not like it is too important or anything to
worry about. He says what is most important to him is that Linux "succeeds".
The attitude is reminiscient of Microsoft in their heyday. Drunk on success.
He even calls out a Microsoft employee he is working with. He says the
companies contribute "selfishly". This is no different from BSD. Users do not
determine how much security is prioritized, the contributors do. However, what
happens when the biggest kernel contributors are companies?

He also indicates he does not agree with Stallman philosophically on
technology issues. We do not get any details of the specifics of their
disagreement.

As a user, I think is it somewhat easier to keep track of Net/OpenBSD kernel
contributions than it is to keep track of Linux kernel contributions. I might
be wrong on that. As far as I can tell, the biggest contributors to
Net/OpenBSD kernels are still individuals and are not acting directly on
behalf of corporations.

~~~
asveikau
I haven't had a chance to watch but it seems like you are berating him for
figures of speech and/or speaking styles. As far as I know his contributions
to the kernel are pretty vast. We are all human and security isn't his only
focus so maybe give him a break.

------
Syzygies
I disable hyperthreading for better performance.

In my experience as a mathematician building parallel compute servers,
hyperthreading generates more heat than it is worth. I can overclock further
without hyperthreading, to more than overtake the faint advantage that
hyperthreading offers at a given clock speed. So I now buy binned, delidded
processors from Silicon Lottery, choosing the best reasonably priced speed of
the best cpu without hyperthreading. That would today be the i7-9700 @ 5.1GHz
for $400.

~~~
hinkley
The number of generations of processors where this has been true is really
astounding to me. It really makes me wonder why they persist with this line of
effort instead of doing something else, like cores that share logic units
only.

~~~
AaronFriel
Because for a large number of workloads, hyperthreading gives real performance
improvements.

The vast majority of consumers aren't running compute heavy workloads that are
more amenable to SIMD work (which it sounds like this might be) than the sort
of highly branching, often stalled work that general purpose programs do.

------
dnautics
Perhaps someone might know better than me: why is hyperthreading necessarily
bad? Can't you just keep it on and give your tenants cores with affinity? For
example, some tenant wants two cores, you give them two vcores on the same
core; some tenant wants four cores, you give them two vcores over two, etc.

~~~
dnautics
actually I just took a shower and answered my question (I think) - many of the
hyperthreading bugs don't breach the process divide, they use errors in
hyperthreading statefulness to use a side channel to breach _memory_ divide,
and the cores share memory, whether or not who's on what core, if any one core
gets compromised, you could potentially access any of the core's memory.

~~~
muricula
Close but not quite -- sibling hyperthreads (logical cores) share cache state.
Physical cores do not share cache state. Different processes, threads, or VMs
on sibling hyperthreads (by definition on the same physical core) can infer
the other's memory state based on the cache state.

If an attacker is pinned to one hyperthread, and the victim is pinned to
another which isn't a sibling hyperthread, none of the spectre attacks will
work since the cache state isn't shared.

As an attacker with code exec on a core, you can theoretically play games with
the OS scheduler until you're running on a sibling core with your victim
thread/process/vm.

~~~
dnautics
That's not true. Spectre works due to speculative execution leaking memory
data through a side channel exposed by hyperthreading. That memory can be in
use by any of the threads, not just the ones on sibling threads.

~~~
muricula
The side channel for most of the spectre variants is the latency of misses on
the cache lines. L1 and L2 cache lines are local to a physical core. As far as
I know, nobody has made any of the spectre variants work by measuring the
latency of L3 cache misses, which are local to NUMA nodes if I understand
correctly, but I'd love to hear otherwise.

The most recent round of spectre variants measured the latency of the line
fill buffers and other parts which are local to a physical core's memory
subsystem.

------
Woodi
"Disable HT when you don't trust your users"

Also:

Do not run not your own code, like eg. JavaScript.

There is way too much short living code.

Why we not build software libraries as we learn using computers and then use
it for the rest our life ?

Programmers should switch to programming human domain problems, not constantly
reimplementing Start Menu hierarchy.

Btw. anybody uses Tripwire ? Eg. with apt-get or pacman and saving hashes to
ro device ? ;)

------
mikece
Why aren’t the *BSD operating systems more popular in the server and
workstation spaces?

~~~
dwheeler
Linux came _after_ the BSDs, so you would think the BSDs would have won.

There are many reasons Linux-based systems are generally much more popular
than the BSDs in the server and workstation spaces. Here's why I think that
happened:

* GPL vs. BSD license. Repeatedly someone in the BSD community had the bright idea of creating a proprietary OS based on a BSD. All their work was then _not_ shared with the OSS BSD community, and the hires removed expertise from the OSS BSD community. In contrast, the GPL forced the Linux kernel and GNU tool improvements to stay in the community, so every company that participated _improved_ the Linux kernel and GNU tools instead of making their development stagnate. This enabled the Linux kernel in particular to rocket past the BSDs in terms of capabilities.

* Bazaar vs. Cathedral. The BSDs had a small group who tried to build things elegantly (cathedral), mostly in "one big tree". GNU + Linux were far more decentralized (bazaar), leading to faster development. That especially applies to the Linux kernel; many GNU tools are more cathedral-like in their development (though not to the extent of the BSDs), and they've paid a price in slower development because of it.

* Multi-boot Installation ease. For many years Linux was _much_ easier to install than the BSDs on standard x86 hardware. Linux used the standard MBR partitioning scheme, while the BSDs required their own scheme that made it extremely difficult to run a BSD multi-boot setup. For many people computers (including storage) were very expensive - it was much easier to try out Linux (where you could dual-boot) than BSDs. The BSDs required an "all-in" commitment that immediately caused many people to ignore them. I think this factor is underappreciated.

* GNU and Linux emphasis on functionality and ease-of-use instead of tiny-ness. GNU tools revel in all sorts of options (case in point: _cat_ has numerous options) and long-name options (which are much easier to read). The BSDs are often excited about how small their code is and how few flags their command lines have... but it turns out many users want functionality. If tiny-ness is truly your goal, then busybox was generally better once that became available circa 1996 (because it focused specifically on tiny-ness instead of trying to be a compromise between functionality and tiny-ness).

Some claim that the AT&T lawsuit hurt the BSDs, but there was lawsuit-rattling
for Linux and GNU as well, so while others will point to that I don't think
that was serious factor.

Here's one article discussing this:

[https://www.channelfutures.com/open-source/open-source-
histo...](https://www.channelfutures.com/open-source/open-source-history-why-
didnt-bsd-beat-out-gnu-and-linux)

~~~
spamizbad
An excellent summary. I think by the late 90s BSD could operate cleanly
alongside Windows but by then Linux had become the default "free" *nix choice.

For me, I ran FreeBSD for about a year around 1999. I lasted about a week
before breaking down and installing a GNU userspace. Excellent CLI ergonomics
for the day.

Linux had quite a few "easy to install" distros where, if your critical
hardware was fully supported, you had something that was easier to get up and
running than Windows 95/98\. X configurations and sound drivers were a
sticking points back then though. BSD had no such "easy mode" gateway drugs.

~~~
693471
No, by the late 90s BSD had dominated the hosting world because the TCP/IP
stack allowed better scaling on the same hardware. Tons of ISPs ran nothing
but FreeBSD.

~~~
cpeterso
Yahoo and Hotmail ran FreeBSD for a long time, too.

------
kiney
I'm not up to date with all the spectre mitigations. Is AMDs SMT
implementation still consideres secure?

~~~
derpherpsson
Yes.

------
tptacek
The title of this should be "OpenBSD Was Right About Hyperthreading", which is
all he says in the video. "They were right for a little bit of the wrong
reasons, but they were right".

------
big_chungus
There will be a large number of people who don't like this. However, consider
the following: most people, even those who use linux, don't want to bother
with too much configuration. For those people, it is probably "good policy" to
disable a potential security risk. Leave the option for those who understand
what they are doing and wish to go ahead, by all means, but make the default
secure. For instance, I have server machines at home which do not handle
critical or particularly sensitive data. They are securely firewalled, and
shouldn't be touched by anything not trusted by me. On those machines, I
sacrifice some potential security for performance. No patches for meltdown,
spectre, foreshadow, l1tf, etc. as they have significant impacts on
performance and these boxes don't really need it. For my firewall, however, I
turn on all patches and sacrifice quite a bit of performance (in other areas,
not just patches which cause slow-downs) for more security. It handles un-
trusted input; other boxes do not. I can plan accordingly.

The important thing to remember is that I am intentionally making those
choices and considering that trade-off. Most users are not. It is somewhat
better to leave things like ASLR enabled and hyperthreading disabled by
default, because that is what is best for the average user.

Lastly, it is of not that if something such as this is disabled and there is a
resulting security breach, every one will run around screaming, "Linux is
insecure; ahh!" This will happen, even if the user is informed he is supposed
to do this for things which need to be secure.

------
Causality1
Would it be that difficult to run chips in a secure mode with no
hyperthreading or branch prediction when handling sensitive information and
then takes the brakes off for normal operation? I mean, I wouldn't really care
if someone was watching everything I do for 95% of my computer use time.

------
lcall
There was a longish recent thread on a mailing list where people described
well (and linked to) why they like OpenBSD.

[https://marc.info/?l=openbsd-
misc&m=156700281107546&w=2](https://marc.info/?l=openbsd-
misc&m=156700281107546&w=2)

The most recent one I saw gave a reasonable sense of some tradeoffs:

[https://marc.info/?l=openbsd-
misc&m=156750048426578&w=2](https://marc.info/?l=openbsd-
misc&m=156750048426578&w=2)

I'm sure they welcome donations. :) Software they have written have benefited
many, directly or indirectly (like OpenSSH).

...but the whole thread was interesting I thought.

------
Animats
If you run BSD on AWS are you running a hypervisor underneath that's still
doing hyperthreading? On most AWS instances, a "virtual CPU" is really a
hyperthread.[1] _" Each vCPU is a thread of either an Intel Xeon core or an
AMD EPYC core, except for T2 and m3.medium."_

Is anybody actually attacking AWS this way? It seems a promising attack
vector; you get to run your own code on the same machines as others.

[1] [https://aws.amazon.com/ec2/instance-types/#instance-
details](https://aws.amazon.com/ec2/instance-types/#instance-details)

~~~
kinghajj
I don't think that vector would work, as only a single tenant's VM is
scheduled on any particular core. That's why the minimum number of vCPUs for
instances on hyperthreaded hosts is 2.

------
imtringued
Hyperthreading is cool in theory, but most workloads that take full advantage
of it are usually filled with code that is doing nothing but following
pointers. It is generally unoptimized code or code written in an interpreted
language that doesn't focus on speed anyway. It also requires the program to
take advantage of multiple threads in the first place. This means languages
like python don't benefit at all but code written in C/C++ are so optimized
that hyperthreading does nothing but divide the cache into two halves.

------
mixmastamyk
I bought a core i5 a few years ago merely to save money as I edit text files
for a living. By chance it turns out to not offer hyper-threading. Not sure it
matters much but interesting none the less.

------
shmerl
Does it refer to Intel specifically (hyperthreading is Intel's term), or it
any processor (then the general term should be SMT).

------
fierarul
From the video: "two weeks ago: another one was released... publicly"

So... there's more to come.

------
vallismortis
I'm still not - um - disabling, ah, hyperthreading, because, I'm paying for,
ah, 4 cores on a 2 core ah, processor.

------
segfaultbuserr
The title has a clickbaity tendency, it should be changed from "OpenBSD was
Right" to "OpenBSD was Right (on disabling hyperthreading)".

------
baybal2
Greg is a very important man, many bet he will be the one to take over the
Linux development after Linus

