
Tale of OpenBSD secure memory allocator internals – malloc(3) - zdw
https://bsdb0y.github.io/blog/deep-dive-into-the-OpenBSD-malloc-and-friends-internals-part-1.html
======
snvzz
Always good to see that some systems still do some engineering. Linux has lost
this. It got so complex that only small patches get in, so the hacks continue
to stack up.

OpenBSD does interesting stuff like this or pledge(2) and unveil(2).

Dragonfly does interesting stuff like their concurrent lockfree/lockless
servers approach instead of using kernel locks, and HAMMER2[0].

Genode[1][2] implements multiserver systems supporting many kernels including
seL4, making good use of capabilities.

[0]:
[https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/sys...](https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/sys/vfs/hammer2/DESIGN)

[1]: [https://www.genode.org](https://www.genode.org)

[2]: [https://www.genodians.org](https://www.genodians.org)

~~~
burfog
Linux has plenty of engineering. Of course that doesn't mean identical
features.

Linux has a VFS with lockless path lookups. The push to get Linux running on
machines like the SGI Altix, with at least 10240 cores running one shared
instance of the kernel, has really helped with locking issues. Another factor
has been the push for real-time performance, with and without choosing the
option to make the kernel preemptable.

~~~
snvzz
And all of this is based on hacks and ugly workarounds, due to the kernel
being a monolithic design.

When the kernel isn't monolithic, kernel preemption isn't necessary, or worth
it[0]. Imagine if instead of the, I argue insane, complexity added by the
fine-grained SMP locks and the unholy mess that is kernel preemption, the
kernel had been split into separate processes, which maps to SMP cleanly.

Actually, I don't have to imagine it, because that's what Dragonfly has been
progressively doing since they forked FreeBSD, and what microkernel
multiserver systems like Genode/seL4 or HelenOS are from the get-go.

It turns out, despite having very little manpower (which is why they chose to
focus on x86-64), Dragonfly has performance and scalability competitive to
Linux today. Imagine if they had more people. I will not be too surprised when
Dragonfly finally leaves Linux in the dust and people wonder how the hell,
which is something I predict will happen. Linux hit a ceiling despite all the
money thrown into it. Dragonfly hasn't yet.

[0]:
[http://ts.csiro.au/publications/nicta_full_text/5859.pdf](http://ts.csiro.au/publications/nicta_full_text/5859.pdf)

~~~
dntbnmpls
> And all of this is based on hacks and ugly workarounds, due to the kernel
> being a monolithic design.

But most BSD ( including OpenBSD which you just praised ) is monolith as well.

> I will not be too surprised when Dragonfly finally leaves Linux in the dust
> and people wonder how the hell, which is something I predict will happen.

Dragonfly would have to first leave FreeBSD in the dust before taking on all
the linux distros.

Microkernels are great for learning about kernels. Monoliths have been great
for general and corporate use. Perhaps the future is hybrid, only time will
tell.

~~~
snvzz
>Dragonfly would have to first leave FreeBSD in the dust before taking on all
the linux distros.

You're late to the party, as Dragonfly is already outdoing Linux in some
areas, such as the network stack[0].

>But most BSD ( including OpenBSD which you just praised ) is monolith as
well.

I praised them for the engineering they've been doing. They still retain
flexibility, despite the monolithic design. Linux went all-in with their smp
locking and kernel preemption, and the costs of getting any work done on the
source code have skyrocketed since.

>Perhaps the future is hybrid, only time will tell.

Hybrid is a marketing term. It's a blurry line between a monolith and a so-
called hybrid. What they're marketing is structure. Structure is good. Netbsd
is an example of a well structured system. Linux is the counterpart: A poorly
structured one.

>Monoliths have been great for general and corporate use.

UNIX is an old design. Corporate hates taking risks. However, there's massive
scale deployments of microkernel systems, they're just not on your face; the
visibility is actually quite low for these. But they do tasks that greatly
benefit from or outright require such a design.

Monolithic design limits are more evident than ever, as we're hitting them
left and right. So this is changing. For instance, besides systems such as
Genode and HelenOS, Google is working on Fuchsia and Huawei on HarmonyOS.
Brace for impact, as waves will be made when this all finally hits.

[0]:
[https://leaf.dragonflybsd.org/~sephe/perf_cmp.pdf](https://leaf.dragonflybsd.org/~sephe/perf_cmp.pdf)

~~~
jayp1418
Maybe rump kernels might be future :

[https://research.csiro.au/tsblog/using-rump-kernels-to-
run-u...](https://research.csiro.au/tsblog/using-rump-kernels-to-run-
unmodified-netbsd-drivers-on-sel4/)

~~~
snvzz
It's amusing you mention this. I just used the rump implementation of fat
filesystem (rump_msdos) to mount a fat floppy on netbsd on an Amiga with a
kernel where I didn't build msdos support in.

------
saagarjha
A bit of constructive criticism: it's nice when people do deep dives of
components, but I think this one is _too_ deep: it basically pastes all the
code that is executed and then explains _exactly_ what the code is doing,
which isn't really that useful; I don't need to be told how to read C, as I
can do so myself. And in fact, I think it's a bit counterproductive: I very
quickly got lost in the minute details presented. What I _really_ want is an
overview of what the code _does_ (and ideally, why) not how it does it.

~~~
bsdb0y
First of all, Thank you for sharing your thoughts.

I am beginner into this.

And, yes, it is deep because OpenBSD malloc(3) has lots of features and I have
tried to cover each and every line of code. So, I have shared all my learning
and the process of how I learned.

In future, if people want to learn anything that requires malloc(3)
understanding they can refer to this.

And, yes I have to paste the code otherwise it would be difficult to track the
explanation and respective code.

I have explained the following things: \- What is the code doing \- Reason on
why is it doing that \- tried to explain the developers' idea behind that code
algorithm \- tried to explain the mathematical calculations on the basis of
sample code through the debugger

And, the blog is for everyone, irrespective of ability to read C, or
understanding of malloc(3) internals.

If you got lost then please keep patience because for me it took exactly 30
days to understand, still, I am learning it. At least read the whole blog
again and then if you want, you can make the remark as per your understanding.

If you go through the whole blog, you will find out mailing lists and twitter
links where the author discussed his queries and the reasons/ideas with the
developer.

Hope I cleared your queries, if you still have some doubts then please read
the whole blog again, or you can also directly jump into the source code and
learn it from there.

------
gigatexal
Let's have a HN style discussion of the article on the website and not a flame
war.

------
carapace
OpenBSD in C: A panic room made out of knives.

~~~
chungy
I feel like either OpenBSD or NetBSD would probably make good candidates for
experimenting with rewriting the operating system in Rust. They're small
enough to make some significant inroads in rapid pace, but also general
purpose enough to not be useless.

~~~
renox
If memory serves, NetBSD's strength is its portability.. So given the current
state of Rust's portability, I doubt that they are very interested.

As for OpenBSD, I wouldn't hold my breath: remember that Rust isn't the first
'safe' low level language but the others (Ada for example) didn't find success
inside OpenBSD.

That said development follow fashions, so who knows?

~~~
chungy
My own feelings didn't reflect that the upstream projects should, nor
necessarily would, switch. I was thinking more in terms of a fork to research
the avenue.

~~~
pjmlp
Better look at Genode OS and what they are doing with C++, Ada and SPARK
integration, than hoping any UNIX clone will ever use anything else other than
C.

