
Integrating “safe” languages into OpenBSD? - dmm
https://marc.info/?l=openbsd-misc&m=151233345723889&w=2
======
reggieband
I remember I stopped frequenting /r/programming when it became this weird
Haskell echo chamber. It was a bubble where a small but vocal faction inside
the community seemed to over react to any criticism. They methodically and
tirelessly responded to every comment with an endless litany of "facts"
showing how Haskell could do anything from os kernels to game programming.
They touted every industry mention of Haskell use and demanded evidential
proof for any claim Haskell might not be suited for every situation. It
reminds me of the Sea Lion comic[1]. I'm not sure if that faction still holds
as much sway today.

I get the same feeling on Hacker News about Rust. There is a point where well-
meaning evangelism does more harm than good. I sometimes get the feeling
people spend more time writing defences of these languages than they spend
time writing programs in them.

1\.
[http://wondermark.com/c/2014-09-19-1062sea.png](http://wondermark.com/c/2014-09-19-1062sea.png)

~~~
stochastic_monk
It's become clear to me that a lot of the Rust fanatics don't actually know
how C++ works and and Rust is their first low-level language. The ones I'm
talking about know enough to make them sound like they know what they're
talking about unless you _actually_ know what they're trying and failing to
accurately describe.

It sounds and seems like a language worthy of more investigation on my part,
but I have to agree that the militant, presumptive attacks on any language
besides their Chosen One has left a bad taste in my mouth.

~~~
HumanDrivenDev
_It sounds and seems like a language worthy of more investigation on my part,
but I have to agree that the militant, presumptive attacks on any language
besides their Chosen One has left a bad taste in my mouth._

This 100 times over. It's a language that seems novel and strong enough to
stand on its own merits - but there's a section of the community that makes me
want to stay away.

I wish they'd spend less effort re-writing already working tools in C and more
effort writing new, better tools. A drop in replacement for "ls" isn't that
exciting for me.

~~~
Manishearth
> I wish they'd spend less effort re-writing already working tools in C and
> more effort writing new, better tools.

Is this really a thing? Folks keep pointing out this bugbear of folks going
out and asking projects to rewrite in Rust and ... there aren't that many
examples of this.

If anything most of the more famous Rust projects (ripgrep, redox, etc) are
clean-slate.

~~~
thatfunkymunki
[http://transitiontech.ca/random/RIIR](http://transitiontech.ca/random/RIIR)
this phenomenon has been documented

------
SCHiM
The points made against switching to newer languages are valid. It will cost
time, it will cost effort, it will take many years of work to convince
stubborn maintainers to switch.

But it will not stop _everyone_ from adopting better languages, and their
efforts will eventually surpass the older, less secure systems.

When there's feature parity, each and every new exploit will be called out:
"this wouldn't have happened in our system'. And that is a good thing. There
_are_ better alternatives to C.

As for the point about how nobody is working on replacements, that's wrong.
There's (partial) replacements for most coreutils written in rust, and a whole
kernel has been under active development for years now.

~~~
joshklein
I think this misunderstands the OpenBSD philosophy. The more likely it is
"everyone else" disagrees, the more likely it is Theo is choosing the right
course for the project.

The system being secure is a secondary benefit to it being comprehendible and
coherent to an individual. It's not enough for the output of a magic box to be
a better widget, even if the widget is better in every measurable way. The box
itself must not be magic.

This isn't "right" or "wrong". It's simply a stance that values an
individual's ability to understand their computing device from top to bottom.

Rust is an amazing and wonderful piece of technology, produced by a great
investment of energy by brilliant minds. I'm incredibly grateful it exists and
look forward to it's continued development and adoption.

But I'm not brilliant, and I don't have a lot of energy, and so I'm also
grateful there exists an operating system I can understand, and other people
who continue to work to make that operating system useful.

Much open source technology is created by people with a financial incentive
for others to use it. That's fine. OpenBSD is written by people who want to
use it. It's usefulness to others is coincidental. That's fine too.

I'm glad there is so much money funding the development of open and free
computing innovations. The world is a much better place for it.

But there isn't a lot of money funding the rest of the unix philosophy besides
the "open" part, like the parts about composability and anti-monolithic
design. Because those things are not valuable in financial terms, and are
potentially even destructive towards the purpose of capturing value at all.

I am grateful there is a small radical free operating system defending our
freedom from monoculture.

And I'm similarly glad for their defacto antagonism towards proselytizing of
all kinds, even if that proselytizing is done with good intention, and even if
that proselytizing turns out to be right.

That's why you need to show them the code. When you show the code, it means
you understand. Until you understand, you're not free.

At least, this is my interpretation. I'm not affiliated with the project in
any way. Just a fan.

~~~
nickpsecurity
"But I'm not brilliant, and I don't have a lot of energy, and so I'm also
grateful there exists an operating system I can understand, and other people
who continue to work to make that operating system useful."

Writing correct, efficient C takes more years to master than people take to
learn Rust that I've seen. Also, the C compiler and many other parts of
OpenBSD are black boxes to their developers. Your worries apply equally to
their situation unless you've read and understood all their dependencies. On
top of it, the OpenBSD people are _always_ rewriting stuff for claimed
benefits in maintainability or security. It's just when we talk a safe,
systems language that can be as simple as a Wirth language or complex as Rust
they suddenly can't justify the effort of even piecemeal replacement.

Then, next week, they'll put piles of effort into a mitigation across their
toolchain whose benefits are so probabilistic even they can't tell you what
attacks will fail or succeed. It's worth it, though, to improve their security
standpoint. Unlike the pain of recoding even one utility in something like
Rust. That's where this email draws the line.

Of course, I encourage people to do exactly what he asks every time another BS
argument is raised. He's worried about drawbacks of a non-C language? Make
something like Cyclone or get a Wirth language better at selectively turning
off safety compiling to C w/ great C FFI. He's worried about compile times?
Fix the compiler. He says utilities aren't rewritten in the better language?
Rewrite them showing its advantages esp against the bug reports in OpenBSD's
tracker. Just keep pounding away at the problems until he runs out of excuses
to import stuff or is extra clear they simply don't like language/method X for
arbitrary reasons. Regardless, you get a pile of safe utilities/modules for
OpenBSD to do useful things on top of fast, safe tooling. Win, win. :)

Alternatively, contribute those ports to OS's that want to bring in best-of-
breed tooling for boosting safety and security. They're usually smaller, less
mature, and need all the help they can get.

~~~
oblio
My impression is that the Rust developers and community are firmly in the
practical camp. They will keep plowing at Rust and its ecosystem. They will
improve and optimize every nook and cranny they can find.

I wouldn't be too worried about Rust, it will have a niche at least as big as
Ruby's is and IMO it will be way more entrenched since its target domain moves
way slower and the barriers to entry are way higher than for scripting
languages and web frameworks, IMO.

------
xwvvvvwx
Maybe I'm misunderstanding but there is a pretty serious effort to rewrite all
the gnu coreutils in Rust:

[https://github.com/uutils/coreutils](https://github.com/uutils/coreutils)

~~~
IshKebab
And Go: [https://github.com/ericlagergren/go-
coreutils](https://github.com/ericlagergren/go-coreutils)

Does he really not know this or is he ignoring them to make a point?

~~~
akerro
It uses remote dependencies. First compilation in a single-run environment
will be very slow, there won't be second compilation. Go compiler is fast when
you add the `-i` flag, without it it takes a couple of seconds to compile a
few hundred lines, a few more minutes when you have to `go get` packages. Now,
github goes down, your build is broken for that time.

~~~
gjem97
Not sure what the "it" is in your first sentence, but allow me to address the
rest of your points. "First compilation ... will be very slow": the go
compiler is in fact quite fast, orders of magnitude faster than C++ or rust
compilers. The fact that there's a noticeable pause when you want to compile
thousands of files does not mean that its slow. "There won't be a second
compilation": not for each top level tool, but there certainly could be shared
packages that don't need to be recompiled. "Go compiler is fast when you add
the `-i` flag", ok, do that then. "Now, github goes down, your build is
broken": you only need to depend on external github references if you want
always to build against the latest version of your referenced code. I can't
imagine anyone interested in stability wants this. There are lots of options
for vendoring your dependencies in tree.

~~~
akerro
> "Go compiler is fast when you add the `-i` flag", ok, do that then

Only useful when you have mutable environment, most build spaces don't have it
because it's insecure. So it's useless for big projects with external
dependencies, you HAVE TO download them on each and every build.

>your build is broken": you only need to depend on external github references

Go projects use not only github repos, there is gopkg, gitlab and some others
which I don't remember. All of them must be online and works fast, any lag
will delay whole build system, which in many cases is pipe-lined. I can't
imagine anyone interested in stability wants this.

------
liveoneggs
The bootstrap problem is real and getting worse: go needs go, haskell needs
haskell, rust needs rust; it's nasty.

There are already "better" and "safer" languages that could easily replace the
entire unix userland. Heck, awk can do half of it, but all of that C code is
already written...

~~~
cesarb
> go needs go, haskell needs haskell, rust needs rust

And C needs C, or for more recent compilers, C++ needs C++. You just don't
notice it because these two languages are already part of the base system.

~~~
progman
> go needs go, haskell needs haskell, rust needs rust

That's painful. It's one big point which Nim [1] does better. It compiles to C
and bootstraps from C. That makes porting to other platforms much easier.

I don't understand why Haskell and Rust don't provide bootstrapping from C.
Did they write the first compilers with Assembler?

> You just don't notice it because these two languages are already part of the
> base system.

C (not C++) has become the essential foundation on _any_ platform. You will
likely not be able to sell your embedded system unless it supports C.

[1] [https://nim-lang.org](https://nim-lang.org)

~~~
steveklabnik
Rust's first compilers were written with OCaml.

We don't provide bootstrapping from C because no C-based toolchain has ever
existed. However, it may in the future; see my link upthread.

~~~
progman
I think bootstrap with OCaml is ok - if just bootstrap from _source_ becomes
possible at all. Binary-only dependency is awful when Rust is wanted on yet
unsupported systems.

~~~
steveklabnik
It's not a significant issue if you can accept cross-compiling. This is how
most platforms get their start.

~~~
npsimons
> This is how most platforms get their start.

The key word here is 'start'. I don't like to arbitrarily constrain tools, but
I've always felt that cross-compilers were sort of a last resort, ie, you
don't have a native compiler ( _you 're_ writing the compiler or operating
system), or are really pressed for time. Once you've got a native compiler,
that should be it: self-hosted from there on.

------
Nokinside
If "safe" language would be real magic pill for safe programming, Ada would
have replaced C and C++ in systems programming and safety critical
applications long time ago.

~~~
agumonkey
I believe Ada suffered from its DoD origins, from the era of giant monolithic
project management and its verbosity.

Rust is a different kind.

Also, I've seen people doing generic low level network stacks (or at least
articles about it) in Ada, seems lovely.

~~~
Nokinside
Ada is extremely well designed language. Verbosity designed for better safety
trough readability and there is science backing it up.

------
fsloth
A bit incendiary:

I fail to see what advantage rewriting existing and proven tools with a new
language would bring. Shouldn't the main value new tools bring to be enable
writing of new things?

Isn't focusing on existing utils more like a lack of imagination and OCD on
optimizing a thing beyond any further value?

~~~
cesarb
> I fail to see what advantage rewriting existing and proven tools with a new
> language would bring.

This is not the first rewrite, and won't be the last one. Earlier environments
were very memory-constrained, which led to optimizing for memory usage; a
rewrite with less memory constraints can focus on speed (as mentioned in the
GNU coding standards: "For example, Unix utilities were generally optimized to
minimize memory use; if you go for speed instead, your program will be very
different. [...]").

The current challenge is the "end of Moore's law" leading to an increasing use
of multiple cores, instead of faster cores. Developers will have to focus on
parallelism instead of raw speed, and new languages can help.

> Shouldn't the main value new tools bring to be enable writing of new things?

Or writing old things in a new way.

> Isn't focusing on existing utils more like a lack of imagination and OCD on
> optimizing a thing beyond any further value?

These tools are the base over which your system is built (their GNU version
isn't named "coreutils" for nothing). Focusing more effort on them makes
sense.

~~~
bad_user
> _The current challenge is the "end of Moore's law" leading to an increasing
> use of multiple cores, instead of faster cores. Developers will have to
> focus on parallelism instead of raw speed, and new languages can help._

Moore's law is indeed nearing its end, but this also ends the multiple core
trend.

If you're betting on multiple CPU cores taking off more than they did already,
don't.

------
shmerl
_> I wasn't implying. I was stating a fact. There has been no attempt to move
the smallest parts of the ecosystem, to provide replacements for base POSIX
utilities. _

Apparently Redox developers disagree :) [https://github.com/redox-
os/coreutils](https://github.com/redox-os/coreutils)

These are even based on BSD coreutils.

------
AtticusTheGreat
It would be interesting to try to cobble together a Linux distribution with a
rust userland, using coreutils
([https://github.com/uutils/coreutils](https://github.com/uutils/coreutils))
and over time building more and more of the userland/interface in rust.

~~~
cat199
whyy couldn't they pick a different name for this project?

rustutils? coreutils-rust?

name collision like this is how we have millions of people running around
thinking that vim is vi while they bash emacs for having bloat as compared to
the 'minimalism' of 'vi'..

------
mjl-
i also saw the same eurobsdcon youtube video. i agree it is somewhat hard to
imagine replacing long-time unix tools with newer versions that only differ in
being implemented in a new language. making sure nothing breaks takes so much
time, with not enough reward.

but writing new tools in safer languages makes more sense to me.

yes, you would need the toolchain in base. not easy, but should be possible. i
only know go. perhaps it's still too much of a moving target for openbsd.

you could write programs in new safe language and put them in openbsd ports.
problem is that it then isn't a "part of openbsd". do openbsd developers
really want to write their next daemon in c? for how long will they stick to
c?

------
jackpot51
Theo is completely ignorant here. I also saw him spout similar inaccuracies in
this video:
[https://youtu.be/fYgG0ds2_UQ?t=2112](https://youtu.be/fYgG0ds2_UQ?t=2112)

Uutils and Redox are setting out to provide POSIX compatible coreutils, and
Redox builds from scratch in less than 30 minutes.

~~~
blub
You might want to add a disclaimer or at least a note in your profile that
you're the same jackpot51 which is the creator of Redox.

~~~
jackpot51
Sorry, I did not mean to be secretive.

------
w8rbt
Someone should start by writing all the setuid apps in Go. OpenBSD would
probably be more open to Go than Rust since Go is a lot like C with a Garbage
Collector. They love C.

~~~
anthk
They already have pledge(4) with C.

------
Polyisoprene
Seems most os developers don’t care about the users and only about their
maintenance and adoption cost.

I WANT my computer and files protected by software written in a safe way using
safe languages. I don’t care about the compile time of grep, I stopped caring
about that kind of dick measurement competitions a long time ago.

Give me a safe, open operating system and I’m willing to trade Posix
compatibility and compile speed.

~~~
dbolgheroni
Curious, but this kind of shitty attitude won't work with OpenBSD nor with any
other open source that I know. It is not about what _you_ care, but about what
_they_ care. Like if they own you something.

That being said, I wonder in what safe language the OS you used to write this
is written.

~~~
Polyisoprene
I wouldn't call protecting my computer and data being shitty attitude. OS
developers need to realize that secure is not something they can compromise on
going forward.

------
nanolith
I have spent a long time using alternative languages. I'm a compiler geek.
However, most of these languages ultimately lead to disappointment. Not
because they necessarily fail, but rather because the problem of writing
software is not a problem of language or platform. It is a problem of thought
and abstraction. So, the big promises that these new languages and platforms
make will ultimately only be fulfilled for a subset of applications. The rest
of the time, we must still trudge through the process of good design, good
testing, and good real-world feedback.

People love to beat up on C, because it does not have feature X or feature Y,
and because there is plenty of bad C software to point at. However, it isn't
the language that leads to bad software. Any entrenched popular language has
plenty of bad software examples, and plenty of security vulnerabilities. The
problem isn't the language, but rather, the developer. A good developer who
understands and appreciates the problem can write good software in machine
code, C, C++, Rust, Java, Haskell, C#, Go, D, or any other language or
platform. A good developer who understands secure programming processes can
write secure software in any of these languages, given enough time.

What these higher-level languages offer is better abstraction, which can make
time-to-market faster. However, in a deeply entrenched system that is already
in market, maintaining the current languages and platforms is typically the
better play. This is especially true when the source code is as meticulously
maintained as it is with OpenBSD. Someone who understands the basics of C can
easily read the OpenBSD kernel or userland, and understand everything about
how a given program works. Some knowledge of BSD Make, Bourne shell, Korn
shell, and Perl may be required to understand the init scripts and build
process, but in all, it is a system that is easy to understand, easy to
maintain, and with an excellent development process in place to deliver two
builds a year. OpenBSD isn't broke, and a new language isn't going to fix it.

That being said, there are tools that the OpenBSD team could use that neither
impact the build time nor require adopting fad language of the week. There are
excellent model checkers out there for C, and quite a few proof assistants are
gaining the ability to check proofs on C code directly, using Separation Logic
and Hoare triplets. These tools can complement an entrenched C system by
providing an external mechanism of verification that allows designers to
formally verify that implementation meets specification. While formally
verifying all of OpenBSD would be a tall order, taking hundreds of man years,
formally verifying critical pieces of the kernel and creating simplified
contract boundaries for the remaining code would be an excellent complement to
the code review process that the OpenBSD team already does. Better still, it
is something that could pay off without reinventing OpenBSD to match the
current fad, as such verification can be run independently of the standard
build process. If I were to make a suggestion to the OpenBSD team, it would be
to explore such tools. But, to de Raadt's point, such a suggestion would only
be valid if someone did the heavy lifting to put such a tool in place, and if
it were of utility to the team. They won't adopt something for the sake of
that thing, but they are keen for anything that can practically improve the
security of their system.

~~~
nickpsecurity
You're missing high-level languages such as Modula-2 or Free Pascal that are
safe-by-default while being close to the machine. They compile faster. They're
easier to analyze. Someone who understands C could easily port it to one of
these languages or even write a translator that converts them to C for easy
integration of incrementally-rewritten pieces of the OS. They don't because
they like C for non-technical reasons or want to do one piece of tooling work
over another.

Another you might find interesting in alternate history where C programmers
adopt better tech is Cyclone that tried to stick to C as much as possible.
Rust's safety scheme took a lot of inspiration from Cyclone.

[https://en.wikipedia.org/wiki/Cyclone_(programming_language)](https://en.wikipedia.org/wiki/Cyclone_\(programming_language\))

[http://trevorjim.com/unfrozen-cyclone/](http://trevorjim.com/unfrozen-
cyclone/)

" There are excellent model checkers out there for C, and quite a few proof
assistants are gaining the ability to check proofs on C code directly"

I agree. Further, just using Design-by-Contract with property-based and fuzz
testing on those contracts would improve things by itself. There was also an
academic who applied Frama-C to one of their smallest, battle-tested
components in I think a string library. Although no coding flaw found, the
exercise did show the documentation was incorrect. Gotta wonder what benefit
might happen for less obvious stuff than string operations.

~~~
nanolith
Cyclone is an interesting example. Something that is backward-compatible with
C and does not incur significant compile-time overhead, like C++ does, is a
potential alternative. Unfortunately, Cyclone was abandoned. It only has
support for 32-bit architectures, and it is not being actively maintained. One
of the biggest reasons why C is so popular is that it works on practically
every platform, even those that Cyclone, Modula-2, and Free Pascal do not.

Of course, if OpenBSD eliminates support for a bunch of the platforms it
currently supports, such alternative languages become viable. However, the
OpenBSD team purposefully maintains these platforms because they expose errors
that would otherwise be missed by a scaled-down release. Software errors
missed in x86_64 or x86 may be picked up in MIPS or SPARC because of
endianness.

An alternative that I think is more tenable is a language that compiles to C.
Such a language can piggy-back off of C's incredible market penetration while
adding safety features. Barring some extensions to Cyclone, most of Cyclone's
features, for instance, could be implemented as a source-source compiler that
targets ANSI C99 or ANSI C11.

~~~
nickpsecurity
That's actually what I said here in my more thorough comment below plus some
of other replies:

[https://lobste.rs/s/4cf21p/re_integrating_safe_languages_int...](https://lobste.rs/s/4cf21p/re_integrating_safe_languages_into#c_4ch7ug)

Ill be replying to followups on that later tonight or in the morning. I
totally agree it should be default. I'll go further to say it should have C
datatypes/sizes, calling convention, FFI with little to no annotations
required, and compile to C. Each check should be removable per midule with a
compiler option, pragma per module, or equivalent keyword/operator marked
unsafe . It also needs inline ASM. For its own advantages, add macros, safe
linking, and a REPL with incremental compilation.

That by itself would be better and faster to develop than with C with seemless
integration with legacy codebases. Side benefit like in my Brute-Force
Assurance method would be it benefiting from all tooling C ecosystem has to
offer esp static analysis and certifying compiler.

What you think on that?

------
andrewmcwatters
The problem "safe" languages have in this context is that they really truly
are no better than C, for the simple sake of them throwing out performance and
resource considerations from the get-go. Additionally, if you take even just a
day to read up on UNIX and C history, you'd quickly find the two are
essentially intertwined, and that to usurp C, you'd have to nix UNIX, too.

I've yet to see anyone with serious industry experience come out and say sane
things about replacing C. It's just not going to happen. Not in my lifetime,
unless someone decides to write a new operating system, and Linux subsequently
loses footing.

------
Annatar
Bravo Theo! Someone finally stood up to hype!

------
jstewartmobile
This moment should be in CS textbooks. Want something?

Wrong way: write an e-mail

Right way: Here is my feature-per-feature, bug-per-bug rewrite of grep in Rust

------
JetSetWilly
> So rather than bothering to begin, you wrote an email.

This is where Theo appears to me to be quite different to eg Linus. He plays
the man, not the ball, with personal attacks.

~~~
jackpot51
I believe that Linus's attacks are usually good intentioned, to get people in
the Linux community to act better. I am not sure what Theo's intent is.

~~~
ams6110
If you want to effect change, do some work, don't write emails.

~~~
cuckcuckspruce
As can be expected from a project with a slogan of "Shut up and hack"[1].

[1]
[https://www.openbsd.org/lyrics.html#51b](https://www.openbsd.org/lyrics.html#51b)

As the notes indicate, this saying goes back much further than this release
song.

