
Reading OpenBSD source code daily - mulander
https://blog.tintagel.pl/2017/06/09/openbsd-daily.html
======
sqs
Completely agree. But the tools don't make this very easy.

Back in college I was working on patches to OpenSSL, Chrome, Firefox, Apache,
etc., to add support for TLS-SRP, and it was a huge pain to jump into these
massive codebases and try to understand them. I was using Emacs and had all of
the various language support modes configured, but go-to-definition and cross-
references barely worked. Searching was slow, and if I wanted to discuss a
piece of code with my CS lab partners, I couldn't just share a link.

A friend felt the same pain but then went to work at Google for a bit. At
Google, they have some pretty amazing code reading/searching tools (see
[https://static.googleusercontent.com/media/research.google.c...](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43835.pdf)),
and these tools helped Google build a culture of thoroughly reading and
reviewing code. The causality is bidirectional, but having good tools
certainly played a role in Google's success.

That friend and I ended up building a product, Sourcegraph, initially for
ourselves to make code reading easier. We've now built a successful business
out of it with the help of an amazing team. Here it is pulling in the OpenBSD
sources:
[https://sourcegraph.com/github.com/openbsd/src/-/blob/lib/li...](https://sourcegraph.com/github.com/openbsd/src/-/blob/lib/libutil/bcrypt_pbkdf.c?q=bcrypt_pbkdf#L98-98:13).
Sourcegraph has advanced features for several languages; see
[https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyh...](https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyhttp/httpserver/https.go#L22:23$references),
for example. If you love to read code (or want to), we hope you'll love our
product. Email me if you have any feedback/requests.

~~~
problems
There's a tool called OpenGrok that pretty much does this except it's free and
open source and it works on C code.

Someone has it run on the openbsd code here:
[http://bxr.su/OpenBSD/](http://bxr.su/OpenBSD/) and it should produce a much
more useful representation of the code, see
[http://bxr.su/OpenBSD/lib/libutil/bcrypt_pbkdf.c#98](http://bxr.su/OpenBSD/lib/libutil/bcrypt_pbkdf.c#98)

Mozilla also has one called DXR which is designed for their large, C++ heavy
codebases: [https://wiki.mozilla.org/DXR](https://wiki.mozilla.org/DXR)

~~~
kbenson
On first impressions, since I have no experience with OpenGrok nor
Sourcegraph, they look to serve about the same core need and provide most the
same core functionality, but Sourcegraph is about what I would expect from a
company providing usability and features on top of what is generally available
for free.

That is, Sourcegraph looks to compare to OpenGrok like Github compares to
Gitweb. At least from a cursory look.

~~~
problems
Considering that Sourcegraph doesn't support C or C++ and those are pretty
much the only languages I write that have less than perfect IDE crossrefence
support I don't really see a point in Sourcegraph at the moment.

~~~
kbenson
Didn't one of the sourcegraph founders post a link to OpenBSD in sourcegraph
at the top of this thread? That's C. Am I misunderstanding what you're taking
about?

~~~
floatboth
"C/C++ is supported for text/regexp search and basic browsing (no advanced
language features yet)." (message in the Sourcegraph UI)

------
peatmoss
I love this idea in part because it's the very opposite of the way I tend to
work, which is to drive very hard to get a surface understanding of a thing in
order to make a very targeted change. I learn lots along the way with this
approach, but don't often get the deep, wholistic understanding of existing
systems that only comes with repeated exposure over a long time.

Some kinds of understanding involve a no shortcuts grind. That sort of a grind
is a big commitment though.

~~~
s_kilk
> but don't often get the deep, wholistic understanding

*holistic

~~~
peatmoss
How embarrassing. That's what I get for thumb-typing a comment on the bus
before I've had my cofveve.

------
kfrzcode
I've been using Typing.io as a platform for reading source code (working my
way through Gitlab now) and practicing typing with the right fingers. I have a
few minor bad habits to correct, and I want to familiarize myself with the
codebase, it's a good way to warm up for the day.

------
VMG
Great idea. What other code bases are there that lend themselves to this?

Some kind of curated genius.com for source code would be interesting.

~~~
thomas11
The Go standard library is very well written and depending on which parts you
read you can learn about lots of things like file operations, HTTP, crypto,
etc.

It's easy to read it all on the web, the docs are here:
[https://golang.org/pkg/](https://golang.org/pkg/) and clicking on a function
name shows the source.

~~~
kampsy
I totally agree with you. The Go standard library's is fun to read. I end up
opening multiple tabs of different packages. Lots of fun.

------
aomix
I've fallen into doing something similar. I read the mailing lists regularly
try to look over the source for something that gets a proposed patch. Because
OpenBSD boils down their software to the essentials and tries to make their
APIs impossible to misuse I find it pretty easy reading even though I'm not
very experienced with C.

------
brynet
Here's the latest daily chat transcript, from Jun 9th: The topic was OpenBSD
nc(1) and libtls, but it wandered over to pledge(2) and other code fixes from
new participants eager to contribute.

[https://junk.tintagel.pl/openbsd-daily-
nc.txt](https://junk.tintagel.pl/openbsd-daily-nc.txt)

~~~
andrestc
May i suggest maintaining a repo on github with such material? Would be easier
to keep up and might bring in some contributions

~~~
brynet
It might be worth reaching out to mulander, he already hosts his blog on
github.

[https://github.com/mulander/blog](https://github.com/mulander/blog)

------
topspin
Anyone have a good exemplar React application for this purpose? I'm building
small experimental stuff and now I need to level up and see how real
applications are being built.

------
sn41
I read the emacs lisp source code in site-lisp/ every day. The goal is to
understand one file about a week or so. It has made my emacs and lisp
knowledge better, and made me aware of several nice emacs features (align-
regexp, for example).

------
ianai
I wonder where best to start for people who haven't taken an OS class before?

~~~
irundebian
The best start would be to read a book about operating systems such as
Tanenbaums. Reading the source of an OS has a really really low signal to
noise ratio in getting important knowledge about operating systems due to
implementation details an OS-specific peculiarities.

~~~
klez
What about MINIX? Being a teaching operating system, do you think it may
contain less nose and thus be more useful as reading material? (I mean,
without the book)

~~~
nickpsecurity
Linus Torvalds started out studying MINIX. He was a Finnish programmer into
working on kernels. He ended up completing one.

------
carlmungz
I started doing this the other day for a JS framework I'm using. Cannot
recommend this practice highly enough. You learn so much.

~~~
jamie__k
That sounds great idea. I thinking about reading the vuejs code. if any tips
for reading, can you share it?

~~~
carlmungz
This Github repo has some very good tips on how to do it:
[https://github.com/aredridel/how-to-read-
code/blob/master/ho...](https://github.com/aredridel/how-to-read-
code/blob/master/how-to-read-code.md)

------
z3t4
What I love about the web is that you can just right click and view source.
See something cool ? Just click and see how they did it.

~~~
woranl
Soon, you won't be able to if WebAssembly becomes popular.

------
err4nt
This is fantastic! I've recently decided to begin reading web browser source
code, even though I understand very little of it at the moment.

For now, what's been fun is to load up the same file in both Chromium and
Firefox source, and compare the two and how both browsers work.

Chromium source:
[https://cs.chromium.org/chromium/src/third_party/WebKit/Sour...](https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/)

Firefox source: [https://dxr.mozilla.org/mozilla-
central/source/](https://dxr.mozilla.org/mozilla-central/source/)

~~~
FreeFull
Could also be fun to compare to the Servo source:
[https://github.com/servo/servo](https://github.com/servo/servo)

~~~
rhelmer
Servo source is also on DXR, which is a lot nicer than the github source
viewer since it understands Rust (and other) languages (using an LLVM compiler
plugin):
[https://dxr.mozilla.org/servo/source/](https://dxr.mozilla.org/servo/source/)

------
irundebian
How about using a static security source code analyzer and going through all
findings? The very good ones, the commercial ones, are free to use for open
source projects. That would be real benefit to the project I think.

~~~
masklinn
It's missing the point entirely, their primary goal is to become better
developer and to improve their C knowledge, hence the effort being mostly
focused on _reading and understanding_ a fair amount of code, including its
context. Fixes and improvements are positive side-effects of the original
effort, not goals in and of themselves.

~~~
irundebian
If you want to improve your C knowledge there are probably more efficient ways
to do it instead of randomly reading OpenBSD sources such as reading more
advanced C books or reading source codes of projects which are known for their
good code quality (sqlite maybe?).

One of problem of static source code analyzers are false positive. Soon or
later you will have to reading code and understand the context. I assume it's
better way to improve your C knowledge because you REALLY must understand the
code . And besides that the positive effect are more valuable.

~~~
masklinn
> more efficient ways to do it instead of _randomly reading OpenBSD sources_
> such as reading source codes of projects which are _known for their good
> code quality_

Yes they could be reading the source of projects known for their code quality…

> OpenSSL is not an OpenBSD project and the code quality is markedly different
> :-)[…] and yes, OpenSSL is a bit of a code quality difference than the
> OpenBSD norm. [nb: these comments were not praising OpenSSL's code quality]

> OpenBSD has proven great at configuration, code quality, and minimalism.

> OpenBSD's incredible code quality quite obviously doesn't apply to the ports
> tree (and that's not their fault)

> OpenBSD […] has a slower evolution pace and a more carefully planned
> development model which leads to better code quality overall. Its well
> deserved reputation of being an ultra secure operating system is the
> byproduct of a no compromise attitude valuing simplicity, correctness, and
> most importantly proactivity. OpenBSD also deletes code, a lot of code.

> After scouring the lists and other resources I've yet to find an official
> reason for OpenBSD dropping LKM support, but would wager it's due to
> security or code quality/openness ideals.

> OpenBSD, a project that has a frankly psychotic focus on code quality. […]
> some examples of great code quality. OpenBSD is undoubtedly one of the pin-
> up projects of the Open Source world, featuring code that is almost
> supernaturally clean, consistent and direct.

> SELinux, etc. is not that picky about audits and code quality as OpenBSD is.

> “I think our code quality is higher, just because that’s really a big focus
> for us,” De Raadt says.

such as OpenBSD.

~~~
irundebian
Ok. Could you give me the sources of these quotes? Or other reputable sources
and papers who claim that the code quality of OpenBSD is high?

~~~
doody12
Code quality is hard to measure, but I think that any competent C programmer
that reads the source code for OpenBSD will agree that the quality is above
average.

OpenBSD is a fork of NetBSD, another project considered to have above average
quality source code. Enough so that Spinellis based his book about code
reading on the NetBSD source code.

------
lbill
This is a smart thing to do! I might do it eventually... not right now though,
because of <insert whatever reason you can think of here>, and stuff, you
know.

