

Ask HN: Do you understand your operating system? - s2r2

Dear HN readers,<p>this is something that bugs me for quite some time, therefore I decided to ask around:<p>Do you feel like you understand your OS?  Have you really have acquired the feeling of trust and mastering its (inner) complexities?  And, most obvious: do you care about this?<p>This question affects free OSes a lot more than proprietary ones, obviously, so if you care, it'd be great to hear from your experienced feelings corresponding to the OS you use or used, as well as the depth your technical understanding had to reach in order to gain the experience of 'groking it'.<p>I hope that's not too vague, maybe it helps to tell you that this question popped into my mind when I thought about switching from GNU/Linux to *BSD once again, this time hoping to leave some mysteries behind.
======
starev
It's hard to say what it is to "understand" an operating system, though.
Knowing how it works in a fairly broad sense, or even knowing how specific
parts of the kernel work doesn't mean you completely grok the system.

I mean, understand in the broad sense of "stuff all OSes do": certainly. I
realize that there is memory management and etc. and etc. that is going on,
and I have a basic idea of how that works, at least on Windows. (I use
Windows, Linux, and Mac regularly, and can reach each from where I'm sitting.)

I have a decent enough understanding of Windows system internals and how it
actually handles memory allocation, permissions, etc., but not down to the
lowest level, just sort of a vague idea of what system files do what.

I assume Linux and OSX handle things somewhat similarly aside from
implementation details, but I don't really know.

I haven't had the time to poke around Linux kernel internals (I have a pretty
good understanding of the system down to that level, though), and I doubt I'll
ever mess with OSX internals beyond what is necessary to have it 'just work'.

I figure I will master what I need to get things to do what I want. I find it
fascinating, and at some point I'll probably have a project which requires me
to hack on the Linux kernel in some way, shape, or form (unless someone else
already has). In the meantime, though, I remain knowledgeably ignorant (I know
that I do not know), which I think will have to do for now.

BIOS-level stuff and all of that is black magic to me. :P

It's hard to say what is and isn't necessary just from a development
standpoint. From a curiosity standpoint, assuming unlimited time, complete
knowledge of a system is better than limited knowledge. From a practical
standpoint, such knowledge may make you a 'better' coder, but it may also tie
you more to one OS rather than another. It cuts both ways there, though,
because it could be that by understanding what does and doesn't change from
one system to another, writing portable code is much easier.

So do I care that I don't know? Yes. Do I think it's the end of the world and
must be remedied immediately? No. I will have plenty of opportunity to remedy
my desire to know more about OS internals if I ever take on a project that
requires me to know them exceptionally well. At some point that will
inevitably happen, considering my interest in them. In the meantime, I'm
content to know that I do not know and that in an ideal future, someday I will
know.

------
pstevensza
I thought I knew FreeBSD until I joined a couple of the lists and realised
that I've hardly scratched the surface. It used to bother me that I didn't
know. These days, I can accept it, and enjoy my occasional dip into the
unknown when I need to change something.

~~~
ovi256

      These days, I can accept it
    

And one day, you buy a Mac. Trust me. You want to.

------
radu_floricica
There is at least one cognitive bias involved when answering this questions.
The less you know about something, the simpler you believe it to be. I'd
expect Torvalds to say he has a "fair" understanding of linux.

------
synnik
Or, at an even lower level... do you understand your hardware? Not just disk
space and memory, as most programmers need to know those... But do you know
how the keyboard works? USB? How the processor receives/manages all the
signals?

I did know all this, 10 years ago... I haven't kept up with newer systems, but
even that basic understanding helps in many subtle ways.

~~~
ErrantX
Slightly as an aside but one source of amusement for me is programmers or
sysadmins who cant tell you the difference between a serial and parallel comms
link (beyond the obvious) and how one might work.

"Explain how a USB keyboard interacts with a computer" is one of my favourite
interview questions at the moment. A surprising number of people can't tell
you an in depth answer and you either hear:

\- it's serial err no idea beyond that

\- serial connection, I guess.. <creative attempt at answer>

\- random bullshit about bits and serial (they can waffle for hours)

Really good for seeing them come up with creative solutions. Couple it with a
whiteboard and work through the electronics with them and you can have a great
interview.

(I started to do the whiteboard thing after one interviewee asked for a
pen/paper to try and sketch out what he was thinking)

~~~
ern
Do the sysadmins and programmers you are interviewing need to understand the
innards of a USB keyboard as part of their job function? If it's not part of
their job description, then I am not surprised at all.

~~~
ErrantX
nope, and their answer is fairly irrelevant.

If they have the balls to say "no idea, why do I need to know" that is someone
we like. If they say that and then attempt to work out an answer extra bonus
points. If they stumble they get no set back - it is a random question. If
they bullshit they are marked down a fair way - not the kind of person we
want.

(we design it to be a question you have to actively fuck up to be marked down
on)

We sit and work through an answer together and you get a real insight into
them outside their area of expertise (where you would expect them to be slick)

------
kbob
There are some things I understand. I've written parts of Unix/Linux virtual
memory, network stacks, process management, threads, interrupt handlers,
synchronization primitives, TTY line disciplines, various kinds of device
drivers. I've debugged parts of block I/O systems, file systems, NFS client
and server, CPU schedulers. I feel like I understand those.

There are things I don't understand. I don't know the I/O architecture of a PC
nor how the OS integrates with the BIOS. I don't know how the subdirectories
of linux/drivers/ are organized, even at the first level. I've never designed
a filesystem, and suspect there are subtleties I don't appreciate.

I never bought Bill Gates' testimony during US vs Microsoft that an operating
system includes its web browser. In my arrogant opinion, an OS starts and ends
with the kernel. So the next question is, how much do I understand about the
rest of the platform? It varies.

I do know how the X11 protocol works, how the classic X server worked, how Xt
intrinsics work. Do understand the classic Unix utilities and libc and their
modern GNU reimplementations.

Don't know how an RDBMS works inside. Haven't kept up to date with compiler
optimizations. Don't know how debug info is stored in object file and
executables or even exactly what it is. Don't have any idea how that huge pile
of libraries in GNOME (or KDE) is organized. Have only a vague idea of how the
current X server is organized.

Even though the last (and first) time I implemented a page fault handler was
1986, I feel like I understand what they have to do, why, and what data
structures they need. I'm sure the specifics are very different in any of
today's OSes than they were in 4.3BSD, but I don't think that matters. OTOH, I
think most of my graphics knowledge is obsolete.

So, in sum, I partly understand my OS. (-:

------
aurora72
You miss something the author's talked about and it's the transition from
GNU/Linux to _BSD

I assume he/she wants to have a more understandable OS; and because the _BSD's
excel at that area, I guess she/he's referring to the _BSD's.

Windows --> Great but I still cannot understand some of its peculiarities such
as unexpected crashes, sudden explorer.exe restarts.

MAC OS --> Great but I still cannot understand just why its USB connection
speeds are interestingly low or why it has such an inadequate Unicode support
for different FileSystems like NTFS.

Linux --> All great, but I cannot figure out why there are so many different
package managers out there?

As for _BSD, because it's structure is so well-founded there's nothing which
is not understandable. Everything's got its place where you'd expect them:
Ports collection under /usr/ports, and package manager is just the pkg_add
command. There are no window-manager or desktop-environments pre-loaded and
configured, you just chose what to do. And the kernel structure is extremely
simple: You can add extra functionality to kernel easily with the Loadable
Kernel Module (KLM) concept. That's it, everthing's under control.

------
pmjordan
I'm currently working on a block device driver, which means I'm digging around
in kernels. I've found the O'Reilly books on the Linux kernel ("
_Understanding the Linux Kernel_ , 3rd ed" and " _Linux Device Drivers_ , 3rd
ed") as well as " _Mac OS X Internals_ " by Amit Singh extremely helpful. The
latter doesn't only cover the darwin/xnu kernel but also a lot of the OS X
user space. All of them are quite technical and can be a bit dry, but they're
certainly densely packed with information - and you can safely skip/skim most
sections if you like. _Linux Device Drivers_ is probably only useful if you're
actually trying to write a kernel driver, I can recommend the others for
general interest as well, though.

I've also got _Windows Internals_ from MS Press, but I've only looked at it
briefly so far. It seems higher-level than even the OSX book.

In general, I've found it extremely interesting to see how it all fits
together, but I've got a general soft-spot for low-level hackery. (until
recently that mainly involved game consoles)

------
rbanffy
I care and I think I pretty much get what's going on under the hood of my
Linux boxes as they seldom surprises me. OTOH, it's a complex system, with
stack upon stack of hardware, firmware and software that makes really groking
it (as I did with my Apple IIs in the early 80's) next to impossible. Even
before the OS loads, an x86 PC is a very complicated animal.

As for the Windows notebook I am typing this in, it never ceases to amaze me
and surprise me in almost always frustrating ways. It's indeed a box full of
magic, but I am not sure its genies always work for me rather than against me.

I have tried a couple times to move to FreeBSD, but I always got frustrated
with, what I perceived, a purist view of what a Unix machine should be -
clean, simple and somewhat bare. I kind of like the magical things that happen
in Linux, particularly package management. I utterly dislike the way it
handles wireless networking, but I guess this is a feature nobody got right.

------
jrockway
I do understand my OS, but it's probably better to not think about it. Knowing
too much about UNIX leads you to want to throw it away and write something
sane... but that is time consuming. Better to close your eyes and not think
about the details.

------
ErrantX
Windows: yes Linux: pretty much (90%) Mac: nope

When I started my current job in computer forensics and security it shocked me
how much I didnt know. There is a LOT of stuff going on in the internals that
is pretty smart.

Believe me: you have no idea how bloody complicated a discussion of the
Windows file system (a fairly simple high level subject right? :P) can become
when you put a couple of forensic guys in a room :)

EDIT: I like a lot of the ways the Linux device/file system is organised. It
makes a little more logical sense (though it might just be that it is more
accessible) than windows.

------
hdx
Getting there with a little help from my friends:
<http://inst.eecs.berkeley.edu/~cs162/sp10/>

------
dirtbox
I've got a pretty indepth knowledge of every OS I've used for any length of
time. My first instinct with all things is to unscrew the lid and find out
what makes it tick and for me, operating systems practically beg to be
tinkered with,

------
javery
I knew Windows inside and out, used it from 3.1 and could do just about
anything I wanted to do and fix just about any issue. 3 years ago I moved to
Mac OS and I am still working on getting myself to the same place.

------
ARR
Windows: never tried to Linux: I know quite a bit but the important thing is
that I know that I will understand everything I don't know. Learn by
experience.

------
simonw
I don't at all, but I reall wish I did. The more experienced I get as a web
developer the more the low level stuff matters to me.

------
zackattack
i run mac os x. no i don't understand its internals. i got it cuz it "just
works"!

