

Introducing Kirk - IanChiles
http://ian.roon.io/introducing-kirk

======
blueblob
I take quite a big issue with the statement "The lack of information and
documentation on developing OSes is, in my opinion, a pretty big problem"

There are entire forums for this, ie osdev
[http://wiki.osdev.org/Expanded_Main_Page](http://wiki.osdev.org/Expanded_Main_Page)

There are entire books for this
[http://www.amazon.com/s/ref=sr_ex_n_1?rh=n%3A283155%2Cn%3A5%...](http://www.amazon.com/s/ref=sr_ex_n_1?rh=n%3A283155%2Cn%3A5%2Ck%3Aoperating+systems&bbn=5&keywords=operating+systems&ie=UTF8&qid=1379950869)
amazon lists about 57,000 results for operating systems books

You will get out of LFS what you put into it. It tells you how to download the
source, extract and install it. When you have the source, nothing is
preventing you from looking at it.

I also take issue with "how can we hope to continue developing and innovating
on the OS level when it’s practically impossible to figure out what anything
does or how it works."

There are general concepts that you will learn from getting a book on os
development (including the kernel) and writing a minimal kernel yourself will
give you insights into the work that goes into a full kernel such as the linux
one. The concepts, not the specifics are what you need to learn, and you do
this through theory, you pick up the rest from implementation (but starting
with a kernel, half of the os is already developed fof you).

~~~
IanChiles
Absolutely agree with you. I struggled with wording this for the longest time
- and when I talk about OS development, I'm referring much more to the
userland development - package managers, core utilities, shells, init systems,
and the like, than I'm referring to kernels. Sorry for the confusion.

~~~
blueblob
I think some of the less explored portions of LFS contain well-documented
reasoning for implementing some of these things yourself. For example there is
a single solution (out of the many ways to solve this problem) to the package
manager dilemma in the hints section of LFS

[http://www.linuxfromscratch.org/hints/downloads/files/packag...](http://www.linuxfromscratch.org/hints/downloads/files/package_management_using_trip.txt)

which explains reasoning for the package manager and how to implement it
yourself (in a much less copy-compile-execute manner than LFS itself).

Implementation is a great way to learn but generally people will survey the
literature first to understand why things are implemented the way that they
are. Understanding what all package managers have in common will allow you to
distill it down to the simplest factor and implement that.

A lot of the core utilities and why they are expected should be covered in the
standard (POSIX in this case).

I guess what I am trying to say is that implementation is a useful way to
learn but it requires you to know the details whereas understanding the
literature first will give you a better idea of what to implement.

Good luck with your project.

------
runjake

      For as long as I can remember, I’ve wanted to create my
      very own operating system, one which I have complete and 
      total control over. Because I have neither the time, nor 
      the skill to write my own kernel, this means that I’d 
      eventually land up creating a OS based on the Linux 
      Kernel.
    

Non sequitur. If you want complete and total control, you're going to have to
learn the ins and outs of a kernel.

I applaud your decision to go forth with this, but is it really worthy of an
HN self-post given that Kirk does not exist and is just a blog post at this
point?

~~~
IanChiles
For me, at this moment, it's simply not a feasible goal to learn the ins and
outs of a kernel in any reasonable amount of time. And Kirk doesn't just exist
as a blog post, I've done a relatively large amount of work on it, but am just
now starting to write about it by request of a friend.

~~~
Patrick_Devine
I think it's a worthwhile endeavour to stick a distro together, however I
agree with a lot of the other commenters here. If you really, truly want to
learn about how an OS works, you're going to need to get your hands dirty and
learn about the kernel. There are so many tradeoffs made on the kernel side of
things which impact userspace, and knowing about them impacts the decisions
you're going to make in terms how how you configure software and which
software you're going to use.

If you can afford it and have a week to spare, I'd really recommend taking the
Linux Foundation's one week crash course on the kernel. It talks about how the
scheduler works, gives you insight into modules, how processes and threads
work and hooks the kernel libraries give you for semaphores, mutexes, etc.
It's expensive, but it's well worth the money.

[http://training.linuxfoundation.org/linux-
courses/developmen...](http://training.linuxfoundation.org/linux-
courses/development-training/linux-kernel-internals-and-debugging)

~~~
jnbiche
I'd actually really love to take a course like this, but a non-profit
foundation charging $2750 for a virtual 4-day course?

I understand that they have probably have access to incredible kernel
expertise, and hopefully the teaching is excellent, but even that doesn't
warrant nearly $3000 for a 4-day course.

I mean, even training companies in the pharmaceutical industry doesn't
typically charge quite that much for short courses! They're probably close,
but is that really who the Linux Foundations wants to model their pricing on?

It's great if they can find corporations that can pay these kinds of prices,
but it means that independent open source developers are virtually shut out,
which means that many of the very people who would be interested in making
contributions to the kernel will not be able to take what is probably the best
introduction around to kernel programming.

Does anyone know of any good alternative resources for learning the kernel?
There's a lot of good stuff on kernelnewbies.org (when it's not offline).
Other ideas?

~~~
Patrick_Devine
Yeah, I know it's expensive. I took the "in person" version of this course
though and it was phenomenal. I'm not sure if it cost the same (the company I
worked for payed for it), but if it is, it works out to $70/hr which is
cheaper than getting specialized instructors in other disciplines like flight
instruction.

For a virtual version of the course $2750 does sound pretty steep. It'd be
nice if there were a "Khan Academy" type version (ie. pre-recorded) for a lot
less.

------
antocv
Kirk does not exist, this is a blog post. Just a reminder to everyone like me
who went looking for it.

And the blog post is about the same dust that was relevant 10 years ago and
sparked the creation of LFS. Nothing new to see here.

------
fusiongyro
> an OS without limits, built to be open from the ground up - with as open of
> a development process as possible.

I see the benefit of documenting each step. The result will be a pretty great
tutorial, or at least a very interesting series of blog posts. But what is
closed about the development of the current distros?

I hope that Ian is hoping to build a very different kind of distro, breaking
the mold. NEXTSTEP was fundamentally a Unix, but everything about it was
synthesized to create a totally new organism. Same with OS X. This kind of
experimentation hasn't been attempted very much with Linux distros, which
mostly amount to different installers and package managers. Etoile comes to
mind, and apparently CoreOS is discarding much of the mold, but a completely
new, holistically built system would be quite interesting to see. I suspect it
hasn't been attempted much because the sheer amount of work is overwhelming.
Writing a new shell is a lot of work. Writing a different init, shell,
filesystem layout, and window manager, all unified around some new concepts,
would be a lot of work. If that's what he's aiming to do, the result will be
really interesting to use and learn from.

~~~
norswap
> Same with OS X

OS X is BSD with a fresh coat of paint. It really depends if you want to
consider the whole experience (which I consider less stellar than most people
would care to admit) or the internals (which are the same old unix stuff).

~~~
mwfunk
Mach and IOKit and and a gazillion other things are hardly "the same old Unix
stuff", if you're talking about the kernel. If you're talking about everything
above that level, read Amit Singh's book if you really think there are just
splashes of superficial changes at and above the POSIX layer. If you have and
you still feel this way, I don't know what to tell you. I'm not saying it's
all great, but there's a LOT there.

------
examancer
I like the concept, but wish this announcement would have come with at least
one or two concrete steps taken towards the stated goal. Maybe your experience
getting a basic shell working or something similar. At this point I have no
reason to believe anything will come of this. Was maybe a bit too early for an
"announcement".

------
v0land
Dreams: "I’ve wanted to create my very own operating system, one which I have
complete and total control over."

Reality: Another Debian clone out there.

~~~
IanChiles
I assure you that I won't do that. If I wanted to make a Debian clone I
wouldn't be doing this

------
olalonde
> the development process of Kirk be accessible enough for anyone interested
> to jump right in and contribute

Would be a good idea to host it on Github.

~~~
IanChiles
Absolutely 100% will be, don't worry

------
jff
I want to echo the recommendations of others: learning about kernels and
writing your own would be a lot more valuable.

I'm speaking from experience when I say that trying to provide a userland
sucks. You've said elsewhere on this page that you want to write your own
shell and such; the problem is that people don't want IanSH, they want bash or
zsh or whatever. If you write IanMACS, they'll ask how to run emacs. And I
hope the libraries you write can compile Firefox.

Or if you just want to write init replacements, modify how configuration is
managed... remember that Ubuntu (and Gentoo, but nobody cares about Gentoo
anymore) did a ton of this and everyone I've talked to hates it.

So here's my concerns:

\- If you intend to write an entirely new userland, you're going to spend a
lot of time replicating existing work, only to have people complain about it.

\- If you intend to do a new init, improve configuration management, change
how logging works, whatever, your life will be consumed with modifying every
program to use your stuff.

If you want to learn about operating systems, I'll echo the others and say
write your own kernel and a minimal userland.

------
Skoofoo
The open source desktop is kept alive mostly through donated spare time. This
time is limited. We would do well to maximize the impact of time spent
creating and maintaining open source desktop software.

Right now the standard language to build Linux software is C, which is great
for performance, but not great for development speed. There are ports of
things like GTK+ to other languages, but ports are less supported and usually
don't take full advantage of their language's unique flavor.

To maximize impact per unit of time spent developing open source software, and
also perhaps to entice more people into contributing to open source software,
I believe it would be a good idea to build a desktop environment/widget
toolkit with an opinionated API in Ruby. Anyone who has experience writing
Ruby on Rails code with an automated red-green-refactor cycle likely could
testify that it is pleasant and fast. Why not bring that sort of experience to
building desktop software?

~~~
IanChiles
I considered Ruby and have worked heavily with it in the past, but it's simply
too slow/heavy for this type of programming.

I can't believe I left this out of the post, but all/most new code will be
written in Go - I'll explain why more clearly in my next post

------
clarkm
If you're interested in how a simple version of unix works, I highly recommend
checking out Xv6, a teaching OS based on UNIX version 6:

[https://pdos.csail.mit.edu/6.828/2012/xv6/book-
rev7.pdf](https://pdos.csail.mit.edu/6.828/2012/xv6/book-rev7.pdf)

------
ivanbrussik
I applaud his efforts - but how is it so many people find main stream Linux
distros that non-configurable that they _need_ to create their own?

I mean, even distros like Ubuntu are highly configurable. You can kill off as
many packages as you want and even compile your own kernel if you really want
to.

~~~
IanChiles
Thanks! There's no problem with them not being configurable enough, the
problem is with understanding how they work behind the scenes. The best way,
IMHO, to address this, is by starting from nothing, and building it from the
ground up - not just compiling things, but writing basic replacements for
things like bash, init, etc. Along the way, this gives me room to try new
ideas out on a much lower level - rather than just trying to hack them into
Ubuntu/another distro.

~~~
lmm
In that case wouldn't you be better off with something like Minix that's
designed for this?

~~~
IanChiles
Well, I'm much more interested in the state of the Userland, not the Kernel
itself. And since Linux is the de-facto FOSS Kernel, it makes sense to base it
around that, instead of trying to port the various compilers/tools that I've
used to Minix. It's just easier to do on Linux, but I've tried using the
FreeBSD Kernel, as well as XNU - both ended pretty horribly, so I'm sticking
with Linux for the time being.

~~~
lmm
Minix comes with its own unix-compatible userland. It won't be as performant
or featureful as the GNU versions, because it's designed for teaching - but I
would've thought that would be what you'd want.

------
diaz
I completly understand the feeling and it's something that I wanted to do for
a long time. I thought of starting with LFS but never got much time to play
with it.

I'll definetly follow your posts about this.

I don't get why so many negative comments here. There were many times that I
really wanted to understand and configure all my machine just for the sake of
it.

Understanding and learning, it's simple. That's the main reason I use Arch,
but even with it I don't usually go really under it to understand what program
calls what and what is the function of program foobar, why it really is needed
or how many switches and power can it give.

This is great :).

------
agumonkey
I'm curious about the result, I share his idea about LFS being a little bit
too 'automatic' and the feeling of blindness that resulted. The joy of booting
a system build from source was there, but didn't last long, in fact I quickly
hit very weird network issues such as elinks seeing the network but not wget,
reinforcing the notion that I there was a need for a different kind of
information about the system.

------
drill_sarge
I don't understand. You want to do a non-GNU/Linux OS or what? Or "just
another GNU/Linux distro"?

~~~
jff
It sounds more like Kirk/Linux.

~~~
IanChiles
Which is exactly what it'll be. I really hope to keep the userland portable -
so Kirk/FreeBSD or Kirk/Darwin could happen in the distant future

------
donniezazen
Arch Linux strikes perfect balance between usability and learning. I am
looking forward to what you have to say.

------
internalnet
How in the hell is this kind of vaporware garbage on the front page?

Shut up and code.

