
Linux from Scratch – build your own Linux distro - d33
http://linuxfromscratch.org/lfs/read.html
======
Jaruzel
I attempted this a few months ago. I'm comfortable around the Linux CLI and
can happily edit files in Nano (or Vim if need be). I wanted to do this to
improvide my knowledge of the inner workings of Linux.

What I found out was, the LFS instructions are out of date, or in places
simply wrong. A lot of the linked-to packages simply no longer exist in the
locations referenced, resulting in the user having to google the package
names, and hope that the archives they find are complete and correct. As such,
there are many false starts getting even the basics compiled, and that's
before you even get to the chroot stage. After several hours of scouting
around, and compiling the dependencies, I hit a fatal compiler problem with
one of the correct packages which, not being a Linux guru, I couldn't solve.
This left me with the only option of starting over, which I didn't do. If I
had infinate free-time I would have kept working on it, but I don't, so the VM
just sits on my server, abandoned.

I like the idea of LFS, but if you are going to present a tutorial on anything
that depends on external resources, then that tutorial should be maintained so
that as long as the end user follows the instructions, it will always work,
all the time.

I might come back to it at some point, but not anytime soon.

~~~
i336_
What would be nice is if the LFS server cached copies of all of the packages
it references.

~~~
dTal
That would be very nice and reliable! For extra reliability we could wrap the
installation instructions into scripts (to save people from making typing
mistakes) and provide them as part of the package download. Ideally the
scripts would be able to handle deinstallation too, in case people want to
experiment with newer components. And it would be good to provide some
convenient central user interface for atomically managing which packages are
installed - a "package manager", if you will...

~~~
i336_
Hmm.

That sounds sort of like [https://buildroot.org/](https://buildroot.org/).
It's a bit full-on, and definitely provides a superset of what you're
describing, but sort of wanders in that general direction.

I think that LFS is intended for people who want to really understand how
Linux works, so there's no automation, and the user is required to have a hand
in every step of the process. I mean to follow up on LFS at some point, but to
deal with the intricacy and fiddliness I'd go very slowly, and the process
would probably take 3 or 4 months. But the finished system would be better
configured than anything else I've ever set up... lol

It's useful to know about issues like this though, now I can prepare to expect
that.

About versioning, that's difficult - since this is entirely manual, the
instructions have to match up perfectly with what the software actually
requires, or confusion will result.

As someone who used Slackware for a couple years, I definitely appreciate the
ability to easily install new software. :D:D:D

~~~
voltagex_
Buildroot is fantastic - I've used it to build single-purpose VMs before. I
suppose it could also be used to build really small Docker rootfs tarballs,
too.

------
gabriel
My first LFS was around 2001, which I think would be LFS v3.0. I can't begin
to express how amazing the project was to my learning and development as a
programmer.

When I started I was completely new to Linux and the command line. Everything
I do today is rooted in the foundation that I learned by doing a close read of
the book. The book requires no coding, just command-line, but soon after my
first and second reads of the book I began to automate the build process that
took me down the path of understanding how distros are put together.

I've only read the book end-to-end once or twice (I think that's all that is
needed), but I've done hundreds of builds. Make no mistake: The book is
difficult to get through. It requires trial and error as well as a close read
of the book to understand why the build is done in the way that it is, which
should be explained thoroughly, but if it isn't I'm sure the editors want to
hear about the difficulty you are having. There are also bound to be problems
bootstrapping from your <insert_favorite_distro>. That's always been a
challenge and it's part of the build methodology that LFS uses to create a
stable foundation to build a proper system (which relates to the efforts today
regarding bit identical reproducible builds).

~~~
kbenson
I went through the same process, at around the exact same time. I haven't done
much with LFS since 2001/2002, but it was invaluable as a resource to learn
how a system is put together, what provides what, and as an exposure to
bootstrapping through chroot.

Building an LFS system is the sysadmin equivalent of a programmer writing
their own DB, ORM, web framework or the like. The vast majority of the time,
you don't want to use the result in production, but the process is an
invaluable learning experience.

------
ytjohn
I went down the LFS path a year ago, but then I found buildroot
([https://buildroot.org/](https://buildroot.org/)).

I had a super old desktop system (running dos and jnos, an amateur radio
packet bbs). We have repurposed it as an APRS node and no longer needed the PC
to do anything, but wanted to have it run minicom and display the aprs
activity. The bios only supported booting off of the hard drive or floppy
drive (could not boot off of cd or usb).

I first tried attaching the hard drive via usb to a virtualbox vm so I could
do a minimal linux install, but kept running into little glitches.

Then I found and used buildroot to make a small OS with just enough to run
minicom. Installed this to the hard drive along with a bootloader and it
amazingly all worked.

I wouldn't have put that much time into something commercially, but I saw it
as a fun project/learning experience. I would definitely recommend buildroot
for anyone looking to build a custom (tiny) linux os.

------
agumonkey
I always ask this: do people feel LFS is more "learn sed and gcc build chain
and error message" rather than about the OS ?

~~~
agumonkey
ps: After my first 'successful' LFS build. I find out I had strange network
bugs, curl/wget wouldn't work, while elinks would but with console formatting
bugs (offsets and reverse video). It was both frustrating and fascinating at
the same time.

~~~
gabriel
Yep, I think that's a pretty big part of the learning process. Realizing that
you had a successful build but some things weren't stable. Naturally this
would pose the question: What went wrong? To solve that problem you have to
figure out a way to debug the system, make a decision as to how to rectify it,
apply a fix, then figure out how to verify.

Now imagine the effort that has to go into making Debian work or what if the
issue was related to sporadic kernel crashes. It's a lot of fun if you're into
that sort of thing.

~~~
agumonkey
> What went wrong? To solve that problem you have to figure out a way to debug
> the system, make a decision as to how to rectify it, apply a fix, then
> figure out how to verify.

Agreed and that's exactly the kind of stuff I was looking for in LFS and I
couldn't find. I don't blame the team too much it's already fun the way it is;
but it's confusing nonetheless.

------
ddalex
If you want to build a Linux from scratch, check Yocto Project. It allows to
configure a mind-numbing number of options, from the package format to
libraries used, to what's in the rootfs, to create read-only rootfs, etc etc.
It is targeted at embedded, but it's good for desktop too.

Disclaimer: worked on it for a couple of years.

------
ElijahLynn
Huge +1 for this resource!!

I have learned so much from this and I haven't even made much progress on this
yet. I get so interested in the details I deep dive into everything. I know it
will all continue to pay off in my career. It is a priceless resource!!!

Just one example, did you know there is a
[https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard](https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard),
go ahead and `man hier` for quick details!

Also, did you know the /usr folder is not really 'user'. Some say it means
'Unix System Resources' and their are plenty of discussions around this.

[http://askubuntu.com/questions/130186/what-is-the-
rationale-...](http://askubuntu.com/questions/130186/what-is-the-rationale-
for-the-usr-directory)

------
zwischenzug
I automated this using Docker, and talked about it here:

[https://www.youtube.com/watch?v=UGp16yvSrFE](https://www.youtube.com/watch?v=UGp16yvSrFE)

[https://github.com/ianmiell/shutit-
distro](https://github.com/ianmiell/shutit-distro)

------
i336_
I currently want to put together a cut-down, minimal userspace rootfs that
boots a VNC viewer for some old laptops. Essentially
busybox+wpa_supplicant+Xorg (for synaptics touchpad support >.>)+tigervnc, and
a bunch of hand-written init scripts to boot everything.

I could go hunting and take my existing system apart until I have a bunch of
binaries and libraries that coexist and don't cause nuclear explosions, but I
can't help but think that rebuilding all the bits I need from scratch might
get me a smaller initramfs image. (At the very least I could compile out all
the features I don't need.)

My question is: how best could I build everything (on Arch Linux) in an
isolated way, such that gcc/clang figures out that I want everything to link
together, without going full crazy and building gcc inside a chroot (like LFS
does), etc etc? I figure I want lots of PREFIX and manual specification of
libraries to ./configure, but is there anything else I should take into
account?

I also vaguely recall a single-floppy Linux distro from years back that came
with a random shell or Perl script that would do code analysis for you and
strip out never-used function calls in libraries so that the resulting library
build was smaller. Is anyone aware of anything like that that's a proper
application?

~~~
wtf_is_up
Check out Yocto Project. I've used it to build a minimal distro for Gumstix
boards, but it also supports generic x86/amd64 targets as well.

It's built on top of bitbake and you essentially enable recipes for packages
you want to have installed.

[https://www.yoctoproject.org/downloads/bsps](https://www.yoctoproject.org/downloads/bsps)

~~~
i336_
Hmm. Interesting, thanks.

------
tcoppi
I did this around 2004-2005 in high school, and it was one of the more
educational learning experiences I've had. Sadly the resulting system was
nigh-unmaintainable, on the level of old-school Slackware, so my install only
really lasted a year or so before I scrapped it because I wanted to run some
newer software.

~~~
morganvachon
> _Sadly the resulting system was nigh-unmaintainable, on the level of old-
> school Slackware_

What is unmaintainable about Slackware, unless by "old-school" you mean older
versions that aren't supported anymore, i.e. not maintained? Slackware has
been one of the easiest to maintain distros since the beginning; I would know,
I've been using it since 1999, and it's one of the few distros that survived
the early/mid 90's birth and adolescence of Linux.

------
jitix
I built and used this back in 2007 in second year of college. While it might
be a bit tricky to get it working, it's a great way for new users of Linux to
understand how Linux distros are structured and how the GNU packages work with
each other. That being said, it's a bit time-taking to use as a day-to-day
distro.

------
64bitbrain
We use LFS at work and its base OS for our firmware. I started working on LFS
like 5 years ago and we still use it(transitioning to CentOS minimal). One of
the reason, my senior devs choose to stay with LFS was it simplicity and small
footprint. We have been using LFS for almost a decade now. It was perfect fit
for OS on 30GB SSD. Too much room for other stuff. One of the drawbacks I did
saw is packages. Not easy as "yum install <>" and be done. Moreover, It got
really tricky to get LFS working on AWS.

------
gravious
I'm running LFS 7.6 as a desktop development environment with Xorg and Awesome
as tiling window manager. If I want to install other packages I systemd-nspawn
the root of a distro like Ubuntu and use apt-get.

It started out as a server environment for an HP thin client. The only
difficulty I remember was booting from a USB stick and having to set the
rootwait/rootdelay kernel parameters otherwise mounting the filesystem would
fail.

~~~
5ilv3r
I do that with slackware running debian and ubuntu guest containers.

------
anocendi
LFS Instructions are not up-to-date or well-maintained lately ....

Plus, most LFS installs will be abandoned sooner or later after the tutorial
ends anyways ....

I would recommend the uninitiated to start with Gentoo/Funtoo install for
similar experience (albeit not as low-level as LFS would be). The up-side here
is that they are real up-to-date / well-maintained distros, which one would
end up using after the install experience ....

------
billylindeman
I did this on an xbox (using xbox-linux kernel) when I was a teenager. It was
such an awesome learning experience. Between this and running gentoo is how I
got all my commandline and linux internals chops. Glad to see it's still
around! :) (I did it ~2004)

------
throwaway2016a
This was actually my first experience with Linux about 13 years ago. I
couldn't get the popular distros at the time to run on my machine and decided
to try this. I never did get it working but I imagine the process has improved
in the decade+.

------
gravious
I'm running LFS 7.6 as a desktop development environment with Xorg and Awesome
as tiling window manager. If I want to install other packages I systemd-nspawn
the root of a distro like Ubuntu and use apt-get.

------
jfe
rob pike compared using unix to listening to david cassidy every day for four
decades. we don't need another unix. we need a dramatically different world-
view.

