What I found out was, the LFS instructions are out of date, or in places simply wrong. A lot of the linked-to packages simply no longer exist in the locations referenced, resulting in the user having to google the package names, and hope that the archives they find are complete and correct. As such, there are many false starts getting even the basics compiled, and that's before you even get to the chroot stage. After several hours of scouting around, and compiling the dependencies, I hit a fatal compiler problem with one of the correct packages which, not being a Linux guru, I couldn't solve. This left me with the only option of starting over, which I didn't do. If I had infinate free-time I would have kept working on it, but I don't, so the VM just sits on my server, abandoned.
I like the idea of LFS, but if you are going to present a tutorial on anything that depends on external resources, then that tutorial should be maintained so that as long as the end user follows the instructions, it will always work, all the time.
I might come back to it at some point, but not anytime soon.
LFS is pretty good at teaching you what a working Linux system is made out of. It's not a productive way of actually creating a useful Linux system; it's simply not practical to select and compile all your dependencies from source without automation. But the lack of automation is exactly what makes it pedagogical.
As such, it's not wholly clear to me that you need to go through the steps. But performing them certainly makes the experience more visceral, and fixing minor issues along the way (if you can) improves your confidence installing software from tarballs when they're not available or are out of date in your distro's repository.
"Welcome to Linux From Scratch!
Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own custom Linux system, entirely from source code."
Some packages were missing but that's not really LFS's fault if upstream doesn't keep old source tarballs around. I was able to find everything by searching for the MD5 sum though.
The hardest part was the kernel configuration. LFS doesn't really give any instructions for it - you need to figure it out on your own. It took lots of trial and error to get all hardware working. If I remember correctly my network card's driver was out-of-kernel and I had to compile it separately.
It also takes a really long time to compile everything. Unix is supposed to be simple but GNU's Not Unix.
This is a pretty tall order for a niche project like LFS, which is most likely maintained by a bunch of guys on their free time. You really can't expect them to be able to keep it up to date all the time.
Fwiw. it's not too difficult to build a small bootable linux distro, but it might be easier to use e.g. Musl+BusyBox rather than GNU userspace. I've put together a small bootable Linux from Git repos, it takes about a day or two to build binutils and GCC for cross compiling, then building kernel, libc, busybox and making an initrd image that boots.
See the bottom section of the page http://www.linuxfromscratch.org/lfs/download.html#ftp
Edit: a generic point that I'd make is that just because something is hard, and there's a reason it's hard, doesn't mean that it has to be hard in every possible way. This is another version of the distinction between accidental and intrinsic complexity.
That sounds sort of like https://buildroot.org/. It's a bit full-on, and definitely provides a superset of what you're describing, but sort of wanders in that general direction.
I think that LFS is intended for people who want to really understand how Linux works, so there's no automation, and the user is required to have a hand in every step of the process. I mean to follow up on LFS at some point, but to deal with the intricacy and fiddliness I'd go very slowly, and the process would probably take 3 or 4 months. But the finished system would be better configured than anything else I've ever set up... lol
It's useful to know about issues like this though, now I can prepare to expect that.
About versioning, that's difficult - since this is entirely manual, the instructions have to match up perfectly with what the software actually requires, or confusion will result.
As someone who used Slackware for a couple years, I definitely appreciate the ability to easily install new software. :D:D:D
This project certainly has been well thought through. Looking forward to figuring it out :D
might read through though
setting up arch really did learn me few tricks - i find LFS similarly beneficial
One can experiment alot with Arch Linux too, if compiling is a burden.
Pause. Perhaps that's the solution: LFS should host internal copies of all the required source for a target version, so the instructions can always be the same. Provide references for who and where the upstream maintainer is, so the intrepid hacker can find it themselves.
But that just takes you to the trac front page.
When I started I was completely new to Linux and the command line. Everything I do today is rooted in the foundation that I learned by doing a close read of the book. The book requires no coding, just command-line, but soon after my first and second reads of the book I began to automate the build process that took me down the path of understanding how distros are put together.
I've only read the book end-to-end once or twice (I think that's all that is needed), but I've done hundreds of builds. Make no mistake: The book is difficult to get through. It requires trial and error as well as a close read of the book to understand why the build is done in the way that it is, which should be explained thoroughly, but if it isn't I'm sure the editors want to hear about the difficulty you are having. There are also bound to be problems bootstrapping from your <insert_favorite_distro>. That's always been a challenge and it's part of the build methodology that LFS uses to create a stable foundation to build a proper system (which relates to the efforts today regarding bit identical reproducible builds).
Building an LFS system is the sysadmin equivalent of a programmer writing their own DB, ORM, web framework or the like. The vast majority of the time, you don't want to use the result in production, but the process is an invaluable learning experience.
And the most recent Gobolinux release made use of a modified version of ALFS to bootstrap the kernel and console environment.
Apparently the real troubles came when trying to get a DE going...
Since you have complete control over what is installed, the system used to boot up within seconds. The downside was maintenance.
I had a super old desktop system (running dos and jnos, an amateur radio packet bbs). We have repurposed it as an APRS node and no longer needed the PC to do anything, but wanted to have it run minicom and display the aprs activity. The bios only supported booting off of the hard drive or floppy drive (could not boot off of cd or usb).
I first tried attaching the hard drive via usb to a virtualbox vm so I could do a minimal linux install, but kept running into little glitches.
Then I found and used buildroot to make a small OS with just enough to run minicom. Installed this to the hard drive along with a bootloader and it amazingly all worked.
I wouldn't have put that much time into something commercially, but I saw it as a fun project/learning experience. I would definitely recommend buildroot for anyone looking to build a custom (tiny) linux os.
Now imagine the effort that has to go into making Debian work or what if the issue was related to sporadic kernel crashes. It's a lot of fun if you're into that sort of thing.
Agreed and that's exactly the kind of stuff I was looking for in LFS and I couldn't find. I don't blame the team too much it's already fun the way it is; but it's confusing nonetheless.
Disclaimer: worked on it for a couple of years.
I have learned so much from this and I haven't even made much progress on this yet. I get so interested in the details I deep dive into everything. I know it will all continue to pay off in my career. It is a priceless resource!!!
Just one example, did you know there is a https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard, go ahead and `man hier` for quick details!
Also, did you know the /usr folder is not really 'user'. Some say it means 'Unix System Resources' and their are plenty of discussions around this.
I could go hunting and take my existing system apart until I have a bunch of binaries and libraries that coexist and don't cause nuclear explosions, but I can't help but think that rebuilding all the bits I need from scratch might get me a smaller initramfs image. (At the very least I could compile out all the features I don't need.)
My question is: how best could I build everything (on Arch Linux) in an isolated way, such that gcc/clang figures out that I want everything to link together, without going full crazy and building gcc inside a chroot (like LFS does), etc etc? I figure I want lots of PREFIX and manual specification of libraries to ./configure, but is there anything else I should take into account?
I also vaguely recall a single-floppy Linux distro from years back that came with a random shell or Perl script that would do code analysis for you and strip out never-used function calls in libraries so that the resulting library build was smaller. Is anyone aware of anything like that that's a proper application?
It's built on top of bitbake and you essentially enable recipes for packages you want to have installed.
It uses Kconfig to give pretty fine grained selection of what packages are included in your file system and supports (painless!?) cross compilation by building the proper compiler toolchain or using a preexisting toolchain.
Once you are happy with your configuration, you can save it e.g. as a "default config" and regenerate it repeatably, reliably, and effortlessly. It takes a while to do a full build, but caching speeds up rebuilds (you have to be careful to "rm" cached parts that you really do want to rebuild).
make distclean && make <defaultconfig> && make
I was just thinking along the lines of building the libraries and binaries together and adding an init script or two, without anything else.
(Also, what's the prototype? That sounds interesting.)
What is unmaintainable about Slackware, unless by "old-school" you mean older versions that aren't supported anymore, i.e. not maintained? Slackware has been one of the easiest to maintain distros since the beginning; I would know, I've been using it since 1999, and it's one of the few distros that survived the early/mid 90's birth and adolescence of Linux.
It started out as a server environment for an HP thin client. The only difficulty I remember was booting from a USB stick and having to set the rootwait/rootdelay kernel parameters otherwise mounting the filesystem would fail.
Plus, most LFS installs will be abandoned sooner or later after the tutorial ends anyways ....
I would recommend the uninitiated to start with Gentoo/Funtoo install for similar experience (albeit not as low-level as LFS would be). The up-side here is that they are real up-to-date / well-maintained distros, which one would end up using after the install experience ....