
Experiment in booting Linux fast - padde
http://gentooexperimental.org/~patrick/weblog/archives/2013-10.html
======
hmottestad
As the author says, it's just an experiment to see how fast he can get the
filesystem and kernel up.

There is no network and most functionality is missing. There are not services
or anything else.

So no point comparing this to Windows or OSX or anything else that requires
any functionality other than kernel and filesystem.

~~~
nilved
I have an pretty standard Arch Linux installation with an SSD and my boot time
is 1.5 seconds. My MacBook with an SSD takes nearly a minute.

~~~
dman
Does that exclude bios init time?

~~~
jlgreco
BIOS init time on my chromebook pixel is absolutely a killer. The "OS
verification is turned off" screen appears instantly, but then I have to
manually press ctrl-l to make seabios start loading, and seabios isn't
particularly snappy about loading then picking the harddrive so grub can kick
off. That whole process takes maybe 4 seconds which I think is a bit absurd.

My solution of course is to just suspend/hibernate instead of powering down,
but it is a little disappointing that I don't get to play the "fast boot"
game.

~~~
dman
Yes if you do unsafe things you can be fast. There is a hilarious open issue
that google refuses to fix - critical information required to boot is written
to a part that is battery backed. So if you have your pixel in developer mode,
and have installed linux on it, just close the screen and and let your battery
drain to zero - your pixel will become non bootable. All of this because
storing this information in TPM where it would be persistent would "slow" down
the boot. Speaking from experience - my pixel has lost data three times
because of this.

~~~
jlgreco
Huuh, that sounds pretty shit, I'll have to increase the frequency of my
backups... Do you have a link to that issue handy?

~~~
dman
Here you go - [https://groups.google.com/forum/#!msg/chromebook-
central/zl9...](https://groups.google.com/forum/#!msg/chromebook-
central/zl9UZd4QvJ0/zER63daiHrIJ)
[https://plus.google.com/u/0/117057264318218846563/posts/hnVn...](https://plus.google.com/u/0/117057264318218846563/posts/hnVnjpF53zY)
[https://plus.google.com/111049168280159033135/posts/4nkSEmGo...](https://plus.google.com/111049168280159033135/posts/4nkSEmGoVF4)

~~~
jlgreco
Thanks!

I've run down my battery a few times while not suspended (just me being
neglectful with plugging in) and haven't run into this issue. Is this just
because the hardware kills power to the processor before the battery
_actually_ hits zero?

~~~
dman
Yes

------
DanBC
If I had the money I'd set up a small yearly prize for "quickest boot on an
RPi". Have some rules for what is or isn't allowed, and maybe different
categories (IE, "anything goes" for people who want to overclock the hardware)
but concentrating on having a usable system when you finish timing.

The boot to a desktop in 5 seconds on an EEE PC 701 is still impressive.

Noodling through the author's site it's pretty interesting to see what they
did to cut times, and they're honest about this just being about boot time.

Does anyone else do this kind of optimization, or does it carry too much risk
or cost?

~~~
masklinn
> and they're honest about this just being about boot time.

 _re_ boot, not boot, boot was already a non-issue after booting into sh and
removing most of the services.

The later posts deal with shutdown, mostly by removing sleeps... which are
there to wait for lying hardware to finish flushing its caches. So some of
TFA's fixes are really recipes for data corruption if you're not using a
readonly FS (which TFA is, so it's no problem for his precise use-case).

~~~
leoh
Hmmm. For hard disks, a call to fsync could be sufficient instead of the
sleeps, no?

~~~
mro
No:
[http://brad.livejournal.com/2116715.html](http://brad.livejournal.com/2116715.html)

~~~
magila
That blog post is from 2005. There was a major rework of how the block layer
handles flushes back around 2010 and I'm pretty sure the issue he was having
with fsync not being reliable has been resolved.

~~~
cvs268
To go into the details, essentially, its the AHCI-driver(SATA) that handles 2
use-cases differently.

The more common being the case where there is an additional VFS driver between
the app attempting sync-I/O and the AHCI driver which simply issues an
asynchronous I/O command to the disk and returns immediately. The new data is
guaranteed to be on the HDD but NOT guaranteed to be written to the non-
volatile platter of the HDD. Data is often still in the HDD internal-cache,
waiting to be written to the disk platter.

The 2nd case (very rare) is when the application attempting sync-I/O opens the
HDD in raw mode i.e. opens the block device directly(without any VFS layer in
between) with O_SYNC. Now following each disk-write, the AHCI driver issues a
CMD_FLUSH to ensure that even the HDD cache is immediately flushed to the
platter. As this eliminates any chance for NCQ to kick in, the performance
drops by an order of magnitude but data-integrity is ensured.

------
RexRollman
I recently started using Linux again and installed Arch Linux. This was the
first time I had used it since the init system was switched to systemd and its
speed is impressive. Both booting and shutting down are very fast and I really
like the journal feature.

That said, when I installed a virgin system on Monday, I was sad to see that
systemd is creating a .local directory in the home directories of users (even
root). This is on a system with no Xorg what-so-ever. Call me old fashioned
but I don't think an init system should be creating directories in a user's
home directory.

~~~
rfnslyr
Is Arch Linux still near impossible to install flawlessly on first try?
Reporting in @ 7 tries here before I finally got it working.

~~~
groovy2shoes
They _used_ to have a decent menu-driven installer. Why they threw it out and
said, "here's zsh and a wiki page, go install," is beyond me (not to mention
the default environment of your installed system is pretty different from the
environment on the live disc). Even OpenBSD is easier to get up and running.

That said, I still contend that pacman is hands-down the best package manager
in existence. I just prefer to use it on Frugalware nowadays.

~~~
yogo
Yea I'm not sure why they got rid of the menu-driven installer and zsh sucks
imo. I can still live with the installation process though, it boils down to:
partitioning your disk, setup filesystems, /mnt and /mnt/boot (for my
desktop), use pacstrap for base system, install bootloader. After one
installation with the manual process you can still install Arch in 30 minutes,
plus you can thank it for somewhat forcing you to understand how you are
setting up your system.

Pacman is really the best. I had to be reminded of that recently when I had
the luxury of dealing with a Ubuntu system. Granted I don't have to deal with
that system everyday but having to use apt, aptitude, dpkg to accomplish
different things? What a fucking mess.

~~~
diaz
I've been living my life mostly using pacman with archlinux and in the last
year I experienced the ubuntu/debian way for the first time... Trying to get
around dozen of apt commands and other tools is truly painfull.

But it's probably just me that got used to a simple and easy life with
pacman... There was a time that I even had to create an ubuntu package, well,
don't get me started, I cried a little and lost around 3 days.

Last time I installed arch was before they removed the menu driven installer
and it sure had some problems and bugs, so even if I have not tried it yet, I
think I prefer this way of installing. It's cleaner, simpler, so better.

------
jnazario
the shutdown loop in one of the blog posts there - sync(), then sleep(2) - has
me worried he may get filesystem corruption at times under those
circumstances. i could be wrong, but i recall that sync() will return
immediately even though it's not done synchronizing the filesystem writes (for
a filesystem that needs that). as such, the sleep(2) gives it some time to get
that done.

is that a correct understanding? if so, is that a reasonable risk i see
(filesystem corruption at times)?

~~~
viraptor
I've definitely seen `sync` itself waiting/blocking (especially if you use
fuse for something network based and disconnect the cable first), but whether
it's guaranteed or not... that's an interesting question.

Edit: after some googling:

    
    
           On Linux, sync is guaranteed only to schedule the dirty blocks for
           writing; it can actually take a short time before all the blocks are
           finally written.  The reboot(8) and halt(8) commands take this into
           account by sleeping for a few seconds after calling sync(2).
    
           This page describes sync as found in the fileutils-4.0 package; other
           versions may differ slightly.
    

So it doesn't look like the writes are guaranteed to take place. Just a best
effort + wait + pray :)

~~~
jabiko
Interesting.
[http://linux.die.net/man/2/sync](http://linux.die.net/man/2/sync) says

    
    
        On Linux, sync is guaranteed only to schedule the dirty blocks for
        According to the standard specification (e.g., POSIX.1-2001), sync()
        schedules the writes, but may return before the actual writing is done.
        
        However, since version 1.3.20 Linux does actually wait. (This still
        does not guarantee data integrity: modern disks have large caches.)
    

So it seems like the sleep(2) is there to give the disk enough time to write
the cache data.

~~~
igravious
Pity no-one thought to comment this little piece of reboot/halt magic -
'twould have saved a bit of digging.

------
virtualwhys
# me> systemd-analyze

Startup finished in 2.480s (kernel) + 623ms (initrd) + 570ms (userspace) =
3.674s

Fedora 18 on Dell Precision M4700

Not quite 0.25 seconds though ;-)

Here's one way to get there: [http://www.harald-
hoyer.de/personal/blog/fedora-17-boot-opti...](http://www.harald-
hoyer.de/personal/blog/fedora-17-boot-optimization-from-15-to-3-seconds)

------
frezik
I've noticed that dhcpcd always gets the lease, then broadcasts on the network
to see if anybody else has that address, waits for a timeout, and then
proceeds. This seems like an unnecessary waste of boot time when you're on
uncomplicated home networks; the DHCP server can be trusted to give a fresh
address almost always. Doubly so if you've set your AP to static MAC address
mappings.

Is there a way to shut that off, or a Linux DHCP client that doesn't do that?

~~~
simcop2387
Switching to dhclient instead of dhcpcd will usually help that. Once it has an
address it will go into the background during that check if the ip address is
actually available. You can stop that by running it with -d if you're
debugging a bad network.

------
okrasz
Amazing! I would like to see such experiment with "boot to browser" on real
hardware.

~~~
ebtalley
I came here to post something like this as well. I'm less concerned with
bootup time and more concerned with how much time it takes to get to actually
using the system. its still in the 5~10 minute range for me.

~~~
ds_
Are you including the time it takes to open whatever applications you need to
be productive?

~~~
ebtalley
absolutely, gnome boots up very quickly but the churn that happens right after
boot slows opening applications to a crawl.

------
ohazi
Fastest "reasonably complete" boot I've ever seen was on a BeOS setup I had on
a 600 MHz laptop (~2000?). ~4 seconds to the login prompt, and another 1-2
seconds to finish "thinking" after login.

Man, that thing was fast.

~~~
e12e
I also seem to recall that the somewhat legendary QNX demo floppy (a 1.44
floppy image that booted to a full graphical desktop) was also pretty fast to
come up. I can't recall exactly _how_ fast though.

On a side note I somewhat recently booted up our old Amiga 2000, running an
upgraded cpu (68020 I think, maybe 10 Mhz) -- booting off its ancient 40 MB
scsi hd -- and I was surprised how slow bootup was. Can't remember that I gave
bootup time much thought when the thing was new (then again, most reboots were
done to boot into a game off of floppies...).

~~~
dchichkov
:)

------
Beltiras
Here's the real world thing we need: Start enough services to enable:
networking, cron, one wsgi server and a database. I need to do this next month
for a deployment.

------
agumonkey
While looking for this :
[http://www.youtube.com/watch?v=-l_DSZe8_F8](http://www.youtube.com/watch?v=-l_DSZe8_F8)
(boot to embedded real time application around 1 sec) I found this more recent
desktop talk
[http://www.youtube.com/watch?v=aVcfjs02Srs](http://www.youtube.com/watch?v=aVcfjs02Srs)
(Integrating systemd: Booting Userspace in Less Than 1 Second - ELCE 2011)

------
nine_k
Experiments like that might benefit server VMs most. How soon your nodes go up
after a reboot or after spinning up a new one can matter. But, of course,
you'll not be cutting corners as the poster does; you'd need a usable system.

On a desktop or laptop, you probably use hibernation, and your biggest time
sinks are typing your password and having your network connection go back up.
Rebooting these things is rarely needed.

~~~
sounds
Apple "cheats" on that regard, getting your wifi back up after sleep.

[http://cafbit.com/entry/rapid_dhcp_or_how_do](http://cafbit.com/entry/rapid_dhcp_or_how_do)

[http://news.ycombinator.com/item?id=2755461](http://news.ycombinator.com/item?id=2755461)
<\-- good discussion

------
cm-t
Booting linux to the graphical server in 0.25 seconds ?

~~~
hrkristian
Short answer: No.

> qemu-kvm -nographic -kernel kernel -boot c -drive file=./kvm-
> squashfs,if=virtio -append "quiet root=/dev/vda console=ttyS0
> init=/sbin/halt"

------
rwmj
libguestfs can boot & shutdown a small appliance in around 3½ seconds. However
we have to use the standard distro kernel, distro udev and distro KVM (for
security policy reasons we cannot ship our own). We could do a lot better if
we could custom compile everything.

[http://libguestfs.org/guestfs-
performance.1.html](http://libguestfs.org/guestfs-performance.1.html)

------
eliben
Chrome OS boots Linux really fast and gives you a usable system within seconds
:)

~~~
metric10
For extremely small values of "usable"...

I attended Google I/O this year and received a Pixel. It's a really nice piece
of hardware, but as a developer it's worthless. I saw Google presenters using
a lot of Macs and a ThinkPad running Linux. Not a single Chromebook.

~~~
joestringer
With secure shell, you can make it significantly more useful. Of course, you
still need a box to SSH into.

[https://chrome.google.com/webstore/detail/secure-
shell/pnhec...](https://chrome.google.com/webstore/detail/secure-
shell/pnhechapfaindjhompbnflcldabbghjo?hl=en)

------
saosebastiao
For comparison, does anybody know what kind of boot times to expect with
CoreOS?

------
moneyrich4
I would be curious how to achieve similar or better results on other distros,
particularly ubuntu.

~~~
claudius
At this point, I don’t think that the distro matters much, as there is quite
exactly nothing there except for the kernel, udev and the very first steps of
startup.

