

Ask HN: Why don't operating systems gzip their startup files? - jsonscripter

I was pondering why modern operating systems take so long to start up... We have a huge amount of processing power on each machine after all, and we should be able to process all the required start up routines thousands of times faster than just a few years ago. I realize that the slowest part of modern computers are the hard drives (about 90 Mb/s is high for non-SSD) and not to mention seek times.<p>Which leads me to the question: why not GZIP all the required startup files? In fact, why not TAR them too, and make them sequential? Our modern CPUs would be more than powerful enough to handle the decoding process. Is there something preventing this compression? Or are they doing this already?
======
profquail
How is "deflate + execution" going to be faster than "execution" when running
the boot process?

The reason that modern operating systems take a fairly long time to start up
is due to things like enumerating hardware for PnP, getting a network address
from DHCP, and starting whatever system services are necessary for you to have
a pleasant (if you want to call it that) user experience. It really doesn't
have anything to do with the size of the executable files, since the CPU is
going to have to run the same code to make it all happen anyway (you can't
execute GZIP'd code and have it run faster).

~~~
TallGuyShort
I think his idea was that since the hard-drive is a bottle-neck, it would be
faster to pull less data (compressed files) off the hard drive and do a little
more work in memory (decompression) where it's faster.

I agree with you, though - I don't think that's really why it's slow - I'm
just pointing out what he meant.

~~~
noss
I think it is slow because of disk latency. To actually seek to find all these
gazillion small configuration files. Just look at how boot up becomes much
snappier when an SSD is used.

~~~
anamax
> I think it is slow because of disk latency.

And you even provided some supporting evidence. I don't think that it's
conclusive, but until I'm willing to do the work....

> To actually seek to find all these gazillion small configuration files.

If that's the actual bottleneck, perhaps a different configuration system
would be a better solution.

~~~
noss
I do not see that happening. It is actually a good thing that each application
can have their own isolated configuration that is tailored for the
application's needs.

It is that most file systems are optimized for large files.

Reiser FS promised to be a killer FS, handling small files and arbitrary file-
meta-data, but well... <insert un-tasteful joke here>.

------
ZachPruckowski
When you're seeking to an idle disk, three things have an impact on speed:
Seek time, file fragmentation, and file size. Given that your max throughput
to any modern HDD is measured in megabytes per second, file size isn't the
limiting factor. You've got to get the drive spinning, get the drive head to
the right spot, and read it. If there's a pile of files, you've got to do that
dozens of times if they're not stored near each other on disk. Compressing the
files doesn't really solve this problem. What one might try, however, is
arranging the disk storage such that the files are physically close to each
other. One could even go so far as to mash all the startup stuff into one
super-file.

To see this in benchmark form, look at the difference in drive speed tests
between large reads (sequential) and small reads (random). You'll go from 60
Mb/s to 6 Mb/s when file size drops from gigabytes to kilobytes, because more
of the time is spent seeking.

------
jacquesm
It's a given in the software world that if you give a programmer a faster
machine they'll produce slower code.

Developers should be using the oldest and least available memory machine they
can lay their hands on.

Want to see a zippy starting system ? Install Xenix 386 on your 2.8 GHz
machine. Or windows 95+trumpet winsock if that's your thing.

------
dkersten
Linux DOES use compressed files - BUT: decompression takes time too, so I'm
not sure exactly how valuable this is. The real slowdown is waiting for
hardware to intialise. The super fast linux booting that some people achieve
is by clever elimination of services they don't need and by starting as many
services as possible concurrently. The real slowdown is probing which hardware
is plugged in.

------
noss
The problem is probably to find out which files these are. Not that it is hard
to measure, but afaik there are no tools in place to do it right now.

------
donw
IIRC, initrd is also compressed.

------
gaius
vmlinuz?

~~~
jsonscripter
Why not _all_ of the start-up files though? There are loads of other things
(such as scripts) that are not compressed...

~~~
gaius
I expect it's for ease of editing when the system is up. Also remember there's
a limit to what you can fit into a bootloader; an entire compressed filesystem
might be too much.

