
Linus says Linux has become 'bloated and huge' - nreece
http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/
======
pmjordan
Rather than bringing out the old monolithic vs microkernel chestnut, let's
hear some realistic ideas on how to cut down the bloat.

Does anyone know how much redundant code is in Linux right now? Would finding,
isolating, generalising and merging it be a good way to start? I know of
examples of this happening, e.g. libata taking over a lot of the old 'ide'
drivers; the many attempts at unifying the various WLAN subsystems and
drivers.

~~~
rythie
Whilst libata and the unifying of the WLAN may make the code easier to
maintain, it will only save a tiny amount of memory and in the short term,
just like any new code, is going to increase the number of bugs.

It sounds like the problem is that everyone is writing new features that
probably not many people actually _need_ (since they did fine with out it
before - I'm not talking about drivers here) but it's slowing the whole kernel
down and people need to go back and optimise that stuff.

~~~
robryan
That's the problem though isn't it, you start off light weight and you
continue working, adding more and more new and useful features until you have
it all but now have a giant slow bloated system.

~~~
jimbokun
This makes me think of Apple's Snow Leopard. It was remarkable that they took
an entire release just to shape up the code. Is something like that feasible
for Linux with the open source model? Do you just have Linus tell everyone "I
will only accept commits that improve performance and reduce memory footprint
in this release?"

~~~
rythie
The difference is that Apple has a very specific idea what their kernel is
for, where as Linux is pushed and pulled in different directions by people who
want something very large scale or secure or embedded or lots of archetictures
and so on.

Also you should put it in perspective, the Linux kernel itself is really not
so bloated, that any normal user would notice, like they do with
Gnome/KDE/Firefox's bloat. Even with that type of bloat, Linux distros still
run well on a 1GB (probably less) and an old P4 CPU which people don't say
about recent releases of OSX or Windows.

------
fnid
I think it just shows that open source isn't immune to the same kinds of
problems that plague commercial software. As the market for a product expands,
so too do the features people require for the software to work within an
ecosystem of other software. More hardware to support requires more code,
bigger systems.

~~~
skwiddor
My operating system is open source but I bought it. What kind of software is
it?

~~~
foole
scamware? ;)

------
scythe
Is it really possible to implement the full functionality of the Linux kernel
in such a way that it would be significantly smaller? Obviously, there's bloat
from bad code, but would a redesign of the kernel be seriously useful, or just
a waste of time? As a (possible) comparison, is NetBSD's kernel any smaller?

It'd be nice to know if Linus had anything to say to that effect - it might
lead to a more productive discussion than waxing lyrical about the wonders of
microkernels.

~~~
davidw
> Torvalds answered. "Acceptable and avoidable are two different things. It's
> unacceptable but it's also probably unavoidable."

I think that's more or less where things are right now. You can't have tons of
features without paying a price. They do a pretty good job of balancing
things, but maybe someone will come along and improve things in the future.

------
jsz0
It's easy to make a tiny super efficient kernel. Just don't support stuff.
WLAN? Who needs it? SMP? That'll never catch on. Take it out. Device driver
support? Bleh. Let hardware makers post the source code on their websites and
users can download and compile the code. No need to include it in the kernel
source tree. And who uses anything but x86 anymore? Get all that PPC, ARM,
SPARC non-sense out of there too. If we all work together we can make Linux
completely useless but hey, it'll be tiny and efficient. We can sit around and
congratulate each other for being such elegant programmers. We won't have to
worry about those pesky end users anymore either.

~~~
dkarl
Device drivers and processor architectures might make the source tree big, but
they don't make the kernel slower or its memory footprint larger. I believe
you can even still compile the kernel without SMP support.

------
ilitirit
I don't understand why he mentioned this.

I understand software bloat to describe the situation that arises when
_unnecessary_ features added to a program increases it's size and/or decreases
performance, without providing significant user benefits.

In the article, Linus says "Acceptable and avoidable are two different things.
It's unacceptable but it's also probably unavoidable."

If it's unavoidable, then the features that were added aren't really bloat,
because it implies that these features are _necessary_ , _and_ that they
provide significant benefits. Necessary additions that provide significant
benefits could be anything from security features to IO.

IMO, Linus should either state what he thinks should be removed from the
kernel, or he shouldn't even bother mentioning it at all.

If on the other hand he's lamenting the fact that there's a cost associated
with adding features (necessary or unnecessary), well then I think most people
understand that you can't have your cake and eat it, so even from that
perspective his point seems redundant.

------
stinkytaco
How about "Linux Lite"? I know, it's probably a silly idea, but is it possible
that the kernel team could provide a "core" kernel, that runs on most modern
hardware, most popular filesystems, etc. Shoot for some arbitrary support
level (80% of systems). Also provide a "large" kernel with everything. Distro
maintainers can use either.

Actually, that brings up another point. Who, besides Slackware maybe, uses a
stock kernel? Don't most distros customize it anyway? Am I being unfair in
saying most Linux users don't really encounter the kernel at all?

------
Gibbon
Roughly 11.5 million lines of code for the kernel.. That's way more than I
would have imagined.. 5k lines a day are added and 2k are modified.
([http://www.itworldcanada.com/a/IT-
Workplace/68f20d4f-8ab9-40...](http://www.itworldcanada.com/a/IT-
Workplace/68f20d4f-8ab9-402c-be30-df47e9812836.html))

IMHO it doesn't really jive with UNIX principles to have such a monolithic
kernel.

~~~
mahmud
It's monolithic but _modular_ 11.5M LOC for every platform it runs on, not to
mention drivers for the devices it supports, and all the modules for other
functionality, pluggable file-systems and networking stacks. Now throw in code
for backward compatibility going further back at least to 2.4 and you have
11.5M lines.

In reality, the core Linux kernel is just the stuff that compiles to vmlinux.
You could read it all in a few weeks and understand it.

~~~
skwiddor
thats right, bloat

filesystems dont belong in the kernel.

~~~
agazso
Yeah, there is the FUSE project: filesystem in a userspace. There are lots of
interesting filesystems using this e.g. NTFS-3G, sshfs or ferrisfuse. These
projects are valuable and most of them doesn't belong to the kernel as they
use userspace libraries for network protocols and xml parsing.

Unfortunately native disk file systems like NTFS-3G is lot more slower than
their native counterparts.

~~~
scythe
&totse users ran tests of zfs-fuse back in the day and found that it was
actually noticeably slower than native ext3. From that, I'm inclined to argue
the opposite: filesystems should almost definitely be in-kernel. Granted,
their tests were on Linux, not on a modern microkernel.

~~~
skwiddor
Linux, fuse

there's your problem. it's not designed for it

------
known
Who should decide if a piece of code to run in kernel space or user space?

    
    
        user/developer/management/customer/congress/judiciary

------
zouhair
Now micro kernel doesn't seem that stupid after all?

~~~
snprbob86
Despite the snarky tone of your comment, you hint at an interesting point:

Unix is NOT the be-all, end-all definitive operating system. Linux is NOT even
the be-all, end-all Unix (or Unix-like). There is A LOT of research left to be
done in operating systems / kernels and many many years of evolution to come.

It's a shame that it is SO HARD to build the ecosystem required to launch a
kernel. New ideas are hard pressed to gain traction.

~~~
pyre
You realize that OS X uses a micro-kernel, right? And that GNU Hurd is a
micro-kernel too?

~~~
vinutheraj
"Mac OS X is sort of microkernelish. Inside, it consists of Berkeley UNIX
riding on top of a modified version of the Mach microkernel. Since all of it
runs in kernel mode (to get that little extra bit of performance) it is not a
true microkernel, but Carnegie Mellon University had Berkeley UNIX running on
Mach in user space years ago, so it probably could be done again, albeit with
a small amount of performance loss, as with L4Linux. Work is underway to port
the Apple BSD code (Darwin) to L4 to make it a true microkernel system." -
Andy Tanenbaum

reference - <http://www.cs.vu.nl/~ast/reliable-os/>

~~~
pyre
I realize that. On the same note, GNU HURD isn't technically a 'micro-kernel'
because GNU Mach is the micro-kernel, but my point was that 'micro-kernel'
instead of 'monolithic kernal' isn't some unexplored territory that only
exists as theory in CompSci textbooks.

------
ecq
it's a monolithic kernel what did he expect.

Remember the Tanenbaum vs. Torvalds debate?

I wonder what Linus thinks about it now.

btw, here they are together.

<http://lwn.net/images/conf/lca2007/lt-ast.jpg>

~~~
Tuna-Fish
I'm sorry, but do you have any idea what you are talking about?

How the kernel is organized has no meaning on how bloated it is. A microkernel
can be just as bloated, in it the bloat is just not in the kernel itself, but
in the processes providing the services a monolithic kernel would provide. If
anything, a microkernel is always more bloated because of the extra code
needed to do all the process sync.

Linux being a microkernel would make the situation _worse_.

(What we need to do is to start spending more time on cutting features.)

~~~
tensor
Does bloat mean too many features? In that case I'd say those features are
there because someone needs them. They don't all have to be loaded. In a
monolithic kernel that has modules, load the ones you need. In a microkernel,
load and unload as needed on the go.

Then there is the support issue. In Linux, all the modules must be maintained
by the kernel team. This may constitute bloat as the team may have an
unnecessary amount of code to maintain. Some modules may only be used by a few
people. The issue isn't the features, but rather Linus's philosophy.

From his posts, it seems he like to keep changing kernel APIs as he sees fit.
If you have many modules, this creates problems. Without a strict API, you
will frequently break modules and thus create a lot of maintenance work. Fixed
APIs are both very important and very good for large scale collaboration.
Without standards like POSIX or the Windows APIs, we wouldn't be where we are
today.

The second aspect to this is security. Say you built a very stable API and let
others make modules as they need them. In a monolithic model, to maintain
security you must audit every module. Thus for Linux to stay secure, the
kernel team would need to either declare many less used modules as "tainting
the kernel" or audit them all: a lot of work. In a microkernel model, the
kernel team, so long as they designed it right, can let users build modules
without audit.

Saying something has too much bloat is ambiguous to the point of being
useless. Many people use that term simply as meaning slow, or a lot of code,
or even poorly designed code. It's always better to describe things precisely.
If it has too many features, say so. If it has bad design, then state that.

~~~
dkarl
_Saying something has too much bloat is ambiguous to the point of being
useless. Many people use that term simply as meaning slow, or a lot of code,
or even poorly designed code. It's always better to describe things
precisely._

The interviewer asked Linus about benchmarks that show the kernel getting
slower and slower each year. Linus acknowledged that the kernel is in fact
slowing down. I don't think they would be worrying about badly configured
kernels, or kernels with unnecessary modules loaded. I don't know what
features Linus is talking about that make a properly configured Linux kernel
bigger and slower than it was ten years ago. Nor do I see any explanation or
examples cited on this page. I wish someone who understands would give some
examples of the feature creep they're talking about.

------
known
Linus is getting old.

------
staunch
Out of context link bait drivel.

~~~
WalterGR
How so?

> "Uh, I'd love to say we have a plan," Torvalds replied to applause and
> chuckles from the audience. "I mean, sometimes it's a bit sad that we are
> definitely not the streamlined, small, hyper-efficient kernel that I
> envisioned 15 years ago...The kernel is huge and bloated, and our icache
> footprint is scary.

~~~
paddy_m
Does anyone know what icache is? instruction cache?

~~~
kmavm
Yes: instruction cache. Big programs are slow programs.

~~~
mahmud
I think joechung is more right here, icache in this instance means _inode
cache_ ; the CPU D-Cache and I-Cache (not to be confused the file-system
related dcache and icache for dentry cache and inode cache respectively) are
of fixed size and you couldn't have "bloat" there. The FS _inode_ cache is
more likely to be bloated given the larger upper limit for disk storage
compared to CPU cache.

~~~
gjm11
No way, for three reasons. (1) Saying "icache" and meaning anything other than
"instruction cache" would be an obvious recipe for confusion, and Linus is not
stupid. (2) It makes excellent sense with the obvious "instruction cache"
meaning: "our icache footprint" means "the number of instruction cache lines
occupied by kernel code". (3) It makes no sense with the "inode cache"
meaning: how could a growth in the amount of code in the kernel possibly have
anything much to do with the size or the occupancy of the inode cache?

~~~
mahmud
And you would be absolutely right. Thanks for the good analysis :-)

