

Ubuntu Server tech lead: The real problem with Java in Linux distros - bokchoi
http://fnords.wordpress.com/2010/09/24/the-real-problem-with-java-in-linux-distros

======
rbanffy
Watching this discussion makes it painfully obvious that the most vocal
opposers to the post are those who have never used unified software package
management. Sorry, guys, but you complaining how broken package management is
when all you know is the google, browse, download, run, next-next-finish dance
doesn't exactly help.

I don't want to know Mozilla released a new Firefox. I don't want to have to
run it to find out. I don't want to know there is a new Java available and I
don't care which version of OpenOffice is the latest and greatest. Or bzlib,
or glibc. Or Python, Ruby, CMUCL... If the packaged version is good enough, I
don't want to waste my time managing my computer, not a single minute. I'll
let the system-wide updater do its job.

And that's what a unified software package management system (I use APT) does
for me. It does its job so that I can do mine.

In the parts I need more control, I skip the whole thing and install my own
programs in any way that makes sense for the specific product. My distro
provides a reasonably up-to-date Python, but if I am to muck around libraries
that conflict with the packages provided by the distro, I will use a
virtualenv, which is a stand-alone Python environment, with its own libraries
and easy_install/pip. This is similar to the "bundle your versions" approach,
but, rather than the norm, it's the exception.

~~~
regularfry
We're quite lucky, really, that Python by default installs cleanly into
/usr/local.

~~~
rbanffy
Indeed. We are the lucky ones.

------
wmf
"Linux distributions must build everything from source code."

No, they don't have to. That's a self-imposed rule that's beneficial in some
cases but not others. When an upstream developer releases nothing but source
code, I appreciate the fact that the distros compile it on my behalf. But when
an upstream developer (like Mozilla or Google or all these Java developers)
releases a fully built and tested binary, I'd prefer the distros to just pass
that on to me.

~~~
wazoox
That's impossible because the distribution is responsible for the updates it
provides. If you package libraries with your program alla Firefox, Chrome, or
Java, you simply make your program unmaintainable and unsupportable from the
distro point-of-view. This is true of Ubuntu, as well as Debian or RedHat.
This way of thinking is incompatible with basically _all_ of the major
distros.

~~~
wmf
Honestly, I don't even want to get apps from distros at all; I'd prefer to get
tested binaries directly from upstream, but all the disagreement over package
formats and such makes that nearly impossible. I feel like distros have
painted themselves into a corner with their rigid policies and now they're
complaining about it.

~~~
theBobMcCormick
I agree. I'd much rather be able to go out to, as an example, mozilla.com and
download the latest firefox in an easy to run installer (like I can do on
Windows), than have to upgrade my entire fricking OS just to get the latest
Firefox.

Again, just using Firefox as an example. The same applies to most end user
apps, stuff like Java and Flash, etc.

~~~
kree10
In the case of Firefox, I don't even want an installer. I like the OS X
install process for Firefox (and many other apps): copy Firefox.app to
/Applications. That's it.

~~~
GeneralMaximus
As much as I love application bundles, I'd like to point out that this method
of distribution leads to code duplication. For example, almost every popular
OS X application ships its own version of Sparkle and Growl (to verify this
yourself, cd into the app bundle and go to Contents/Frameworks).

~~~
theBobMcCormick
Does it matter? Disk space and ram are both cheap and plentiful. The
difference in disk space or ram taken by apps on an average OSX desktop, and
average Windows desktop, and an average Linux desktop seems to be negligible.

~~~
kelnos
_Does it matter? Disk space and ram are both cheap and plentiful._

That just sounds like promoting a culture of waste to me. "Do we need to care
about resources?" "Nah, disk and RAM are cheap, and our users won't mind
paying for more of it."

What happened to elegance, efficiency, and modular, maintainable design? Just
because you have a shiny box with 8GB of RAM and a terabyte of space, it
doesn't mean everyone does, or everyone should have to.

 _The difference in disk space or ram taken by apps on an average OSX desktop,
and average Windows desktop, and an average Linux desktop seems to be
negligible._

I'm not sure you can meaningfully make this comparison since the platforms (at
least at the GUI level) are quite different. A better test would be to take a
"standard" Linux system, and then create a "bundled" system that has each app
in its own directory with all its dependencies bundled with it (perhaps
leaving out some of the fundamental ones like libc). A little bit of common
reasoning would suggest that memory and disk usage would be affected greatly.

~~~
vilya
> What happened to elegance, efficiency, and modular, maintainable design?

Each app having it's own copy of the libraries it needs is actually MORE
maintainable.

It's not uncommon for a newer version of a library to introduce a change which
breaks something in your app. One example from my work: we tried building with
a newer version of Qt and all our text was suddenly getting rendered upside-
down due to changes in how Qt uses OpenGL. If we had been using the system
copy of Qt, it would have been our customers seeing this problem instead of
our dev team.

There are other advantages too, but to my mind this alone is enough reason to
bundle libraries with your app.

~~~
kelnos
But is that the common case? Or is the common case that, on occasion every now
and then an incompatible change is introduced, but much much more often bugs
and security issues are fixed that your users will benefit from immediately,
and not have to wait for you to release a new version with new bundled
libraries.

I'm a bit surprised that the Qt guys would make an incompatible change during
a stable series. Is it possible you were relying on undocumented behavior that
was subject to change?

------
bokchoi
_The next obvious solution is to make separate packages for every version of
library that the software uses. The problem is that there is no real
convergence on “commonly-used” versions of libraries. There is no ABI
protection, nor general guidelines on versioning. You end up having to package
each and every minor version of a library that the software happens to want._

I don't understand why this is a problem exactly. If a dev runs and tests
against a particular version of a library, use that version. Even minor
updates in libraries can cause problems. If the library has a security issue,
blacklist the version and force the upstream dev of both the library and the
app to acknowledge the issue and release a new versions of their projects.

~~~
etal
Force upstream to fix a bug and release a new version? And what if they don't?
Quite a few packages in Debian have somewhat dormant upstream authors - the
packages work, perhaps needing a few patches to compile with the latest
versions of common libraries, but the original author has essentially moved
on.

Consider:

 _libfoobar has a bug in at least one version seen in the wild. Is my system
safe?_

If your distro packages just one or two stable versions of libfoobar, any
package that depends on libfoobar is either OK or not OK, and if it's not OK,
you can patch the bug in one place and you're safe again. If upstream is
dormant, perhaps the current package maintainer can fix it.

If there are various versions of libfoobar being linked by individual apps,
you need to check every app for the flaw and work with both the app author and
libfoobar's author to determine whether the flaw exists and how to fix it.
Upstream libfoobar might say the bug has been fixed in the latest release, so
just upgrade to that. Upstream app then has more work to do, and may be in
denial about the importance of the bug. And if the source isn't available for
the precise libfoobar bundled with the app, the package maintainer would have
to either (a) rework the app to work with a stable system version of
libfoobar, (b) package the odd version of libfoobar separately, and link that,
or (c) delete the package from the distribution.

~~~
bokchoi
If upstream won't, can't, doesn't want to fix an upstream bug, then no one
wins. Distributions patching upstream outside of the original source is a bad
a idea -- other distributions need the same fix most likely. Kindly convincing
the upstream app to rework their app with the new stable library is the ideal.

I realize it's a give and take, but the blog post is putting the blame
squarely on the java dev's shoulders.

------
swaits
This is synonymous with Zed's recent Python rant, and the "DLL hell" mentioned
previously.

FreeBSD went through the pain of extracting itself away from its perl
dependency years ago. And, it's way better for it. There's nothing like a
minimal install. A nice, clean, empty slate.

------
hvs
This sounds peculiarly like the issue of "DLL Hell" on the Windows platform.
Ultimately, you either hope that the shared libraries match, ship with the
DLLs you need, or (as a newer option) throw your chips in with WinSxS -- where
the cure might actually be worse than the disease.

Maybe the fact that Java grew out of the proprietary world is part of the
reason why it doesn't play nicely with open-source/free operating systems.

~~~
barrkel
I think it's because Java sees itself as its own operating system, and because
it started out with a pretty crummy module system that's only been
incrementally improved over the years.

~~~
bokchoi
And unfortunately the crummy module system improvements slated for JDK7
(Project Jigsaw) have been pushed to JDK8.

------
jedwhite
It's not just Java. Python is stuck at 2.4 in most Linux distros, and arguably
Python's been a symbiotic part of Linux's success.

~~~
dochtman
I just don't see this being true. In what distro is it _stuck_ at 2.4?

~~~
wazoox
Maybe RedHat, but it's about it. Even Debian Stable comes with 2.5.

BTW this rant about impossibility to upgrade is a classic one and nonetheless
wrong. You're perfectly able to install new versions of python, or perl or
whatever given that you do that in /usr/local to keep the official version
alongside the new one. I do this all the time, my system comes with perl 5.10
but I regularly install the newest releases in /usr/local, or even development
versions. No problem.

------
zokier
"The problem is that Java open source upstream projects do not really release
code"

Wait what... If projects do not release source which you can modify, build and
repackage, does it really deserve to be called open source project?

And I thought also that integrating different projects was distributions main
task. Isn't that exactly what he is talking about? Distros like Debian already
backport massive amounts of code in a lifetime of a relase to get all stuff
working with their specific versions. Does Java really differ that much?

~~~
bokchoi
Of course open-source projects release their source code. He's just being
silly.

------
barrkel
How is this not also true for things like Ruby (gems), Perl (cpan), etc?

~~~
etal
Ruby, Perl and Python packages usually come with a README that says:

    
    
      This depends on these external packages: ...
    

Java programs usually come with a bunch of .jar files which were once
independent packages, but have been dropped into the release itself. No
dependency problems!

Then, if someone wants to package a Java application for Debian, the process
is:

1\. Look through the collection of .jar files in the release

2\. Do you recognize one of these as already being packaged for Debian?

3\. Work with upstream to delete that .jar from Debian's copy and depend on
the system's version instead

4\. Repeat for every other .jar in the release, until you hit a wall

5\. Upload the package to Debian with an acceptably small number of bundled
.jars

6\. Time permitting, get someone to package the other .jars that aren't
available in Debian yet

That said, you can cause a similar amount of trouble in other languages, it's
just not the convention (thanks to the success of gems, CPAN, PyPI). For
example, Ubuntu appears to have deleted the sagemath package because upstream
keeps their own patched copies of dozens of libraries they depend on:

[http://packages.ubuntu.com/search?keywords=sagemath&sear...](http://packages.ubuntu.com/search?keywords=sagemath&searchon=names&suite=all&section=all)

~~~
freiheit
They generally do much better than a README: machine-parseable is the way to
go.

Perl (CPAN) packages include a YAML file that specifies what other modules
(packages) and what minimum version of each is required to configure, build
and run the package. There's an ecosystem of tools available for turning a
Perl package into an RPM or deb, some of which can even work recursively. Even
before the YAML was standard in CPAN packages, it just wasn't that hard to
parse out all the "use" and "require" statements to automatically detect all
the required packages (and minimum versions of those). There's also a unit
testing framework to make sure you don't accidentally introduce any
incompatibilities with untested newer versions of required packages.

I haven't dealt with Ruby quite as much, but the gem format also includes
dependency information and there's gem2rpm for RPM and dpkg-gem for deb
packages.

I think I've only ever once had to build an RPM or deb package of a Python
package, but Python seems to natively support building both formats; just call
the same "build and install" method you'd usually use with an extra argument
and you get a native package, which will use dependency information if the
python package provided it.

------
zokier
I think the real problem is that there are too few package maintainers for
Java packages, and the upstream binaries are usable enough that there is less
incentive to become a Java pkg maintainer compared to eg C packages.

edit: also, I don't believe in either of his solutions. Real solution imho
would be to patch upstream source to work with distro provided libraries (of
course in some cases patching the library is also viable alternative).

~~~
flomo
I think the real real problem is that assuming a "package maintainer" must
exist for every single goddamn software package for every single goddamn Linux
distribution has very obvious human scalability problems.

Just because an infinite number of monkeys appeared to create Debian does not
mean it is a safe assumption to assume that there are many other groups of
infinite monkeys out there to support every other software ecosystem.

Eventually, the Linux community will figure out that they've run out of bodies
to build the same crap over-and-over and 'binary compatibility' will stop
being a dirty word.

------
nerd_in_rage
The real problem is Linux package management.

Stop worrying about duplicating files: disk is cheap.

Stop worrying about building everything from source: I'd rather use the
official Firefox than the screwy patched version. (I'm looking at you, Debian
Iceweasel.)

Just look at the way OS X does things.

I've been using Linux since 1993 and it's amazing this crap hasn't been fixed
by now.

------
polynomial
_We want to avoid code duplication (so that a security update in a library
package benefits all software that uses it)_

The crux of the problem seems to be the Big Brother attitude distros take
toward users and apps, specficially protecting users from unpatched
vulnerabilities in upstream apps.

While this is crucial on servers and mass deployments, it's entirely possible
users find this more of an annoyance than a feature, and thus we may have to
wait just a little while longer for the so-called Year-of-Linux-on-the-
Desktop.

------
emarcotte
Some have suggested that maven as a solution. I guess the only part I'm
missing is how maven ties into the actual system. From what I've seen it is
always pulling from my ~/.m2 repository, or a full repository upstream. Is
there a way to have like a "System" repository that yum/apt/etc could install
into?

~~~
lallysingh
Getting things to build with maven's idea of a build process has traditionally
been tough, but it's pretty good for auto-downloading dependencies. Usually I
needed a shell script that had maven download some, then wget'd a few more
(that, for licensing reasons, couldn't go in an upstream server).

Unless a slightly-customized maven is distributed, you'd have to put a
~/.m2/settings.xml in the new-user template that specified your local
repository. Which isn't too bad.

~~~
bokchoi
Apache Ivy is another solution for auto-downloading dependencies but plays
nice with Ant.

------
fauigerzigerk
I think we need a radical break with the past to solve the horrible dependency
nightmare we're looking at today. These would be my initial principles:

There should be _one_ unified Linux OS that only includes the bare minimum set
of applications. It should be very clear what belongs to the OS and what
doesn't. The root directory should contain exactly three sub directories: os,
app and home.

There should be no dependencies on any non OS software. Each application
should live in its own subdirectory of /app. Libraries that are not part of
the OS should not be shared across applications. The only external dependency
an application should have is the OS.

Whatever the new solution is, it should not include package mangers or package
maintainers. Their existence is a symptom of an overly complicated system.

I realise that having a central registry of all installed software components
does have advantages like being able to fix some security issues in one place.
However, I think the idea has failed. It creates too many intractable
dependencies, it is too complicated and hence ultimately insecure and
unproductive.

------
rbanffy
One thing that always shocked me is how many Java apps play "library bingo".

There is no reason to use _every_ third-party library available. IDEs make it
easy to introduce all kinds of weird dependency in your code, but, please,
don't.

------
c00p3r
Why should one solve that jar-hell?

There is a "standard" JRE. OpenJDK is already included in any distro.

You think we should include all up-to-date crap with all dependencies? No. It
is not our problem.

~~~
zokier
Who is 'us' here?

~~~
c00p3r
In this case - sysadmins.

