
Making Debian Responsible For Its Actions - alexkay
http://sheddingbikes.com/posts/1285659877.html
======
davidw
I didn't see the words "bug report" anywhere there. Debian is a volunteer
organization, open to anyone. If you want to improve it, sign up and help
instead of just ranting about crazy paranoid theories like:

> It's simply a tactic to make sure that you are stuck on Debian.

Of course I'm biased, as I'm a former Debian maintainer, but I think that
Zed's connection with the reality of the situation is tenuous, at best, in
this case.

Making a distribution is difficult, and doing so with 1000's of volunteers who
are not working on it full time (and since you're not paying for it, they
don't really owe you anything) and are distributed throughout the world adds
to the difficulty, so yes, there are bugs and problems and challenges to
overcome. That said, if each author of each package in Debian got their way
about the exact location of files and so forth, the system would be utter
chaos. If you think your package isn't being treated right in Debian, get on
the mailing list, file a bug report, make your case, and get things fixed,
rather than treating Debian as "the enemy"... Sheez.

~~~
JoachimSchipper
There is a reason that Debian is being singled out, which you ignore - Debian
likes to add lots of distribution-specific patches to _everything_ they
package. Other OSes/distributions are not as problematic, because they are
much more likely to ship software as the author wrote it (unless it's
ridiculously broken _and_ unmaintained _and_ important.)

Also: why should software authors have to write bug reports to Debian?

~~~
davidw
With 1000's of packages, and 1000's of maintainers, saying that "Debian" likes
to patch "everything" is a pretty big claim.

A lot of the patches that do go in make whatever package integrate better with
the rest of the system.

> Also: why should software authors have to write bug reports to Debian?

Because there's a bug?

> "Even though the piece of software I asked aptitude to install requires
> these libraries to run, they refuse to install it."

At first glance, that sounds like a packaging bug to me, with Rubygems, not
with one of Zed's packages. The best thing to do in those cases is report the
bug, rather than attempt to "make Debian pay" (for the thousands and thousands
of hours of work to give you a free operating system, presumably?)

~~~
heresy
Why should the author report a bug when it's the incompetence of the Debian
maintainer?

The 'gem' command not working after installing the 'rubygems' package does
seem like a bit of a bug, to put it mildly, you'd think a cursory test by the
maintainer would catch that.

~~~
regularfry
As far as I'm concerned, having rubygems available _at all_ in Debian is a
bug. Gems should be repackaged as .debs.

~~~
prodigal_erik
This. The point of Gems and Maven and the CPAN client and all this other crap
is to smuggle code into a production box while carelessly failing to get it
into the platform's database of what's installed where and what depends on it.
They replace one tool that knows everything that's going on, with many tools
each of which can't.

Somehow we got stuck with a critical mass of small-scale developers who never
faced the nightmare of somehow keeping the right tarballs on hundreds of
machines in different roles without good tools, so now we all get to relive
it.

~~~
cemerick
That's hilarious. Users of Maven and CPAN represent a "critical mass of small-
scale developers"? Holy shit.

A similar idea was voiced a few days ago, specifically regarding Java
libraries ([http://fnords.wordpress.com/2010/09/24/the-real-problem-
with...](http://fnords.wordpress.com/2010/09/24/the-real-problem-with-java-in-
linux-distros/)).

My response remains the same: Debian (or what-have-you) is but one distro
among many, and those of us that treat such things as important but
fundamentally commodity deployment platforms have no wish to spend our lives
babysitting whatever package manager happens to be on the operating system
we're using that month/year/whatever. Further, plenty of people have plenty of
good reasons for deploying on Windows and OS X, so using tools that work well
there make sense.

The priorities of the various linux distros aren't everyone else's priorities
– which somehow seems to come as a galloping shock to some.

~~~
billswift
That is why I use Slackware. Installing new software by hand from tarballs is
a bit tedious compared to package managers, but it is transparent, and when I
am done I can be pretty sure it is going to work. I have tried other distros,
even Debian once (for some reason the Debian wouldn't let me sign in as root,
I never did figure out why), but I keep coming back to Slackware - for its
transparency.

~~~
koenigdavidmj
And Slackware's packaging is light enough that it is fairly simple to write
wrappers around gems/CPAN modules/Python eggs. The latter two have prewritten
SlackBuilds that you can use, assuming that the setup.py is not doing anything
too shady.

------
thristian
> But here's the problem, Debian package maintainers don't want to give up
> control to the responsible parties. I would more than gladly make my own
> .deb packages, but they refuse to let me. In fact, I plan on making packages
> for the major Unices in order to head them off. That's what everyone ends up
> doing.

As a user, and amateur administrator of my home machines, I've learned the
hard way that third-party packages supplied by the original software vendor
are, in general, utter crap. They're built against a distro two or three
versions old; or they're built for Mandrake or Ubuntu instead of Red Hat or
Debian; they "install" a tarball to /tmp which the post-installation script
untars in the root directory; they don't have any dependencies declared at all
but crash at startup if you don't have a particular version of libjpeg... if
you're relying on the packaging system to detect file conflicts or outdated
dependencies, third-party packages can be very, very scary.

The single biggest reason I choose Debian (and sometimes Ubuntu) is the
uniformly high-quality packaging. Zed's found one problem package, and a trawl
through Debian's bug tracker will no doubt find others, but the fact of the
matter is that with a single command I can install almost any Free software in
the world, and have it installed and set up within seconds - and Debian has
years of experience figuring out how to make that work. I don't think that's a
thing to be dismissed lightly.

~~~
bad_user
I used Debian on the desktop for 2 years (2008-2009) ... the whole time the
Iceweasel package (their repackaged Firefox) has had its renderer broken,
making some web pages unusable for me ... this wasn't a problem if I manually
downloaded and installed Firefox, and it finally got fixed just before I
switched to Ubuntu.

It does have the best software repository ever, but boy how much it sucks when
something's broken.

------
blasdel
This is in no way specific to Ruby — if you look at the way any sufficiently
advanced software you're familiar with is packaged in Debian, you'll find it
to be completely fucked. Ancient versions, nonstandard configuration files,
random things disabled at compile-time (often for ideological reasons), files
scattered everywhere with the new locations hardcoded, with basic features
broken into separate packages. My favorite is the random patches, which when
they aren't in the service of the aforementioned ridiculousness, are mostly
cherry-picked from current upstream versions to 'fix bugs' without
accidentally introducing features because they're afraid of new version
numbers. When a patch doesn't fit those categories you really have to worry,
because now they're _helping_ (see OpenSSL)

The result is that any program or library that you use directly must be
sourced from upstream, _especially_ if it's less than 15 years old or written
in a language other than C or C++. Luckily pretty much all of the modern
programming language environments have evolved to cope with this onanistic
clusterfuck.

Haskell has more fucked by Debian than any other language I know of — when I
last had to deal with it a year ago there were two broken+old builds of GHC
with different package names and mutually-exclusive sets of packages that
depended on them. On top of that the version of cabal (Haskell's packaging
tool) in the repository was so far out of date that you couldn't use it to
build anything remotely recent (including useful versions of itself), nor
could you use it with anything in Hackage (the central repo).

My old roommate had listened to me bitch about this stuff for years, and
always dismissed me as crazy for thinking that the packaging was fucked
(though he did share my hate of _debian-legal_ ). Last week he called me out
of the blue and apologized — he'd installed Wordpress through Debian and
they'd broken it up into a bunch of library packages, but still left a base
skeleton of random php files and symlinks, accomplishing nothing but breakage
and unportability.

------
btilly
Here is my beef with Debian.

A lot of the software they package comes with unit tests. Those unit tests
have a purpose. They are meant to see whether or not the software as
configured and installed, works.

Debian systematically strips those unit tests out, and _never_ runs them to
see how much stuff they are shipping already broken. Why? Why not package the
unit tests as an optional package, and make sure they have a wide variety of
systems in different configurations running them to notice problems?

I can't count how many times I've tried to install a Perl module from CPAN,
found it was failing its unit tests, installed the Debian package with it, ran
the unit tests, and found that the unit tests said the package, as installed,
was broken. It's not as if the package developer didn't try to tell you you
were missing something. Why throw that information away?

If they did this then they'd automatically catch a lot of problems that
currently get missed. Heck, insert a test for ruby gems that says, "Does this
software start?" They'd have caught this bug automatically.

Until Debian catches up with standard best practices, I completely can't agree
with the meme that they run software better than everyone else. It isn't as if
unit testing is a new idea. It has been pretty mainstream for most of the last
decade. Some were doing it earlier. Perl has been doing it since the 1980s.

~~~
__david__
I don't think it's so cut and dry. In the ideal case, the packages don't need
tests because they are going into the system already tested. _If_ the package
is correct then running the unit tests is just redundant work because you are
getting an identical copy of a package that already tested good with the
appropriate dependencies. I think that's the theory anyway. On the other hand,
theory does not always match reality.

Debian CPAN packages _do_ run the unit tests--they just run them when you
create the package and not when you install the package. In fact, the package
will not build unless the tests pass.

~~~
btilly
Sorry, but it really is that cut and dry. In the ideal case you don't need
unit tests. In the real world, they are useful.

One of the big things that unit tests are supposed to catch are hidden
dependencies. By nature, hidden dependencies are likely to be installed on the
maintainer's machine, so unit tests only help if they are run somewhere else.

As a prime example, the bug that Zed was complaining about was a dependency
problem of EXACTLY this sort. It worked fine on the maintainer's machine. It
didn't work on Zed's machine because he didn't have a key library installed.
No unit test the maintainer can run would catch this. Even a stupid unit test
Zed can run would have flagged this immediately.

Incidentally about the Debian CPAN packages, my experience with them a few
years back was that they were so consistently broken (usually because of
locale issues) that I have a hard time believing that unit tests had actually
been run by anyone. Or if they were, they were ignored.

~~~
__david__
Well, then maybe what they really need is a single server/farm out there that
just goes through all the perl/ruby/whatever modules and installs them and
runs their unit tests. That way you get the same coverage but you don't waste
my time. I, personally, don't want a bunch of unit tests to run when I install
stuff--it's slow enough as it is and it's (usually) completely redundant.

It seems the same to me as slackware or one of the other "compile everything
yourself" distros--they force you (the installer) to do a bunch of redundant
calculations. If someone can compile on my architecture then why should I
_ever_ have to compile it? The same with unit tests--if someone can run the
unit tests on my exact configuration then why should I _ever_ have to run
them?

~~~
btilly
You still don't get it.

If you're installing item after item, then the odds are that at some point
you're going to install the dependencies, and after that the tests will pass
and mean nothing. So you keep on wiping, installing one thing, and testing.
Fine. Now you have cases where package A conflicts with package B and you
never find it. And now what? The number of combinations to test is incredibly
large, and nobody is likely to have tested your exact system.

As for the unit tests, make them available, make them optional. That's OK by
me. I'd opt to use them. Particularly since I learned the hard way that most
of what I want in CPAN either isn't in Debian, or isn't up to date there. So I
install things directly from CPAN. That means that when I install from Debian,
there is a possibility of library conflict that I want to notice.

~~~
caf
Perhaps a compromise would be to have the package-install-time unit tests run
in "testing" and "unstable", but be turned off for "stable"?

~~~
btilly
That would be worlds better than what we now have. But I'd like the option to
turn it on. (And I'm sure others want the option to turn it off.)

~~~
phaylon
I'm not sure what the -dbg packages in Debian/Ubuntu are strictly for, but
wouldn't this be a case for it? If one could post-run the tests by saying
"apttest Foo" or something along those things, integration into utilities to
automatically install test packages and run them shouldn't be a huge hurdle.

~~~
joeyo
I can't speak as to whether or not it would be a good idea for -dbg packages
to run unit tests, but their nominal purpose is that they contain extra
symbols to allow for debugging.

<http://wiki.debian.org/DebugPackage>

------
ComputerGuru
...and people ask me why I write tons of freeware, but make so little of it
open source.

The reason is simple: control. If I can't control every stage of the
development, deployment, and distribution process, I don't want in (yes, I'm a
control freak. No, I don't think necessarily a blemish on my personality, it's
just who I am). If there's something wrong with how my users perceive my
software, it's because of something _I_ did wrong, not because someone took my
hard work and toil and perverted it with their own changes, be it making the
code ugly with nasty function names or dirty hacks (in my opinion, of course)
or if they distribute it in a way that makes users cringe. It's my hard work,
and I deserve to be (a) in control of the user experience, and (b) attributed.

If you're willing to make your awesome utility/code that you've spent 5 years
developing and maintaining fully available to the public, giving up all
control of the end-users' perception of the package, you have a bigger heart
than me. But me, I'm a selfish guy when it comes to my users, and I don't want
anyone to even have the possibility of hurting them. I have _my_ users' best
interest at heart, you probably don't. At least, not to the same extent that I
would.

~~~
VMG
You could still make it open source but craft the license in a way to prevent
redistribution

~~~
davidw
> You could still make it open source but craft the license in a way to
> prevent redistribution

Then it would not be open source.

~~~
narag
There are certain clauses that don't affect the definition of a program as
open source, but would prevent it to be included by, let's say, Debian. I'm
thinking on the old BSD license or the branding problem of Firefox.

~~~
davidw
Debian includes the Firefox code, though. You can certainly say "you can
redistribute my code, but not call it XYZ if you've modified it", that still
lies within the realm of open source.

~~~
narag
That was my point. It's open source and still it's not distributed under the
Firefox brand, so users won't fill bugs in Mozilla's bugtrack.

------
mycroftiv
I sympathize with frustration at poor software packaging, but the proposed
solution seems completely disproportionate to the offense. I also don't see
any real evidence presented for the assertion that Debian's policies are based
on a corrupt motive. What OS is actually pure and innocent of technical flaws
and bad policies? If I recall, openSuse is in bed with Microsoft, Ubuntu fails
to contribute code to important projects, Fedora is really just Red Hat's beta
testing, etc. One of the pains of open source software is that things you
write will probably be messed up in lots of ways by other people. If seeing
software packaged or modified in ways you don't like makes you angry and
advocate retaliation, is that really consistent with the philosophy of a free
software license?

~~~
blasdel
The efforts of both Zed and Debian are based on the same corrupt motive:
_self-satisfaction_

As the upstream developer, Zed wants to make sure that his software is
distributed in a manner that is useful to people and actually works and shit.
This gives him jollies.

As the distributors, the Debian contributors want to fullfil ideological
goals, fit everything into their neat hierarchy, assuage neckbeard sysadmins
that the version number is sufficiently ancient (not that the software is,
given the distro patches), pretend to care about people running it on the PA-
RISC port of Debian GNU/kFreeBSD, wank about licensing, and reach consensus.
This gives self-important teenagers and manchildren all kinds of joillies.

~~~
Mod_daniel
I can't believe this getting upvoted. I wonder how much software you use
_every single day_ thats created and maintained by "self-important teenagers
and manchildren".

------
poet
I've always thought that it was a bad idea for Linux distros to package any
language libraries at all. It seems like a lot of repeated work and your users
will probably end up needing to install things manually in the end. Things
will require the lastest version of gems, new and essential features will be
added, etc. Just give your users a manual on how to install gems (or pip, or
whatever) and let it be. The users of language libraries are exclusively
programmers after all; I think it's safe to assume they can handle library
installation.

That said, Zed is blowing this way out of proportion.

~~~
davidw
> Just give your users a manual on how to install gems (or pip, or whatever)
> and let it be.

Uh... that might work for one language, but for 10? 20? And some poor sysadmin
is supposed to use all these disparate tools to install security fixes too?

And you're forgetting that there are "end-user" programs that depend on those
libraries as well, so they really do need to be packaged up.

> That said, Zed is blowing this way out of proportion.

That goes without saying.

~~~
poet
_And you're forgetting that there are "end-user" programs that depend on those
libraries as well, so they really do need to be packaged up._

I'm not forgetting this. I don't think the user's programs should use the same
libraries (or even the same language install) that the distro's programs
depend on. This way, if the user messed something up (like trying to upgrade a
library), the entire distro wont be hosed. It's a really bad idea from a
stability, security, and design standpoint if you don't keep these two things
separate.

------
Corrado
Actually, the Ruby and Rubygems Squeeze packages have been in quite a flux
lately. One of the more recent Ruby1.9.1 packages broke Rubygems completely
due to some changes in the Ruby 1.9.2 source and they way they handle gems.
The latest version of Ruby has really integrated the Gem package management
system into its core. It has therefor been decided to stop building a separate
Rubygems package for the 1.9.x series and let the Ruby1.9.x package deal with
gems.

Have their been clashes between the Ruby and Debian communities? Yes! Are we
all working toward a solution? Yes! Will it happen instantly? Unfortunately
no, but I think the two groups are now talking and are making some good
progress. Keep an eye out for some good stuff in the future.

------
legooolas
[Disclaimer: I'm not a Ruby developer, but have developed in plenty of
languages, as well as being a sysadmin for a fair while]

IME, separate package-management for each and every language is more painful
than the problem he is describing.

Rubygems, CPAN, whatever other languages use.

As soon as you install things outside of the distro package-manager, it has no
idea what is going on with those packages and so continues to think that they
aren't installed. If there was a way to get the package-managers for language
libraries and the distro to play together nicely and work for dependencies etc
then this would be OK, but as it is it's a bit of a sysadmin nightmare.

Edit: I thought that installing distro-supplied packages into /usr/local was
against the FHS?

~~~
davidw
> way to get the package-managers for language libraries and the distro to
> play together nicely and work for dependencies etc

Yes, hacking the language's package manager to talk to the distribution
package manager is probably the only way to really go about solving the
problem in an elegant way, and even that is not without its pain points.

~~~
legooolas
> pain points

Absolutely, but this problem has been around for a long time, and fighting
with distributions on how to get it to work won't help anyone.

Of course, it also means that if you install a buggy version of a library
from, say, CPAN then you could break distro-supplied packages... I guess it
depends on which you value more: distributor-provided testing or being able to
install out-of-band packages onto your distribution more easily and with
proper dependencies.

------
mgunes
It might be worth re-evaluating this rant in the light of Lucas Nussbaum's
(Debian Ruby maintainer) recent "mythbusting and FAQ" post:

<http://www.lucas-nussbaum.net/blog/?p=566>

~~~
auxbuss
This is far from being a rant. It's a concise exposition on the Debian/Ruby
issues.

Thanks for posting the link, but being labelled a rant, I nearly didn't go
there. Just want to encourage others to read it for an excellent explanation
of the issues at hand from the guy who is doing the work.

~~~
telemachos
I'm pretty sure that "this rant" in the parent refers to Zed's post not
Lucas's. The parent's point being "we should look at Zed's rant again after
reading this discussion by a Debian Ruby maintainer."

~~~
mgunes
Exactly.

------
pbiggar
I think there is an important point here. Debian has a lot of control, and
takes very little responsibility. A similar situation:

<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=380731>

~~~
protomyth
wow, just wow. You would think that if you volunteer to maintain something,
you would actually like what you are maintaining.

------
grandalf
Hypothetical question: Now that rvm has been created, would anyone still use
apt-get to install ruby even if the latest version was supported?

Sure you could argue that yes, ruby should be installed with apt-get and
alternative versions should be handled with Debian's alternatives
infrastructure...

I think this is an interesting case in which version numbers are more than
just version numbers, they are more like sister projects, and they don't fall
neatly into the "conservative and stable" or "bleeding edge and risky" camps
the way maintainers typically view different versions.

From the perspective of a package maintainer, if we had to include an
alternatives list for every dot version of every package, then the distro
would explode in complexity.

Ruby and Python just happened to grow so quickly that their growth didn't
immediately trigger the appropriate response from distro maintainers, and very
quickly the community worked around the problem.

~~~
prodigal_erik
apt-get or yum for ruby? Hell yes. After wasting countless hours on random
versions of crap scattered around some servers' filesystems but not others,
now nothing hits production without being in a package that installs and
uninstalls cleanly and declares everything it relies on being present. That
includes my own code, which we package ourselves.

~~~
__david__
Hear hear. Nothing strikes fear in my heart like a /usr/local hierarchy filled
with a million random versions of random programs. BTW, if you are going to
install something into /usr/local, at _least_ use a cheap and easy packager
like GNU "stow".

~~~
grandalf
I used to think that. But consider:

A user can manage his/her own stuff in ~/bin and add it to path if he/she
wants.

A sysadmin can put things somewhere like /usr/local/experiment33 and make
sweeping changes just by changing the default path, which a user could
override for customization.

------
nakkiel
Obviously, he should spend just a few hours in the shoes of a package manager.
Honestly. What would it be without packages/ports? A mess.

The only people trying to make it bearable here are the package/ports managers
and yet they don't get any kind of reward for their job. They have to come up
with crazy tricks to make things just work because people who write software
are unable to write proper install notes, list dependencies correctly, etc..

This process is heavy, slow and doesn't always produces the expected results
(there's no doubt about it). So people thought it would be great to have just
language-specific package management systems and make it unbearable again.
Alright. I personally never use them and install stuff at hand unobtrusively.

Now, do your job. Go fill a bugreport. Or better yet; fork. This is not
discussing things, this is not helping. Tears don't help.

------
whatajoke
I have never used gentoo. Does gentoo patch packages as much as Debian? Or is
their build process sophisticated to allow parallel install of multiple
versions of say postgresql?

~~~
blasdel
Gentoo avoids making patches specific to itself wherever possible, when it
does it's generally just to make it compile+work, the changes are
straightforward, and they are _applied live on your machine_ in a self-
documenting way.

The build process is also sophisticated enough to be able to 'slot'
potentially-incompatible versions of the same package so that they can be
installed simultaneously. From my recent snapshot of the package repo I can
install 5 different simultaneous major versions of postgres, with a variety of
independent build options.

------
mariana
You know, maybe somebody would want to know the Debian POV on this issue:

* <http://www.lucas-nussbaum.net/blog/?p=566>

* <http://www.lucas-nussbaum.net/blog/?p=575>

Debian is all about stability and volunteer effort.

Whining is not going to fix things.

------
xinuc
I'm okay with Debian packages, unless someone include locusts as dependency
<http://xkcd.com/797/>

------
wazoox
This is an Ubuntu package problem, not Debian. In case you didn't notice,
Ubuntu is built from a snapshot of Debian /unstable/. They call it unstable
for a reason, I guess.

On Debian stable "aptitude install rubygems" followed by "gem install rake"
just works. This looks like a completely gratuitous rant, I've seen Jed Shaw
better inspired.

