

Why developers should care about system packages - garethr
http://morethanseven.net/2011/01/16/Why-developers-should-care-about-system-packages.html

======
tedunangst
I have never shipped unix software that used the system package manager, for a
number of reasons. If you want it to be cross platform, you need to
build/test/ship the fallback tgz manual install anyway. Packages don't save
any work, they only increase the size of your build farm and testing matrix.
The software was usually built against specific versions (often modified) of
3rd party libraries. We can't rely on the system to have them, so we need to
ship them too, so it's not like relying on system packages would reduce our
download size at all. All told, the number of customers who care is very
small, either because people just don't care or because the people who care
don't pay for software.

That was for commercial software of course, but even for open source software,
I'm not sure why it's the author's responsibility to go setup a dozen
different VMs. That really is the package maintainer's job.

~~~
zdw
* The software was usually built against specific versions (often modified) of 3rd party libraries. *

That sounds like a disaster waiting to happen... maintaining a different
version of libraries outside of their main development trunk means that:

1\. Extra steps when a new main version comes out - you have to repatch your
differences in.

2\. #1 incurs a delay, thus a longer time that your customer are running
without patches, which may be a security issue.

3\. You won't do #1 at all. Thus you become reliant on old, unmaintained,
buggy, security-holey libraries.

The only time I can think this is acceptable is to fix problems that upstream
won't fix, or are platform specific (ie, not with your custom software, but
with the OS underneath everything).

~~~
tedunangst
Patching in new versions isn't usually a huge deal, it's just not worth the
trouble. Let's say we're using version 1.2, and we fixed a few bugs in it over
the past year. Now version 1.3 is out. Should we upgrade?

Maybe 1.3 fixes a few bugs we haven't encountered yet. But that's unlikely.
We've been using it for a year. Far more likely, as in practically a
certainty, is that 1.3 will introduce some regression that won't be discovered
until it affects some customer.

Not every bug in every program is a security vulnerability, and framing the
issue that way distorts things.

~~~
ldng
Well, if you had uptreamed your fixes against 1.2, they would be in 1.3,
wouldn't they ? I find it a poor excuse for not upgrade assuming API and ABI
are stable.

It's just laziness. It's easier because that way you don't have to argue with
Management. But you also don't educate on the necessity of fixing security
issues. Hooray for 1.4 JVM still in production for instance. And it's
perfectly ok. Until some breach happens .....

~~~
regularfry
Not every upstream will take your patches. Not every upstream _can_ take your
patches.

------
SwellJoe
I consider it a significant black mark on a product, open source or otherwise,
if it doesn't have native packages for the most popular Linux distros. It's
even better if they provide a yum or apt repository.

It _is_ a challenge to do so, but if we can do it for our products (which have
vast and complicated dependency chains and need to perform a huge amount of
post-configuration in order to be easy for non-technical users to use), damned
near any other project ought to be able to do it.

The vast majority of really ugly security issues I've seen have been due to
people having installed something from source, sometimes years ago, and not
realizing they're running exploitable software because their package manager
tells them they're up to date. This reason alone should be enough to keep
people using the native package manager for as much as possible. But there are
many other good reasons, a few of which have been touched on in this article.

~~~
mapleoin
Collaborating with distribution packagers is a lot better than maintaining
your own repositories. Packagers should always be up to speed with the latest
guidelines and best practices for their distribution. I have often seen badly
built packages by upstreams.

~~~
SwellJoe
That's not feasible in a lot of cases. Commercial software, for instance, is
not generally welcome in distribution repositories.

~~~
regularfry
No, but running a public repository for a given distribution is the way
forward in that case, and that does involve knowing the lay of the land.

~~~
SwellJoe
Which is exactly what I said in my previous comment.

~~~
regularfry
Which is what I was agreeing with. Vociferously :-)

------
jolan
Your post boils down to something like "developers should ship Debian packages
because I use Debian and all my favorite software packages are out of date or
non-existent".

Maybe you should move away from a distro that freezes for 6 months to roll a
stable release which won't get anything more than security/crash fixes for 2+
years.

~~~
garethr
That wasn't my aim. In fact I'm not using Debian at the moment as I indicated,
I just know the commands better to give examples. This isn't about a specific
distro, it's about the huge benefits of system packages and their toolchains
in general. I'm not even really concerned with official packages and the
official blessed package repos.

~~~
jolan
I've created hundreds of packages for OpenBSD and you just seem pretty
gullible about the whole process.

You can't just write about the packaging process and hope people come forward
to volunteer to do the actual work. There's a large amount of thankless sweat
equity when you provide packages. People whine when things are broken and
rarely send praise when it just works as it should.

If you really want to change the world, start with creating a Ruby apt
repository for Debian/Ubuntu/whatever you use that people can actually deploy
in production.

------
regularfry
For my money, here's what needs to happen (from a Ruby perspective; salt to
taste):

1\. Debian and Ubuntu stop packaging any Ruby libraries which aren't
dependencies of an actual end-user application. If rubygems itself is a
library dependency, package it but _take away the gem command_.

2\. Make a public policy saying "If you want to do development, use RVM," and
have metapackages with the dependencies for building each RVM ruby (so ruby-
dev-mri, ruby-dev-jruby, and so on). Make sure that users know that if they
think they need to run "gem install foo", then they need RVM.

It'd be nice for RVM to be able to build system packages, but I don't think
it's necessary: there are already systems (nascent, but improving) for
converting gems to system packages which handle dependencies nicely.

Is there anything wrong with this picture?

~~~
garethr
RVM is great. But by design it's a solution to compiling software on every
machine. This simply doesn't work for lots of sysadmins, especially with lots
of machines - it means installing a complete build environment which opens up
much more evil.

~~~
regularfry
Well, yes. And that's the effective situation we've got at the moment.

What I'm proposing is a full acknowledgement of that: if you want to live on
the bleeding edge, then that should be supported as well as possible, but a
very clear delineation should be drawn between the responsibilities of the
distribution and those of the developer/sysadmin. RVM does that very cleanly,
even if you only use it with the system ruby.

------
PKeeble
One of the reasons software gets released in scripts rather than packages is
because the root package installation mechanism is root accessible only. Most
companies restrict root access and package installation is blocked by proxies.
The combination of which means that scripts that can be downloaded separately
and run are much easier for enterprise developers to use and get started with.

Until the average enterprise realises that packaging and installation needs to
be done better we will always have the need for mechanisms that hack around
the problem that system admins cause for the users of the servers.

~~~
ldng
I agree, but developers have enough on their plate and knowing how to
administer your workstation isn't the same as administrating a server.

That said, allowing user system wide installation could be possible. I think,
at least under Linux, combining selinux and cgroup to provide sandboxed system
wide user install could be possible but you'd possibly need to bypass classic
unix directory ownership and permissions. That might require to much work or
not even be impossible.

Anyone here with enough knowledge to confirm or refute my feeling ?

------
joevandyk
Is there a decently written guide for creating ubuntu packages? The package
maintainers guide confuses the heck out of me and it's way too long and
disorganized.

~~~
rmc
Theres a great programme called checkinstall that gets you 90% there. Rather
than running 'make install' run 'checkinstall', or rather than 'sudo python
installscript.pt', run 'sudo checkinstall python installscript.py'

~~~
garethr
Checkinstall is brilliant: <http://checkinstall.izto.org/>

It's also guilty of being unattractive to the developers I'd love to see use
it. The site says "Recent News: Dec 26th, 2009" and then concentrates on a
NOTE TO SLACKWARE 8.0 USERS.

Showing the hello world of checkinstall would be much better imho, and it's
what developers from the web world expect.

~~~
regularfry
Checkinstall of Python and VirtualEnv is the only sane way I've found to do
python development and deployment when you're developing on Maverick and
deploying to Lenny.

------
njharman
devs, esp polygot/startup devs, need sysadmin xp/knowledge.

