

Drowning Good Ideas with Bloat. The tale of pkg.m4. - nkurz
http://tirania.org/blog/archive/2012/Oct-20.html

======
bjourne
There are several wrong things with that post. First of the PKG_CHECK_MODULES
macro checks that pkg-config is present on the system and presents a nice
error message if it isn't. Second, PKG_CHECK_MODULES can be used to specify
several dependencies:

    
    
        PKG_CHECK_MODULES(DEP, foo udev >= 999 neverhere)
    

Then if one or more of the dependencies aren't met, all problems are shown:

    
    
        Package foo was not found in the pkg-config search path.
        Perhaps you should add the directory containing `foo.pc'
        to the PKG_CONFIG_PATH environment variable
        No package 'foo' found
        Requested 'udev >= 999' but version of udev is 175
        Package neverhere was not found in the pkg-config search     path.
        Perhaps you should add the directory containing `neverhere.pc'
        to the PKG_CONFIG_PATH environment variable
        No package 'neverhere' found
    

It may not seem like much, but for someone not used to compiling stuff from
source it means a lot that all missing dependencies are shown at once. A
sloppy shell script to check for the same dependencies would mean that the
user would first see "foo not found" and go and download that package, then
"udev >= 175 required" and when that is fixed the third error and so on.

Also, the error message "You can download mono from www.go.mono.com" isn't
very useful for most Linux users. The correct remedy on distros that ship with
devel packages is to figure out the name of the package that supplies the
include headers and pkg-config file and istall that. pkg-configs error message
aren't that great either, but at least they are easily recognizable.

In my opinion, autoconf is a very misguided build system. It is designed so
that it should have no other requirements on the compiling users system other
than a POSIX environment, a shell and a C compiler. It is a completely
unnecessary constraint because nowadays everyone is able to install great
languages like Python, Ruby, Scheme and many others much more suitable for
writing build configuration systems than shell script. There is no point
anymore in using the M4 language to generate a 1000 line configure script when
a 50 line Python/Ruby/$OtherNiceScriptingLanguage would work just as well.

~~~
klodolph
> It is a completely unnecessary constraint because nowadays everyone is able
> to install great languages like Python, Ruby, Scheme and many others much
> more suitable for writing build configuration systems than shell script.
> There is no point anymore in using the M4 language to generate a 1000 line
> configure script when a 50 line Python/Ruby/$OtherNiceScriptingLanguage
> would work just as well.

First, autoconf isn't a build system. It's just a 'configure' script
generator.

It's nice that the configuration script runs anywhere you can find a POSIX
environment. It's not actually that hard, either; it mostly just runs commands
and checks the result code. The bulk of the knowledge stored in autoconf is
the knowledge of how to write those tests, which tests are necessary, and what
flags you need on various platforms.

I think the big problem here is not that the configure script is a portable
shell script, but rather, that it's generated by an M4 script. Don't forget
that Python itself has a configure script, and writing the configure script in
Python would create a bootstrapping problem (it would add Python to the list
of programs that need to be cross-compiled on new systems).

------
Tobu
So Miguel objects to any use of third-party m4 macros at all? Because these
are the possibilities:

1\. You install from your package manager

2\. You install from a source tarball, which contains a configure script

3\. You install the bleeding edge version from a development repository

4\. You do 3. but somehow you managed to break your autotools install (and I
don't really want to know how you did), or you're running a long-eoled distro
that ships an ancient version of pkg-config that doesn't include the pkg.m4
macro.

The fourth scenario can cause macro trouble, but it takes a sophisticated kind
of user to install from development repositories rather than a better-
supported option (not to mention breaking autotools), so I think it's on those
users to troubleshoot and fetch the missing macro.

------
ambrop7
The "simple" code completely breaks with cross-compilation. This is because
the autoconf macros will first look for the CHOST-pkg-config program (e.g.
armv6j-hardfloat-linux-gnueabi-pkg-config), which will normally be a wrapper
around pkg-config specifying PKG_CONFIG_SYSROOT_DIR to make pkg-config look
for stuff in the target prefix instead of the host. For example, Gentoo's
"crossdev" tool automatically generates such a wrapper.

So if we all just start "simplifying" the code like that, we'll end up with
lots of cross-compile breakages for packages which cross compiled all right
before, and I suppose many people will find the best fix to just revert to
using the autoconf macros again, as opposed to hardcoding the procedure for
finding the right pkg-config.

------
haberman
To me, this rage is misdirected.

The instinct to take a repetitive pattern and replace it a higher-level
abstraction is a good one. It is only because we do this that software is
comprehensible at all.

If a person's attempt at doing this leads to terrible problems in practice,
problems that have nothing to do with the difficulty of the problem itself but
rather with configuration or peculiarities of the implementation, then that is
a sign that your entire system is built on a mountain of sand.

His conclusion is that "pkg.m4 is a poison that is holding us back." No. The
poison is a pile of m4, shell, and autotools that are held together with
bubble gum and chicken wire.

------
klodolph
I came to the conclusion long ago that the world is ready for an autoconf
replacement (note I did not say "new build system" or "replacement for make").
It's too arcane, and it's long since grown too large for M4.

Projects like CMake, SCons, and Jam try to obviate the need to write make
files, but they treat the configuration step as an afterthought.

~~~
Tobu
One thing that improves reliability is in fact to get rid of the configuration
step. Not just the workarounds for broken shells or old Solaris/HP-UX versions
with anemic libcs, but the principle of disabling some features or switching
to alternative implementations when something isn't found. It just creates new
code paths and combinatorial complexity, probably broken because the
developer's environment will never trigger them. Distro packagers already
prefer hard fails to know when they are missing a dependency.

To compensate, the replacement just needs to offer better facilities for
triggering installs of the missing dependencies. That's basically the path
taken by modern languages (Go, Python, Ruby, Node… gnome's jh_build is part of
the same trend): they have tools to fetch source dependencies from the
network, to install in custom prefixes that don't require root, to create
isolated prefixes so that multiple versions can be installed in parallel, to
freeze known-good versions of the dependency graph.

But those little knobs so you can build your own customised version with
decals and your own red spoiler? No one will miss them.

~~~
klodolph
Yes, there are lots of little knobs used by those wacky Gentoo folks, and
those dumb old shells on Unix of decades past. My ideal replacement would get
rid of those checks and assume you have a working POSIX shell and C89.

However, there are also optional dependencies. SoX can use FFTW to make
spectrograms, but it's optional. You can use x264 from ffmpeg but it's not
necessary. Emacs will build with GTK 2 support, GTK 3 support, or neither; and
it will use whatever PNG/JPEG/XML/etc libraries happen to be installed but
doesn't care if they're not present. GCC will use Graphite whet available. A
few programs will happily play audio through libao or Core Audio or ALSA.
There's combinatorial complexity there in theory, but it's not hard to isolate
optional dependencies in practice, most of the time.

And suppose you want to use some feature available on new Linux kernels. It's
just _easier_ to write an autoconf test rather than try to come up with some
convoluted preprocessor test.

There are also important knobs to tune: look at LibGMP, ATLAS, or FFTW for
examples. Old PowerPC Macs could run 64-bit code if you asked but ran 32-bit
by default, so you'd compile GMP for 64-bit and get a huge performance boost.

And some options are only useful to developers, like enabling -Werror (which
should never be enabled by default in a source release). The configuration
step bakes these options into the makefile so you can still just type "make".

You have a good point about hard failing when dependencies aren't present, and
the configuration step allows you to fail early and provide an intelligent
error message.

I'm not convinced that eliminating a configuration step improves reliability.
Just think of how dirt-common autoconf scripts are, in spite of how much of a
pain autoconf is to learn.

~~~
justincormack
A short set of config options works well for most things though, rather than
autodetect. After all most people will be packaging and need to control exact
dependencies. I dont mind reading through a config file to tweak these
settings.

------
jlouis
I am of the opinion that the whole autohell toolset has outlived its
usefulness. It was important back in the day where when there were a million
different UNIX systems. But today, we have something like 5-6 left and most of
those are doing the majority of POSIX right.

It is kind of odd we still are doing this today. It is somewhat of a mistake.

------
Vivtek
To paraphrase Jamie Zawinski, some people, when confronted with a problem,
think "I know, I'll use m4." Now they have two problems.

I know I'd rather have fifty interlocking regular expressions than ever have
to touch m4 again.

------
jff
This from a GNOME/Mono guy--oh the irony!

~~~
klodolph
From "a" GNOME/Mono guy? Read the byline again.

~~~
jff
Oh, I know who he is, I just didn't want to imply that he's the only person
driving GNOME or Mono development.

------
steevdave
Miguel IS a Gnome/Mono guy. Or at least he was a gnome guy, and his
contributions have helped many people all over the world.

