
How to Write Portable C Without Complicating Your Build - ingve
http://nullprogram.com/blog/2017/03/30/
======
antirez
Just to reality-proof that: Redis managed to survive 7 years without autotools
so far, using similar techniques, you just type "make" regardless of the unix
system you want to compile Redis with. No major complexity was faced in order
to do so, however Redis depends on nothing externally if not libc. For
software projects with many external deps it may be worth to use autotools
instead.

~~~
thesmallestcat
Quick, how do you install redis to a different prefix? Everybody knows
`./configure --prefix=...` (or if they forget, there's `./configure --help`),
`make PREFIX=...`, not so much.

~~~
antirez
No way other than "cp redis-server /your/path".

------
vbernat
The article downplays a little bit the value of the autotools. It's not just
about building for 30-year ago hosts, it's also about providing a lot of
convenient tools and providing the user with a known interface. If you write
your own configure, people may expect to be able to pass additional CFLAGS
(not overriding the existing one), setting the path to various elements,
installing in a different root, having certain targets for the generated
makefiles (like dist, clean, install, uninstall), having cross-compilation
work (including crosscompiling to Windows with mingw64 which solves the
toolchain problem of releasing binaries for Windows), ... It's a tedious task
and easy to get wrong.

On the other hand, using the autotools in a modern way is dead easy. You don't
need to add many many tests if you don't intend to support old stuff. You get
access to automake which is a fantastic tool on its own.

Don't read old tutorials, don't look at how big established projects are
doing, look at the Autotools Mythbuster instead (autotools.io) and start with
a minimal configure.ac.

~~~
zbuf
> If you write your own configure, people may expect to be able to [...] It's
> a tedious task and easy to get wrong.

The author's point is to do <em>without</em> a configure step entirely, not
write your own.

A standard Makefile is perfectly capable of implementing these things in a
manner which is clean, reasonably portable and without the level of
indirection that makes things hard to debug.

My own experience is that autotools has evolved to feel less standard than the
modern platforms it purports to smooth over the differences in; I seem to
frequently find that I can't generate a 'configure' for a 3rd party software
or libraries from Git repositories because I of the wrong autotools versions.
Efforts to investigate this by unpicking the various macros etc. provided in
the build have almost always been unsuccessful, leaving me building from
distributed .tar.gz files. With something that feels like such a moving target
after 30 years, I'm glad to see people realising the benefit of a simple
Makefile that's easy to customise for the edge cases.

~~~
zuzun
There's no way you can do what autotools does in a Makefile without
implementing half of autotools yourself. How do you check for headers? How do
you check for functions? How do you test if qsort_r expects

    
    
        (*)(void*, const void*, const void*) 
    

or

    
    
        (*)(const void*, const void*, void*)
    

as function pointer? If people are forced to detect that kind of stuff in
Makefiles, they get the urge to match `uname` against some hard-coded
platforms and that's a terrible solution.

~~~
coldtea
> _There 's no way you can do what autotools does in a Makefile without
> implementing half of autotools yourself._

That's part of the argument: with standards compliant code, you don't need to
do "what autotools does".

~~~
zuzun
ENOTSUP is part of the standard and I do have autotools macros checking if
some function calls produce that error value on the host system.

------
metafunctor
In the past, I've wrangled with autotools more than I'd care to admit. I spent
very many hours indeed perfecting my automake files, configure scripts,
libtool and gettext integrations, making sure you can pass in CPPFLAGS and
LDFLAGS as one might expect, cross compiling, with all the .in and .in.in and
.h.in files, with all the m4 macros, publishing some of my own autoconf
macros, all the while figuring out how to make it all work on Linux, mingw,
the various bsd derivatives, AIX, Solaris, HP-UX, OS X, Visual C compilers,
and what have you.

From the developer's perspective, it is an absolute nightmare – and I consider
myself a seasoned professional with the various unix-like systems and their
shells. Even with this deliberate care, dedication, and time spent preparing
the scripts beforehand, the build usually doesn't work properly out of the box
on a new platform.

My mistake was in thinking that the value in autotools system is that you can
build a system that will compile for anything. I imagined all you need to do
is use (or create) feature tests to sniff out any differences between
platforms, and your code will build and work anywhere.

The real value of autotools is for the user. The user can run "./configure &&
make && make check && make install".

Don't make the mistake I did, thinking that autotools will save you time. It
absolutely will not.

~~~
ufo
I really wish there were an alternative to the autotools that offered that
configure make install interface without the nightmarish developer experiente.
Perhaps by compromising on the portability, if that is the only way.

~~~
js2
CMake, SCons, maybe. This question was closed as not constructive, but the
discussion looks reasonable to me:

[http://stackoverflow.com/questions/600274/alternatives-to-
au...](http://stackoverflow.com/questions/600274/alternatives-to-autoconf-and-
autotools)

~~~
ryandrake
CMake was a step in the right direction, but it, too, suffers from thousands
of under-documented options and fiddly behavior. In a former life I was tasked
with maintaining a CMakeLists.txt for a project with multiple dependent
libraries, binary blobs, open source dependencies that needed to get pulled
in, and the whole thing had to be built for Windows, Linux, Mac, iOS, Android,
and a handful of other lesser-known mobile platforms. I wouldn't wish that on
my worst enemy.

~~~
cardiffspaceman
It's not so bad. I have experience with two slightly different build systems
based on CMake, which had to have customized toolchain definitions and build
Cairo and WebKit. The Cairo build had a custom backend, which we integrated
into Cairo's autotools. Cairo was driven using its autotools via CMake's
"superproject" system.

------
wahern

      If you’re coding to POSIX, you must define the
      _POSIX_C_SOURCE feature test macro to the standard you
      intend to use prior to any system header includes:
    

Nooooooo.....

Take it from someone who religiously keeps many of his very complex libraries
portable across many systems (AIX, FreeBSD, Linux/glibc, Linux/musl, NetBSD,
OpenBSD, Solaris, and others), the _last_ thing you want to do is define the
POSIX portability macros.

Once you do that, you're in for a _world_ of hurt trying to use any extension,
including routines from newer POSIX standards that are almost universally
supported, but because some systems don't claim 100% compliance to the latest
POSIX release, become hidden once you start using the user-definable feature
macros.

Most systems (particularly the BSDs) make _everything_ available by default.
On Linux, however, you should define _GNU_SOURCE; on Solaris, define
__EXTENSIONS__ and _POSIX_PTHREAD_SEMANTICS; on AIX define _ALL_SOURCE; on
Minix define _MINIX. This way, anything you might possibly want to use is
available.

There are a few cases where a native extension conflicts with a POSIX routine.
The classic case is strerror_r on glibc. Dealing with these cases is easier to
deal with than fumbling with feature macros.

Remember, you'll _always_ want to use extensions unless you're writing pure
ANSI C (C90) code. Portable is not the same thing as POSIX compliant. And many
POSIX routines are effectively extensions on systems not yet certified for the
latest POSIX standard.

------
0xfeba
In my experience, I've found that the configuration management aspect of
writing software is the least understood and hardest to learn.

All the books on learning languages I've read just skip it entirely, or
provide a default setup with no explanation. Which is somewhat understandable.
But then the documentation for various projects also do the same thing. I'm
left reading the man pages for various tools that some project says I need and
going "Why do I need this?" on a higher level than the man pages.

And in my professional life it's much the same. On one project we have one or
two people who understand how the system is setup, reject any kind of change
or improvement, and just keep throwing bailing wire at it to keep the process
going.

On my own, partly as a result to institutional intertia, we have an ancient
TFS build controller running a powershell script running gulp running webpack
running babel, and then deploy it using robocopy to network share in IIS
(using iisnode). It's amazing it all works. It often breaks, rarely with the
same error. It also takes 10 times as long as building on my local machine.

------
liveoneggs
I will write this rant here because it will get a bigger audience than any
blog I would have.

Speaking of someone who occasionally ports software to less common
environments I have a few things to say that I was hoping to find in this
article.

First the case for and against autotools:

autotools are _well_ supported by every packaging system out there (freebsd
ports, pkgsrc, apt, rpm, etc) where it's really just a line or three to do the
build in the package definition.

The main things that are not usually thought about by developers but are
_loved_ by porters and packagers (and come out of the box with autotools) are:
DESTDIR support (installing into an intermediate path, so
DESTDIR/PREFIX/usr/bin/myapp) and cross-compilation, which is one of the
biggest strengths of using C in the first place!

The biggest argument _against_ autotools is that feature discovery at build
time is complete bullshit. I hate it. It reduces determinism and ties
execution environment to the build environment.

So if you can support DESTDIR and cross compilation without autotools, go for
it! It's not that much extra work!

\---

The second point I was hoping this article would address is project layout. In
the last few years a nice, standard-ish layout for portable projects has
emerged but package maintainers end up having to teach it to every new (big)
project.

Here are examples where it has gone okay:
[https://github.com/nodejs/node/tree/63243bcb330408d511b3945c...](https://github.com/nodejs/node/tree/63243bcb330408d511b3945c53719425d8b7abb8/deps/uv/src/unix)

[https://github.com/golang/go/tree/964639cc338db650ccadeafb74...](https://github.com/golang/go/tree/964639cc338db650ccadeafb7424bc8ebb2c0f6c/src/net)

Where you are writing to non-portable parts of unix you can build non-portable
stuff into their own files named simply after the platforms and include the
correct stuff higher up in a .h

and an exmaple of it not going so well: [https://github.com/dart-
lang/sdk/issues/10260](https://github.com/dart-lang/sdk/issues/10260)

(fwiw you can find [https://github.com/mulander](https://github.com/mulander)
all over this "teaching" effort, so points to him)

~~~
iainmerrick
Totally agreed about autotools. It's fine (if very slow) when it works, when
it doesn't work right away it's a massive pain.

Sometimes I need to build C code for iOS and Android. Libraries with a nice
clean setup like the author of this article describes are great. Libraries
that use autotools are more trouble than they're worth.

------
newhouseb
One of the nice (perhaps underappreciated) side effects of Rust has been
first-class support for all major platforms. If you're linking against other
Windows bits you still need to think about mingw vs. msvc, but beyond that you
don't need to customize/hack up a build system to get things compiling sanely
on both. I recently switched back to C for a particular project and there's
been quite a bit of reverse-culture shock when it comes to setting up builds &
projects.

~~~
pjmlp
Rust still doesn't have first class support for UWP.

~~~
sqeaky
UWP?

EDIT - I should clarify. I google UWP and it could be Universal Windows
Platform. If so, how is that different than the windows builds that Rust does
support?

~~~
pjmlp
Think of UWP as Windows Longhorn reborn.

It is the same programming model as .NET, but built on top of COM and just the
minimal set of Win32 APIs to support COM based APIs.

It is also sandboxed, just like on OS X.

For Rust to support UWP, it needs to be able to consume COM with the new UWP
semantics and at the same time expose traits as COM interfaces accessible to
any programming language able to consume UWP libraries.

~~~
sqeaky
That doesn't sound like a good idea at all, any of it. Why is microsoft doing
this?

------
stinos
_The nice alternative is MinGW(-w64) with MSYS or Cygwin supplying the unix
utilities, though it has the problem of linking against msvcrt.dll_

You can use MingW's make but compile with cl instead of gcc and that problem
is not present. You will have to check if all compiler flags are supported
though. Or use a wrapper around cl which translates compiler flags, pretty
sure that exists already.

 _My preferred approach lately is an amalgamation build_

That won't scale well though. For large projects you'll really want to use
something as mentioned above, or have seperate Makefiles and VS projects, or
use a build file generator like CMake.

 _edit_ another thing I wanted to add: instead of using

    
    
      #if defined(_WIN32)
      void foo() {
        //win implementation
      }
      #else
      void foo() {
        //unix implementation
      }
      #endif
    

all over the place another option is to split the implementations over source
files like impl/win32.c and impl/unix.c then have the build system deal with
selecting the correct one. Especially when there are more than a couple of
platforms this is much cleaner and more convenient.

~~~
Keyframe
#if defined(PLATFORM)

I would recommend against that as well.

I have a directory structure where there's a module/hello.c (stub or generic,
platform independent code) and then there's module/platform/hello.c which my
Makefile selects over the generic one based on the platform. Primary issue is
that there's some code duplication, and it's hard to keep .h files general
across platforms all the time (not possible all the time), but in general it
does make life a lot easier - especially if platform number grows over time or
certain are removed.

~~~
logicchains
That's quite similar to how Go does it (Go would have something like
hello_platform.c), which in my opinion works quite well. I actually find it to
be one of the nicest things about Go, compared even to Haskell, which for some
bizarre reason uses the C preprocessor (!!) to pepper code with #ifdefs.

~~~
ycmbntrthrwaway
Go took this idea from Plan 9. In fact, Go shares a lot with Plan 9 C compiler
structure.

Related:
[http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful](http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful)

------
ryanpepper
I don't mind Make for small projects, but it can be a bit of a nightmare for
large ones - there are some very unwieldy Makefiles out there. It gets more
complicated when you want to have several libraries and/or executables in a
project, each of which link against different external libraries. I've always
found CMake makes this much easier IMO, and while the language is a bit janky,
it does the job.

~~~
Keyframe
Make is kind of write only. I have this library of mine for image and graphics
processing that I wrote and which I'm rolling through years. In the beginning
I wrote in all this functionality I thought I needed, basically an
architecture of project and it does what I need - I can make a static or
dynamic library, it works on Windows, MacOS, and Linux, it has "modules" which
can be overriden based on if there's a certain subdirectory within a module
with OS/Platform name (basically a compile time multiplatform specialisation)
and several other things.. It works and is rock solid. However, if you've
asked me now what's in there and why and how... Just by glancing at that
spaghetti code in there I know in general what's where, but damnit if I would
be brave enough to make significant changes to it. It works though and it
works great. I must also point out that I use gcc (different versions) and
clang here and there on all three platforms and I change compiler/tools in the
environment itself, not within Makefile.

If I would to do it again (I'm not active developer anymore though - it's more
for personal use now) I would do it again with make. There's certain
straightforwardness to it when writing it, and from my experience there's zero
to almost none maintenance, but ymmv of course. I would probably look into
CMake if I were using vastly different compiler tools (configuration-wise),
but I'm not.

------
_RPM
This guy has some really interesting blog posts about system level
programming.

------
mschuster91
autotools sucks for developers, but as a user I highly value "make uninstall".

Every time I see software without a make uninstall, I have to manually create
a textfile with the stuff the application installed so I can do a clean
upgrade... it sucks.

Hell, most Windows software ships (mostly crappy) uninstallers, and most OS X
software can be uninstalled by dragging the app folder into the Trash, but
there are loads of .nix programs without uninstall support? What the f..k?

Oh, and .nix software using standard build systems has another advantage: it's
usually trivial to package them in .deb/.rpm if you want to distribute e.g.
custom ffmpeg/vlc builds on a fleet of servers.

------
ape4
For the thorny Windows platform: how about using cygwin so you have the shell
and regular Linux commands but use the Visual C++ compiler for a more native
exe.

------
kyberias
Slightly off-topic or meta but still: the layout, CSS styles, fonts, colors
and contrast on this blog (article) are just PERFECT. Rare occurrence.

------
coldtea
> _The man page also documents secure_getenv(), which is a GNU extension: to
> be avoided in anything intended to be portable._

GCC itself is portable, so?

~~~
andreareina
That's like arguing that because cows are vegetarian, so is beef. Portable
means that any system implementing the standard interface can use it;
secure_getenv isn't part of the standard, therefore it's not portable.

~~~
coldtea
No, it's more like arguing that if one is OK with the (lots) of platforms GCC
covers, then it shouldn't matter if some extension is GCC specific. Your
program is still portable for your use cases.

~~~
andreareina
Not everyone uses GCC, or glibc.

------
wahern
I don't like plugging my projects on HN, but in this case I'll make an
exception.

Though I've recently warmed to autoconf, I still agree it's overkill for most
cases. For many years I have, like many others, maintained an ad hoc library
of purely preprocessor-based feature checks that I would copy from project to
project. When recently writing a comprehensive Unix system API module for Lua
I basically ended up with a ridiculous number of feature checks. I broke those
out into a separate project I called autoguess.

    
    
      https://github.com/wahern/autoguess
    

Autoguess is a config.h file that exclusively uses preprocessor-based feature
checks. It's very comprehensive. Much more comprehensive than most need, but
compared to autoconf, or relative to modern C++ projects, all those
preprocessor conditionals are effectively free.

autoguess doesn't provide any compatibility routines--just the detection. (For
compatibility routines see my Lua module, lunix. Compatibility interfaces can
be tricky, and context matters, so I've learned to stay away from trying to
comprehensively "solve" that problem. One needn't look further than Gnulib to
understand the pitfalls of trying to maintain such a beast.)

Most of the code in the autoguess repository is a framework for running
autoconf checks and comparing feature detection results with the autoguess
header. The library is just the single file "config.h.guess".

Autoguess uses the same naming conventions as autoconf, though in the autoconf
universe consistency can be poor and HAVE_FOO names can differ from project-
to-project. However, autoguess recommends to use "#if HAVE_FOO" arithmetic
conditionals, not "#ifdef HAVE_FOO". That makes it possible to override
feature detection macros from CPPFLAGS. Autoconf's rule about using #ifdef is
an archaic solution to a mostly non-existent problem these days--broken C
preprocessors not evaluating an undefined macro as 0 in arithmetic
expressions. (There's a 6 line preamble at the top of configure.ac in my
repository that will fix how autoconf generates config.h files.)

Of course, as new operating system releases are made, the autoguess checks can
become outdated. Though many autoguess checks aren't directly reliant on
version numbers, autoconf feature checking is, I agree, a more robust method.
But autoconf checks aren't immune to regressions, either. There's no
substitute for regularly building your software on various platforms.

Autoguess does much more than detect system and compiler versions, but I've
benefitted greatly over the years from this project:

    
    
      https://sourceforge.net/p/predef/wiki/Home/

------
kodfodrasz
Why are people still so obsessed by C? There are better alternatives for
almost any usecase. Where there are none, there portability is a non-issue
(eg. embedded stuff. although C is also not especially suitable for that use,
yet still prevails, because of the existing tooling.)

~~~
exDM69
Obsessed? I don't think so.

It's just that C has been around for 40 years, it'll stay around for at least
another 40 years. You can still compile source code from 20-30 years ago with
little or no modifications.

C gets the job done fast and efficient.

That said, I'm waiting for a good excuse to learn Rust. Prior to that there
have been very few alternatives to C.

~~~
kodfodrasz
I've seen C used for safety critical software. I've used it for that. I've
seen the insane amount of tooling and support and processes and rules to make
it suitable for the task it is utterly not suitable for.

It does not get the job done fast and efficient, if you consider the
develpment costs. The fast code the compiler generates could be genrated from
other sources just as well, it's not especially the merit of that language.

~~~
exDM69
I write security critical software with C and I know exactly what you're
talking about.

It's written in C because the tooling to analyze and certify it for security
in embedded/automotive/aerospace is targetting C (or C++). For some
industries, Ada might be an alternative.

It may be paradoxical, but you're not going to be able to write safety
critical software in Rust because the tooling to certify it for security
doesn't exist and it'll still take years to get there.

Do I like this situation? No, I do not. Do I think it's a big problem? No, not
big enough that it can't be solved by pouring money and engineering resources
on it.

~~~
adrianN
I also write safety critical software, in C++. I disagree that the necessary
tooling for certification doesn't exist for Rust. You don't need that much
tooling. Measuring test coverage is the hardest part (and Rust doesn't do this
very well currently, but it is possible). Most of the things coding standards
for C (e.g. MISRA) require are irrelevant for safer languages, so you don't
need complex checkers. For static analysis the story is similar.

~~~
jacobush
In safety stuff, not only has the source to be vetted. Also the compiler
building the target binary must be blessed and hopefully well understood.

~~~
kodfodrasz
I have seen a certified compiler (GreenHill) compiling invalid code happily
(not one relying on undefined behaviors), despite its certifications, for
which even an ancient 3.x gcc complained (rightfully!)

These certifications unfortunately often are on the level of the golden
shields CAs provide to customers to put on their sites as sign of
trustworthiness.

So yes, _blessed_ is a nice word for it.

~~~
jacobush
I hate GreenHill C compiler as much as the next guy [insert huge rant] but the
Blessed Compiler of Choice in its frozen buggy state at least usually means
it's a KNOWN ENTITY. We code around the bugs, we compile in -O0 mode etc etc,
we analyze the assembler output to death and so on.

I see these "certified" stamps less as "secure" and more of a huge slowing
down of progress and change - which strangely can translate to more secure
since you know what you are dealing with.

If I can dream, I'd take a "certified" GNU Ada, or Rust, or something over C.
But that is bound to take many years to happen if ever. I think we sooner will
see Rust compiled to C which is fed to something like the abominable Greenhill
so security consultants can pour over the intermediate C output and put it
through the abominable Greenhill.

