
Lessons from the Unix Stdio ABI: 40 Years Later - polm23
https://fingolfin.org/blog/20200327/stdio-abi.html
======
moonchild
> FreeBSD, NetBSD, and OpenBSD today are all still limited to a short, which
> on current mainstay processors is a signed 16-bit quantity or 32767

Why did they not take advantage of the move to 64-bit to change that? (Or, at
the very least, more recent architecture ports like riscv.)

~~~
jandrese
Some of that is the developers being practical. If they went through and
updated all of the ABIs it would take years (it took years to update just
time_t to 64 bits!) and people would be constantly hounding them to just let
them use their 8GB of memory. Stdio is just one of thousands of cases where
decisions from the 70s no longer make sense on modern machines. Plus every
change you make is going to break some app somewhere, so if you change
everything at once it causes chaos in the userland.

This is one reason why developing new languages is so appealing. You don't
have to worry about breaking old code or maintaining workarounds for long-
extinct machine architectures. Of course that's ruined the first time someone
writes something in your new language.

~~~
moonchild
I can't imagine there's any api-level dependency on the fd field being 8 or 16
bits. The size of the struct is going to change anyway, because it has
pointers and those are going to change.

Even new languages frequently rely on libc and the FILE struct, underlying
whatever higher-level abstractions they have.

~~~
jandrese
It can be exposed in surprising ways, like the select() macros. That's the
point of the article. You would think there would be no good reason for an
application to mess with that directly, and then you change it and discover
all of the applications that did so anyway.

~~~
wahern
Those environments support more than SHRT_MAX descriptors in a process. It's
only fopen and fdopen with this quirk. Looking at the OpenBSD source code,
both fopen and fdopen explicitly check the descriptor value (which is an int,
locally) and return EMFILE if it's above SHRT_MAX. See
[https://cvsweb.openbsd.org/cgi-
bin/cvsweb/src/lib/libc/stdio...](https://cvsweb.openbsd.org/cgi-
bin/cvsweb/src/lib/libc/stdio/fopen.c.diff?r1=1.5&r2=1.6) and
[https://cvsweb.openbsd.org/cgi-
bin/cvsweb/src/lib/libc/stdio...](https://cvsweb.openbsd.org/cgi-
bin/cvsweb/src/lib/libc/stdio/fdopen.c.diff?r1=1.5&r2=1.6)

Also, maybe select is a bad example as the type of struct fd_set is unrelated
to the type used to store a descriptor. It's just a bitfield which can be any
arbitrary integral type. On OpenBSD it's uint32_t. Perhaps you had in mind the
internal array size used for the bitfield, which is a commonly encountered
landmine. But that applies to select specifically, and Linux has the same
problem to the same degree.

poll might be a better example, but struct pollfd already uses an int for the
.fd member in all those environments.

pid_t and the value range of an exit status code are open issues currently
causing pain not just for ABI compatibility but also API compatibility, both
in practice (headaches with Linux syscalls) and with POSIX (bugs and debate on
the committees). pid_t is a fairly standard issue--there are several APIs that
take a pointer to pid_t, but also pid_t is conflated with int in some APIs (I
forget which). Exit value is more problematic because a program can return an
int-ranged value, but wait(2), waitpid(2), etc, take a pointer to int for the
status argument, which encodes not only the exit value but additional metadata
like the reason for exiting. Moreover, traditionally the exit value has always
been encoded in 7 (or 8?) bits in the status value. These issues also bleed
over into shell semantics. The next version of POSIX will address this by
adding some constraints and explicit caveats to the permissible exit value
range.

Another interesting case is sigset_t, which must be statically sized. But
arguably that's more a feature than a bug. Attempts at substantially
increasing the signal number range (e.g. with real-time signals) turned out to
be bad for other reasons and good examples of when dynamic limits can be bad
design.

------
bogomipz
The article states:

>"Here, while there are a number of members which have the same types, the
layout of these such as the order of the members is rather different. There’s
nothing wrong with that; however, it’s pieces like this that are part of each
system’s ABI."

The author linked to the System V ABI and the processor-specific ABI docs and
in the "Further Reading" section has links to the stdio header files. Are the
system ABIs that are not covered by the System V ABI and the processor-
specific ABIs always defined in header files then? There's no singular or
centralized spec?

------
hootbootscoot
This is a funny and informative piece.

