
Why undefined behavior may call a never-called function - pavel_lishin
https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html
======
kps
I have become convinced that the current screw-the-programmer interpretation
of ‘undefined behaviour’ was not intended, or even imagined, by the original
ANSI C committee. Within the committee's mandate to ‘standardize existing
practice’, it was simply an acknowledgement that C compilers translated
straightforward C code into straightforward machine code without adding safety
checks, and that simple code might —​ in some circumstances on some machines —
do very strange things.

In the Rationale, the committee listed places where they intentionally
diverged from existing practice. They considered “the most serious semantic
change” to be requiring value-preserving (vs unsigned-preserving) integer
promotion. They didn't mention ‘undefined behaviour' at all.

During the standards process, Dennis Ritchie described the ‘noalias’ proposal
as “a license for the compiler to undertake aggressive optimizations that are
completely legal by the committee's rules, but make hash of apparently safe
programs”. That's exactly what ‘undefined behavior’ has turned into. If anyone
had foreseen that at the time, the reaction would have been the same: “Noalias
must go. This is non-negotiable.”

~~~
pcwalton
The original ANSI C committee had no idea about modern optimization pipelines.
If people had continually pushed back against undefined behavior back then,
there's a good chance that by 2017 the result would have been that C would be
_dead_ , replaced by a language that allows for modern optimization
techniques.

~~~
kps
You're the last person I'd have expected to make that sound like a _bad_
thing.

While benchmark games played a part in the modern ‘undefined behavior’, I'm
not so sure that it would made much difference to adoption of the language.
Consider Linus Torvald's well-known rant as a point in the opposite direction.

In the universe where ‘undefined behaviour’ had been clearly specified as
‘what you write is what you get’, C might have gone to consistent use of
optimization- _enabling_ annotations, following ‘const’ (and later
‘restrict’). Along those lines, ‘volatile’ was ANSI's other big mistake, as
creating it broke existing code; I now think that should have been the
default, with fragile-optimizable status being explicitly indicated by
extending the use of ‘register’ keyword.

~~~
pcwalton
It's not just benchmark games. It's people's real-world code.

Look at how often folks on HN complain that the Web platform is useless
because JS is slow. C without optimizations is just as slow if not slower.
Compiler optimizations are so good that people don't realize how much they
rely on them.

Linus was wrong when he complained about the compiler treating null
dereference as undefined behavior. This is a very important optimization,
because it allows the compiler to omit a lot of useless code in libraries.
Often times, the easiest way to prove that "obviously" dead code can never be
executed is to observe that the only way it could be executed is for null to
be dereferenced.

Opt-in optimization keywords wouldn't scale to the sheer number of
optimizations a modern compiler framework performs. Restrict hasn't been a
success because it's too subtle of an invariant. It's the kind of thing
compiler authors, not application developers, should be thinking about.

~~~
sillysaurus3
This is an important point, and it's why the "C should die" crowd is hard to
take seriously. They've even started labeling people who use C as somehow
morally suspect, as if we're bad people for choosing to use an unsafe
language. We're knowingly putting people in danger! Right.

It's strange that the word "unsafe" has tainted people's thoughts so
dramatically. Like calling torrenting music "piracy."

~~~
pcwalton
I'm not endorsing C. Don't use C for anything you need to be secure.

~~~
sillysaurus3
I need Emacs to be secure. It's written in C. It interfaces with the internet.

Ditto for Bitcoin. It's the basis of a new financial system. The core software
is written in C++.

Same for Linux. C.

Prejudice generally isn't helpful, and it's a bit strange that you can
recognize C's merits while also decrying it.

~~~
pcwalton
I haven't been "recognizing C's merits" either. In fact, the real reason
behind this problem is that C is not type safe, so optimizations (such as the
one in this very article!) that are perfectly fine to do in type-safe
languages are not possible to do in C without aggressively "exploiting"
undefined behavior.

------
lisper
This is _literally_ a religious argument. No sane person would consider this
acceptable behavior if not for the fact that there is a holy text ("the
standard") that says it's acceptable. Well, it's not acceptable. It is no more
acceptable than, say, a car that explodes if you push the wrong button at the
wrong time, which would be clearly unacceptable even if there were a document
blessed by a standards committee that said otherwise. Faulty code can
literally make things blow up in today's world, so there is literally (and I
really do mean literally) no difference between these two scenarios. It is
truly a sad reflection on the state of our profession that we are even
spending time arguing about these things instead of fixing the standard so
that the language it defines is actually useful for writing programs rather
than just a source of material for games of intellectual one-upsmanship, to
say nothing of myriad real-world problems. You'd think that decades of
security breaches caused by buffer overflows would make people think, "You
know, it's 2017. Maybe array dereferencing without bounds checks is a bad idea
even if it does let my code run a little faster." Alas.

~~~
a_t48
> You'd think that decades of security breaches caused by buffer overflows
> would make people think, "You know, it's 2017. Maybe array dereferencing
> without bounds checks is a bad idea even if it does let my code run a little
> faster." Alas.

And there are dozens of languages that will let you sacrifice that bit of
speed for some safety.

~~~
lisper
If you meant to imply that this is not a problem because there are other
languages one can use, I disagree. C holds a unique position in the computing
world. There is an enormous corpus of C source code out there, and more is
being written all the time notwithstanding that C as currently specified in
not a sane language. So what C compilers do matters whether you like it or
not.

~~~
umanwizard
And why do you think that is the case?

~~~
lisper
Inertia. It's very hard to replace infrastructure.

------
kazinator
The problem here is that the behavior is in fact defined. It is not defined by
ISO C, so it is "(ISO C) undefined behavior". But requirements for C program
behavior do not only come from ISO C. It takes more requirements than just
those from ISO C to make a complete implementation.

On "modern", Unix-like, virtual memory platforms, we have an understanding
that the null pointer corresponds to an unmapped page, and that this is the
mechanism for trapping null pointer dereferences (at least ones that don't
index out of that page).

A compiler which interferes with this by replacing valid null pointers with
garbage at point where they are dereferenced is running afoul of this
_platform_ requirement.

Look, the generated code is not even pretending to use the value of the null
variable. We cannot reasonably call this machine code a translation of the
program's intent into machine code.

------
beatbrokedown
If you make one small change to this file you can cause clang and gcc to both
prevent this from compiling if you are using warnings.

    
    
      namespace {
        void NeverCalled() { Do = EraseAll; }
      }
    

or marking NeverCalled as static itself.

Results in warning: unused function NeverCalled.

In general this is best practice for functions defined and used in a single
translation unit.

~~~
saalweachter
That's a vital part of the setup.

You could hypothetically link this compilation unit against another unit which
included:

    
    
      void NeverCalled();
      struct A {
        A() { NeverCalled(); }
      };
      A a;

~~~
eridius
You don't even need C++, you can just use C and __attribute__((constructor)).

~~~
pjmlp
Which is a GCC specific extension and not ANSI C.

------
tyingq
Interesting choice of code for a demo. I wonder if anyone hosed themselves
running this.

It seems gnu rm has "-preserve-root" as a default, but that's not guaranteed
to be on every rm.

~~~
ramshorns
If it's run as an unprivileged user rather than root, it won't be able to
delete the whole system. I'm pretty sure that means it won't do anything, but
I don't feel like trying it.

~~~
tyingq
>I'm pretty sure that means it won't do anything

Assuming an rm without the "-preserve-root" default, it would remove
everything that it had permission to. So, eventually, for example, it would
wipe your home directory.

I suspect this to be the case for OSX, Alpine Linux (or other distros that use
busybox), probably some of the BSD distributions, etc.

~~~
jwilk

      $ uname -sr
      FreeBSD 11.1-RELEASE
    
      $ rm -rf /
      rm: "/" may not be removed
    

No idea about other BSDs.

Busybox's rm is indeed happy to nuke /.

~~~
jwilk
Also:

    
    
      $ uname -sr
      OpenBSD 6.1
      
      $ rm -rf /
      rm: "/" may not be removed

~~~
jwilk
The latest POSIX standard actually requires this behavior:

 _If […] an operand resolves to the root directory, rm shall write a
diagnostic message to standard error and do nothing more with such operands._

Source:
[http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm...](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html)

------
valuearb
I wrote in C for 20 years and now code in Swift (which I love). Reading this
reminds me that Chris Lattner created Swift because he wanted a language to do
more aggressive compiler optimizations than he could do for C.

------
IncRnd
rm -rf /

Using this type of code to show unintended consequences is itself a hotbed of
unintended consequences!

Just put in code that prints "pwned" instead of running code that would delete
someone's system or home folder.

~~~
apk-d
Do any up-to-date operating systems actually run this without the _\--no-
preserve-root_ flag?

~~~
pksadiq
btw, this would work with slightly older rm from GNU coreutils:

    
    
      rm -rf --n /
    

any GNU long option (getopt_long) for any command can be shortened as much as
there isn't any ambiguity. And hence the above (this is fixed in latest GNU
coreutils release).

------
sbov
If the compiler can take advantage of undefined behavior to optimize a
program, what is keeping the compiler from also warning you that it is doing
so?

~~~
jjnoakes
Because, after inlining and macro expansion and other passes, you'd get
warnings about everything under the sun and they would be useless.

~~~
kazinator
"I'm statically assuming that a pointer that can never be anything other than
null actually refers to NeverCalled" is definitely worth a diagnostic.

~~~
jjnoakes
The compiler doesn't know the pointer isn't set, only the linker may know
that. So you'd have to put a whole lot more smarts into multiple tools to get
that warning.

I'm not convinced it is useful.

~~~
kazinator
The point of the warning is precisely _that_ the compiler is making a
dangerous assumption without proof.

More smarts is needed only to eliminate false positives occurrences in the
warning.

~~~
cesarb
> the compiler is making a dangerous assumption without proof.

The whole issue is that, from the compiler point of view, it has a proof! It
can prove from the language rules that the pointer can only have NULL and
EraseAll as its value; since a call through the NULL pointer is invalid, at
that line the only value left is EraseAll; QED.

It might not be the proof you wanted, since you disagree with the premises,
but it's still a valid proof.

~~~
kazinator
It isn't a valid proof, because it's perfectly possible that the variable has
a null value and that the call is invalid.

Detection of that null value is already there and essentially free of charge.

The implementation is going out of its way to _prevent_ an instance of
undefined behavior from being detected, without providing a useful, documented
extension in its place, and in a situation when the detection costs nothing.

~~~
jjnoakes
If the variable has a null value and the call is invalid then the compiler
isn't required to compile it to anything specific; this includes the idea that
the compiler isn't required to compile it to a jump-to-address-zero.

~~~
kazinator
> _the compiler isn 't required to compile it to anything specific_

That might be true if the ISO C standard were the only source of requirements
going into the making of that compiler; it isn't.

There are other issues.

Obviously, the compiler __is in fact compiling it to something very specific
__. It 's not simply an accident due to the situation being ignored that the
indirect call gets replaced by a direct jump. The translation is deliberate.

From ISO C: _Possible undefined behavior ranges from ignoring the situation
completely with unpredictable results, to behaving during translation or
program execution in a documented manner characteristic of the environment
(with or without the issuance of a diagnostic message), to terminating a
translation or execution (with the issuance of a diagnostic message)._

This is definitely not the result of "ignoring the situation completely";
ignoring the situation completely means translating the code earnestly and
subsequently the translation doing the indirect jump attempt through a null
pointer. That's what it means not to do anything specific. It's not
terminating the translation, so it's not the third kind of behavior. So it
must be "behaving during translation or programing execution in a documented
manner characteristic of the environment". I don't see how this is a
characteristic of the environment; nobody can claim with a straight face that
this is environmentally dictated!

Also, the undefined behavior never actually occurs. A pointer variable may be
null; null is a valid value. Undefined behavior only ensues when the attempt
is made to dereference. That's not actually happening in the program; the
translation is devoid of any attempt to dereference null. The idea that the
null pointer dereference caused the translation and the subsequent call to
that function requires a perverted view of causality in which later events
cause earlier events.

~~~
jjnoakes
There are two possibilities the compiler is faced with, that the function
pointer is null or that the external function was called and the function
pointer is set to a concrete function.

Since the null pointer option would lead to undefined behavior at runtime (no
one is confusing cause and effect here) the compiler can ignore this case.

That leaves only one choice, and that's the result of the compilation.

It can be surprising but it makes sense.

------
iainmerrick
I've been playing around with Clang and I can't find any flags that avoid this
problem, other than -O0.

Oh, but if you add a second "NeverCalled" function that sets the pointer to
something else, the optimization is skipped and you get a bus error at
runtime.

No flags I can find, not -Wall -Wextra -Werror, warn me at compile time that
anything unusual might be going on.

This is _weird_. I understand the explanation of the optimization, but a)
isn't this rather unreasonable behavior? and b) couldn't there at least be a
flag that skips this optimization, without having to disable _all_
optimizations?

Maybe this UB propagation / dead code elimination idea is so deeply ingrained
in Clang's design that it would simply be impossible to selectively remove it.
If so, that seems very unfortunate.

------
userbinator
I suspect this is another critical point:

 _as NeverCalled may have been called from, for example, a global constructor
in another file before main is run_

clang doesn't analyse across module boundaries --- even when it theoretically
could --- so it doesn't know for certain that NeverCalled() is indeed never
called.

Throughout the years I've grown increasingly displeased at how compilers
handle even trivial cases like this[1][2][3], and in particular their
treatment of UB[4][5][6][7]; as one of the comments on the article implies, UB
was unlikely intended by the standard's authors as a "you _should_ let the
most inane things happen" but more as a "do what makes the most sense".

A more sensical approach to analysing this program, e.g. as employed by a
human, would be to see that NeverCalled() is the only function that can write
Do, _but then further ascertain whether it is actually called_. Since this is
the entire program, it's trivial to see that it's not, and thus that
possibility should also have been removed from the set of possible values for
Do. Thus, Do can neither be EraseAll nor 0 --- so a "contradiction" has
occurred, the code is likely bugged or the programmer intentionally wants the
UB, and the sane choice at this point would be to forget about trying to
optimise and just generate the obvious code. "I can't figure out how to
optimise this, so I'll do the simplest thing that works."

The question then becomes, why can a human see something so straightforward
but the compiler can't? I think that's the deeper issue here with how the
compilers like gcc/clang today work --- they're too opaque and complex, and
their authors take The Holy Standard as gospel while ignoring the practical
realities of their decisions.

Unfortunately a lot of programmers have gotten the notion that they can rely
on the compiler to do "amazing" optimisation, and therefore they can write
horrible code, leading to horribly unintuitive and "overly aggressive"
optimisation like this.

I'm sure that me, along with quite a few others, have some ideas for how to
make a C/C++ compiler which is both powerful in optimisation and code
generation, but more predictable and "obvious" in terms of UB. Unfortunately,
I'm also sure that we don't have the time to do it.

[1]
[https://news.ycombinator.com/item?id=15006090](https://news.ycombinator.com/item?id=15006090)

[2]
[https://news.ycombinator.com/item?id=9397924](https://news.ycombinator.com/item?id=9397924)

[3]
[https://news.ycombinator.com/item?id=15188416](https://news.ycombinator.com/item?id=15188416)

[4]
[https://news.ycombinator.com/item?id=11147598](https://news.ycombinator.com/item?id=11147598)

[5] [http://blog.metaobject.com/2014/04/cc-
osmartass.html](http://blog.metaobject.com/2014/04/cc-osmartass.html)

[6]
[https://news.ycombinator.com/item?id=7960219](https://news.ycombinator.com/item?id=7960219)

[7]
[https://news.ycombinator.com/item?id=9809885](https://news.ycombinator.com/item?id=9809885)

~~~
connorcpu
I assumed this was a side-effect of devirtualization. Obviously indirections
are slower, so if the compiler can look at a dynamic call and realize that
there's only 1 possible function it could be calling right there, that's a
win. Only my 2c

~~~
mikeash
Exactly. This isn't "fuck the programmer if he fucks up," it's "let's try to
do really good optimizations."

It's really nice to be able to use abstractions that cost nothing because the
compiler is smart. In this particular case, you might have a function pointer
that exists for future expansion, but which currently only ever holds one
value. In a case like that, it's really nice if the compiler can remove the
indirection (and potentially go further and do clever things like inline the
callee or do cross-call optimizations).

The other piece of this puzzle is straightforward data flow analysis. The
compiler knows that there are only two possible values for this function
pointer: NULL and EraseAll. It also knows that it can't be NULL at the call
site. Thus, it must be EraseAll.

For every person complaining that the compiler is screwing them over with
stuff like this, there's another person who would complain that the compiler
is too stupid to figure out obvious optimizations.

I'm very much in favor of making things safer, but I don't think avoiding
optimizations like this is the answer. C just does not accommodate safety
well. For this particular scenario, the language should encode the nullability
of Do as part of the type. If it's non-nullable, then it should require
explicit initialization. If it's nullable, then it should require an explicit
check before making the call. The trouble with C isn't clever optimizers, it's
that basic things like nullability are context-dependent rather than being
spelled out.

~~~
nshepperd
> It also knows that it can't be NULL at the call site

Ah, this is obviously some strange use of the word "can't" that I wasn't
previously aware of. Or possibly of "be" or "at".

The pointer clearly _is_ NULL at the call site. Observe:
[http://lpaste.net/358687](http://lpaste.net/358687). Hypotheticals about the
program being linked against some library that that calls NeverCalled are just
that, hypothetical. In the actual program that is actually executed, the
pointer is NULL.

In what sense is the function pointer "not NULL", then, given that – in what
one might call the "factual" sense – it _is_ NULL?

~~~
mikeash
"Can't" here means that your program is not well-formed otherwise, and the
compiler assumes well-formedness.

I assume you don't like that, but I wonder if you'd apply that to other
optimizations? For a random example:

    
    
      int x = 42;
      SomeFunc();
      printf("%d\n", x);
    

Should the compiler be allowed to hard-code 42 as the second parameter to
printf, or should it always store 42 to the stack before calling SomeFunc(),
then loading back out? SomeFunc might lightly smash the stack and change the
value of x, after all.

~~~
nshepperd
Hardcoding 42 as the parameter to printf here is far more defensible for
several reasons. Here's one: the value actually _is_ 42, and assuming that it
continues to be 42 doesn't require the compiler to hallucinate any
_additional_ instructions outside this compilation unit.

There's a difference between assuming that a function like SomeFunc internally
obeys the language semantics for the sake of code around its call site (this
is the definition of modularity), and assuming that because the code around
the call site "must" be "well-formed" this allows you to hallucinate whatever
code you need to add elsewhere to retroactively make the call site "well-
formed" (this is the definition of non-modularity).

~~~
mikeash
What's the difference between assuming that a function you call will obey the
language semantics, and assuming that the function that calls you will obey
the language semantics? That's the only difference I can see.

~~~
nshepperd
> assuming that the function that calls you will obey the language semantics

That's not what I said.

What the compiler is doing in this NeverCalled example is observing: \- that
the code in the current compilation unit is not "well-formed", but \- that the
compilation unit can be "rescued" by some other module that _could_ be linked
in, _if_ that other module did something specific, and therefore concluding
that it should imagine that this other module exists and does this exact
thing, despite the fact that such module is in fact entirely a hallucination.

This is very different from simply assuming that a thing that in fact exists
really does implement its stated interface.

Here's a different example:

    
    
      #include <stdio.h>
    
      typedef int (*Function)();
    
      static Function Do;
    
      static int Boom() {
        return printf("<boom>\n");
      }
    
      void NeverCalled() {
        Do = Boom;
      }
    
      void MightDoSomething();
    
      int main() {
        printf("Do = %p\n", Do);
        MightDoSomething();
        return Do();
        printf("after Do\n");
      }
    

In this case, it is _possible_ that MightDoSomething could call NeverCalled,
and that's one way this module could rescued from not being "well-formed".
Should the compiler assume that MightDoSomething calls NeverCalled at some
point then? No, that's absurd. There's nothing about the "void()" function
interface that obliges such a function to clean up after you if write code
that dereferences a null pointer or divides by zero.

We trust that a random void() function won't smash the stack and overwrite
local variables, because that's a reasonable interface for a function to have.
That's composable. That's different from expecting it to do "whatever it
takes" to fix local undefined behaviour.

~~~
mikeash
When you say "its stated interface," are you referring purely to the
prototype, or are you referring to documented behaviors, or what? Because it
seems reasonable to me for a function with no parameters to have prerequisites
before you call it, and it seems unreasonable to say that it must be valid to
call a function with no parameters in any and all circumstances.

------
XCSme
Very interesting, to me seems like a "compiler bug". The compiler should not
automatically set the static pointer value if the function that sets it is
never called.

Anyway, I guess "undefined behavior" is really undefined and it means anything
can happen, so as per specs it's not a bug. Ultimately it's the programmer's
mistake for having undefined behavior in his code.

~~~
electrograv
Unfortunately, "it's not a bug, it's a feature!" \-- there are long-standing
design choices in C/C++ where various circumstances are explicitly designed to
yield "undefined" behavior where literally anything goes. I believe the
original intent of these are to give the compiler/optimizer more room to speed
up the executable.

Edit: The 'undefined' clause here is due to invoking a function at address 0,
rather than any lack of variable initialization (since global variables are
automatically initialized to 0, as the poster below points out).

~~~
lisper
It is a bug, but it's a bug in the spec. Saying that a common mistake like
dereferencing the null pointer is undefined and therefore your program can do
anything is not useful behavior. The only sane design is for any attempt to
dereference the null pointer to cause the program to signal an error somehow.
Exactly how that happens can be left unspecified, but that it must happen
cannot be unspecified in a sane design. I don't see how any reasonable person
could possibly dispute this.

~~~
jcranmer
> Exactly how that happens can be left unspecified, but that it must happen
> cannot be unspecified in a sane design. I don't see how any reasonable
> person could possibly dispute this.

Your proposal is basically tantamount to saying that the compiler can never
ever delete any read or write to memory that is unused if it can't prove that
the memory pointer is non-null (for example, the pointer is an argument to the
function--clearly a very common case).

Trying to formally specify what can and can't be done with common undefined
cases (like dereferencing memory that results in a trap or reading
uninitialized values) turns out to run into issues where the specification
unintentionally precludes common optimizations, and it's not always clear how
to word semantics in such a way to not do that.

~~~
lisper
> Your proposal is basically tantamount to saying that the compiler can never
> ever delete any read or write to memory that is unused if it can't prove
> that the memory pointer is non-null

That's right. I would much prefer to take a small performance hit if I might
be dereferencing a null pointer than to literally have _anything at all_
happen in that case. If I really need that last little bit of performance I
really do want my compiler to insist that my code be correct before it will
give it to me rather than take a wild guess at what I really meant to do and
get it wrong.

~~~
richardwhiuk
How big a performance hit? Double run time? Triple? 10x?

~~~
lisper
I will take any performance hit over the potential for catastrophic failure by
default.

~~~
umanwizard
Then there is an easy solution: compile with -O0. What's the problem with
optimizations being available for those who want them?

~~~
lisper
There is absolutely no problem with optimizations being available. The problem
is that the standard gives the compiler license to muck with the semantics of
the program in highly non-intuitive and potentially dangerous ways, and this
is true regardless of what flags are passed to the compiler. So I can't
_depend_ on anything working _even if_ I specify -O0, at least not by the
standard. I am entirely at the mercy of the benevolence of my compiler vendor.

If the compiler is going to be given license to make those kinds of dangerous
non-intuitive changes to the program semantics I want that to be evident in
the source code. For example, I would like to have to say something like
YOU_MAY_ASSUME(NOTNULL(X)) before the compiler can assume that X is a pointer
that can be dereferenced without signaling an error if it's null. That way the
optimizations are still available, but it is easy to tell by looking at the
source code if the author prioritized speed over safety and if there are
potential gotchas hiding in the code as a result. The way it is now, the
potential gotchas are like hidden land mines, nearly impossible to see, and
ready to blow you to bits (pun intended :-) without warning.

~~~
jcranmer
> So I can't depend on anything working even if I specify -O0, at least not by
> the standard. I am entirely at the mercy of the benevolence of my compiler
> vendor.

What you're basically saying is that you want semantics that's basically
defined by the compiler vendor (or, more accurately, compiler/operating
system/hardware trio), but you're pissed that the standard leaves it up to the
compiler vendor as to whether or not you will get that option. You're already
"at the mercy of the benevolence of [your] compiler vendor" to give you things
like uint32_t, why is the presence of a -O0 inherently worse?

~~~
lisper
No, uint32_t has been part of the C standard library since C99.

------
amelius
So how do you intentionally generate a core-dump these days?

~~~
gizmo686

        abort();
    

[https://linux.die.net/man/3/abort](https://linux.die.net/man/3/abort)

