The compiler is the implementation; it cannot cause "undefined behavior".
What it can be is "nonconforming".
A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming.
An implementation is not bound by the standard in its own use of the standard library functions. The implementation can use a function like memcpy in ways that would be formally undefined if they occured in user code; its implementation of memcpy just has to harmonize with that use, so that the intended behavior is ensured.
This is because implementations can add their own requirements to areas that the standard leaves undefined. For instance, an implementation can add the requirement to its memcpy that the copy may overlap, if the destination has a lower address. Then, the implementation can generate code which uses memcpy that way, or make internal uses of memcpy from other library functions which use it that way. It's just using its own (possibly not publicly documented) extension.
> An implementation is not bound by the standard in its own use of the standard library functions. The implementation can use a function like memcpy in ways that would be formally undefined if they occured in user code; its implementation of memcpy just has to harmonize with that use, so that the intended behavior is ensured. This is because implementations can add their own requirements to areas that the standard leaves undefined. For instance, an implementation can add the requirement to its memcpy that the copy may overlap, if the destination has a lower address. Then, the implementation can generate code which uses memcpy that way, or make internal uses of memcpy from other library functions which use it that way. It's just using its own (possibly not publicly documented) extension.
Yes. But right now they do not do that, and it's not their implementation of memcpy they rely on, and they are exactly doing the kind of things they disallow their users to do, so the whole picture is just plain insane. They just call libc memcpy in some cases, and their rational is that "it happens to work". Except nobody is actually sure about that, it might be already buggy on PPC, and the guarantee is not documented anywhere. Completely insane - especially given their very own approach in similar situations. As usual, they are prioritizing "optimizations" over correctness, and lets not even think about defense in depth. It's a brand new level of insanity, even beyond their usual "the standard allows us to do that" excuse, because here it does not even allow that, and it seems that neither do officially all the used implementations!
Software engineers must loose the habit of taking all kind of crazy risks that can only lead to marginally faster execution of newly introduced bugs. That does not serve any purpose.
Yes, if "undefined behavior" doesn't just mean "a situation where the ISO language standard doesn't impose a requirement", but has a broader meaning like "behavior not defined by anyone at all".
The compiler is using the ISO language standard to define the behavior of an internal piece. It is then using that piece in the wrong way. It's not generic "not defined" behavior, it is specific C "undefined behavior".
Think of that part of the transformed code as still C for relevant purposes.
I want to assure you that the compiler does not exclusively use the ISO language standard to define the behavior of internal components of the implementation.
The compiler is not producing C code that calls memcpy. It is leveraging a specific implementation of a memcpy symbol in a particular system library. The contract between Clang and that system library can include guarantees not provided by any ISO standard.
The bug report is not wrong. Rather, whether the current behavior is correct depends on the platform.
For example, Darwin's libc offers this contract for memcpy and clang is perfectly within its rights to generate such code (note that the IR it generates is still violating LLVM's contract on the memcpy intrinsic); glibc offers no such contract for memcpy, and so clang's code is nonconformant.
Are you saying that Clang guarantees its own memcpy implementation will be used? I don't know; but it seems odd that if this is fully intended that the compiler/linker would generate warnings from its own handling of perfectly valid C code (as shown in the bug).
No, Clang cannot guarantee its own memcpy implementation will be used. But it can be aware of which implementation will be used and make specific assumptions about that implementation in some cases.
The specific cases are when it has a genuine guarantee provided by that implementation vendor that it is OK. In the case discussed on this bug, at least one implementation provides that guarantee (Darwin). The problem is that not all provide it, and further than other parts of LLVM have started assuming that the memcpy more closely matches C semantics. At least something will have to change here, but the C semantics are really relevant at all, or what is "valid C code".
Clang is not a complete implementation without a library, right? Whoever puts Clang together with a library becomes an integrator who provides a language implementation. It's up to that integrator to ensure that the result conforms.
Clang could be also used "freestanding". Then the user might be informed: please supply your own memcpy function with such and such characteristics, since generated code needs it.
If those characteristics are just those described in ISO C, but the compiler needs something more, then that's a problem between clang and the user.
But I think Clang can actually do better here. The open source project is essentially providing integration with glibc and other libcs and we don't do a reasonable job of being conservative there where we don't know what requirements that layer places upon things like memcpy.
> The compiler is the implementation; it cannot cause "undefined behavior".
The compiler is only part of the implementation. So is the standard library. So would be a processor. An implementation as a whole, perhaps, cannot be said to cause "undefined behavior", but I think it's correct to say it's parts certainly can.
If a preprocessor took valid C, and emitted valid C that could be fed into any other complete freestanding C implementation, would it not be accurate to say that this preprocessor introduced undefined behavior if it replaced, say, memmove(a, a, s) with memcpy(a, a, s) - sans any guarantees about which compiler it would have the emitted C fed into?
When it's LLVM IR invoking glibc's memcpy instead of a memmove, why is that not "causing undefined behavior"? Because it's not a C program? And if I misinvoke it via C++, is it only undefined behavior because the C++ standard references the C one? What if I use C# to invoke memcpy, which doesn't?
> An implementation is not bound by the standard in its own use of the standard library functions.
This is all well and true from a theoretical sense, but an implementation is bound to it's own standards as a means of achieving a conforming compiler, and in this case those standards are clearly documented to mirror the standard's standards. These standards are being violated, leading to possible future misbehavior when combined with other optimizations which may leave the code behaving in indeterminate ways - and, for that matter, current misbehavior in the form of triggered "false" positive valgrind checks.
> The implementation can use a function like memcpy in ways that would be formally undefined if they occured in user code; its implementation of memcpy just has to harmonize with that use, so that the intended behavior is ensured. This is because implementations can add their own requirements to areas that the standard leaves undefined. For instance, an implementation can add the requirement to its memcpy that the copy may overlap, if the destination has a lower address. Then, the implementation can generate code which uses memcpy that way, or make internal uses of memcpy from other library functions which use it that way. It's just using its own (possibly not publicly documented) extension.
It can do these things, if the implementation as a whole is coordinated in this way, but I would hope for sanity's sake it would not. And in this case the implementation as a whole has not been coordinated in this way.
Invoke copy_forward from memcpy if you like, where the former has fewer preconditions - but to write code that relies on an implementation's memcpy having fewer preconditions than the standard, even if you control the entire implementation, sounds untenable. Someone will forget that your memcpy has fewer preconditions, and write optimizations, debug checks, etc. which misassume additional preconditions that are outlined by the C++ standard. They will also also accidentally write code assuming fewer preconditions in another context where they're not guaranteed some specific implementation's relaxed preconditions.
The compiler causing undefined behavior on its own reminds me of one of the fun bugs we found[1] with 32-bit x86 clang where one compiler pass fools another into thinking there's an undefined behavior by combining three struct field initializations into a single 4-byte load, but the struct itself was 3-bytes long. Another pass looked at this and thought there's an undefined behavior writing past the struct boundary, silently eliminating the initialization altogether, resulting in bad codegen.
Compilers are way too overzealous eliminating code because of undefined behavior. Legally it is allowed and I'm sure it helps in synthetic benchmarks. But writers of real programs won't be impressed of the perf gains when you stop running half their program.
Rather, they are getting zealous in eliminating code because of the assumption that there is no undefined behavior. The scope of what undefined behaviors do this is increasing and that breaks programs that make some assumptions on traditional compiler behavior, or that have hidden bugs.
I believe that C should have safer semantics, plus standard-defined modes of operation which restore the unsafe semantics selectively.
The optimizer then must obey whichever semantics in effect (safer or less safe).
Then, in addition to telling your compiler you want a certain optimization level, you could tell it which standard-defined modes of operation you would like.
Currently, this is already de facto the case outside of the standard. For instance, users of GCC who want to do certain pointer aliasing without unpredictable behavior tell the compiler to be in a mode in which that is allowed (effectively an altered C dialect) using -fno-strict-aliasing.
This kind of thing should be a standard, portable feature. Look, everywhere in this program, I want nice left-to-right evaluation within all expressions and among initializers and function arguments. Except in super-fast.c; when compiling super-fast.c to super-fast.o, please do it in a mode where unspecified-evaluation-order semantics applies and optimize accordingly. (I might even want this on a more finer-grained scale than translation units.)
We need safety, and we need to sometimes throw away safe semantics in exchange for better speed (whereby we still ensure that the code we are writing is correct with regard to the weakened semantics; i.e. we promise to the compiler that even if it relaxes the evaluation order, or whatever, we know what we are doing and everything is cool).
It would be good to do this without compiler-specific guesswork.
Optimization control alone over a language with a single set of semantics (which is largely unsafe) is a bad way to achieve this.
I agree you should be able to opt-in to a safety level. For example most programmers do expect signed integer overflows to follow 2s compliment behavior - regardless of what the standard says.
There really seems to be a large disconnect between the tool writers and the users of said tools. I've had countless discussions with compiler writers where we've essentially talked passed each other. The compiler writer arguing from a legalistic (and dare I say moral perspective) that the programmer must not write undefined code, and myself from the perspective that developers do write undefined code and we should therefore find a way to handle it in a safe way. Perhaps even by standardizing more than we do currently.
In reflection its not unlike the drug legalization debate.
Keep in mind that the compiler doesn't always know whether code is undefined or not. Is x+y at risk of overflow? Maybe, depending on some logic in a separate compilation unit.
So what your asking is to define behavior for all code, such that the compiler can prove locally that's it's well-defined. That's not a crazy idea, and a lot of languages do that.
But C counts itself as special, for better or worse.
I think we really just need more alternatives. There's C++, Rust, Ada, and not much else. I am very hopeful about rust.
The problem is that C compilers don't bother proving anything, when it comes to taking advantage of what is defined.
For instance, if x is signed, a C compiler can assume that x + 1 produces a value which is greater than x, and optimize accordingly. It's up to the programmer (and failing that, user) to ensure that x is less than INT_MAX prior to the execution of that x + 1. In fact there is a proof involved, but it is trivial. The compiler assumes that x + 1 > x because, by logical inference, the only way this can be false is if x == INT_MAX. But in that case, the behavior of x + 1 is undefined, and so, who cares about that; it is the programmer's responsibility that this is not the case. Q.E.D.
Rather than require compilers to implement complicated proofs, which essentially consists of a whole lot of work whose end result is only to curtail optimizations, it's far simpler to have ways in the language to tweak the semantics.
If there is a safe mode in which x + 1 is well-defined, even if x is INT_MAX, then the compiler can no longer assume that x + 1 > x. That assumption is simply off the table. (Now the difficult work of proof kicks in if you still want the same optimization: the compiler can still optimize on the assumption that x + 1 > x, but that assumption can be false, yet x + 1 can be well-defined! If an assumption can be false in correct code, then you have hard work to do: you must prove the assumption true before relying on it.)
This safe mode is not linked to optimization level. Unless turned off by the programmer, it is in effect regardless of optimization level.
Right now we have a situation in which compiler optimization or "code generation" options are used for controlling safety. Programmers know that de facto these options affect the actual semantics of C code, and so they get used that way. We know that if we tell GCC "-fno-strict-aliasing", we are not simply defeating some optimizations, but we are in effect requesting a C dialect in which cases of certain aliasing are well defined.
This is a poor situation. Why? Because it's nonportable (the next compiler you use after GCC might not have this option). But, more importantly, semantics and optimization must be decoupled from each other. We pin down the semantics we want from the language (we pin down what is well-defined), and then optimization preserves that semantics: whatever is well-defined behaves the same way regardless of how hard we optimize.
Semantics and optimization have to be independently controlled.
That is how you can eat your cake and have it too. If you think some code benefits from an optimization which assumes that x + 1 > x, you just compile that under suitable semantics. Then if you don't ensure x < INT_MAX, it's really your fault: you explicitly chose a mode in which things break if you don't ensure that, and then you didn't ensure that.
Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.
I don't see anyone being blasé about this. The issue comes down to something really simple:
- The compiler can emit substantially simpler code if it has a guarantee from the platform's implementation of memcpy.
- The compiler authors thought they had such a guarantee and so chose to leverage it.
Now, maybe they don't have that guarantee on all platforms. If that's the case, its flat out and simply a bug in the compiler. And you'll find folks on that bug arguing that it doesn't actually hold, which is good and important.
But it's actually quite reasonable to have a guarantee from a platform's libc that this will work and leverage that. The compiler isn't generating C code or code bound by any standard. This is about two parts of the implementation of C agreeing about what internal invariants will hold.
The problem this bug highlights (and it is a real problem, and not one anyone takes lightly) is that we have insufficient agreement about these invariants, both for certain platforms (glibc) and even within LLVM. Fixing this remains absolutely critical, but may only involve documenting the agreement.
I was surprised to see: "I don't think it's really worth warning about, considering I don't know of any real-world implementation where it doesn't work."
That sounds a lot like someone knowingly writing UB in a C program because no compiler they know of breaks the intended behavior. I doubt any memcpy() has ever guaranteed in any kind of official way that overlapping regions are ok, even if it has historically worked in practice.
I mean I'm sure it'll get fixed now, I was just surprised to see this kind of reasoning coming from someone who I presumes works on the compiler.
btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. There's a new memcpy() symbol in glibc these days with a different symbol version that no longer provides correct behavior in the case of overlapping memory regions. This is all second-hand from some other people I was working with, but if I got their meaning correctly I think this issue will actually have relatively widespread impact.
That comment was actually a considered comment: when we look at what warnings we should produce, we consider cases where everyone has written code a particular way and the fact that it isn't defined is actually a language bug.
We (the Clang community members, and I suspect the GCC community members as well) have also worked to change language standards to provide guarantees relied on consistently and where the lack of guarantee has no reasonable basis. Also that comment is very old, and predates us learning about all of why this is probably something actually worth warning on. That discussion ended up in a different forum, and we do actually warn on stuff like self assignment.
Understand that it is often precisely the people working on a compiler that do deal with the cases of when we discover widespread UB in real code and whether or not the code is incorrect and must be changed or the standard is incorrect and must be fixed.
But all of that is about a warning, and not about the actual question of how the compiler and the system library interact.
Note that both Clang and GCC have at least some variants of this bug, and so I don't actually expect glibc to break this in the foreseeable future. But we're still actively working to defend against this on platforms for which we don't have a guarantee.
> I don't actually expect glibc to break this in the foreseeable future
Would you have said this if Ulrich Drepper was still in charge of glibc? Under his ownership glibc broke all sorts of things that happened to rely on memcpy behaving like memmove[1]. I'm not sure he would have been any more accommodating of non-conforming compilers than of non-conforming flash implementations.
I can't see an argument here for why the standard should be changed. memcpy() does one thing and memmove() does another. Changing the standard so they are the same just because memcpy() gets misused in real-world programs doesn't make sense to me. If people misuse memcpy(), then have a compiler flag akin to -fstrict-aliasing where memcpy() gets rewritten to memmove() unless people compile with -fstrict-memcpy.
That seems a separate issue though from what calls the compiler should emit. I am not following why LLVM would emit memcpy() when the semantics it actually requires from libc are memmove(). Am I missing something?
It is not about making memcpy behave the same as memmove: in this case the difference would still be between exact identity (src == dest) and partial overlap.
LLVM is a very impressive compiler. I'm really impressed with the warning messages it's able to produce (especially compared with what was available even 2 or 3 years ago).
What is more frustrating about modern compilers (and this is not limited to LLVM) is the desire to squeeze out a tiny bit more performance by exploiting undefined behavior. I'm thinking of things like eliminating calls to memset that "don't seem to do anything user-visible," eliminating code based on signed overflow being undefined, and so on.
Most of the the undefined behavior that exists in C is there because different machines used to do different things. For example, there used to be non-two's-complement machines. It wasn't put there because C was somehow designed to be hyper-optimized. People who relied on the undefined behavior could legitimately make the same argument that you are now making-- that it worked fine on their machine, which they knew very well.
I am curious why nobody has brought up the idea of making memmove as efficiently handled as memset, rather than using undefined behavior here. UB feels like the wrong way to go.
Can't you see the pattern here? Even LLVM devs are incapable of doing what they are requiring from random regular developers. If that does not motivate to address that kind of craziness more in depth in a systematic way to restore safety and defense in depth, I don't know what will.
- Programmers can and do emit substantially simpler code if they have a guarantee from the platform's implementation of <feature x>.
- Programmers think they have such a guarantee. (I don't know a single programmer around me who knows the complete list of undefined behavior, myself included)
- Compiler injects bug -- and LLVM/CLang seek excuse in the "technically not a bug" completely useless answer.
- Sane projects actually disable all that kind of crazyness (adding -fno-<everything> options every now and then) so all that kind of cute optims are useless in the real world (either dangerous, or completely disabled)
It's actually reasonable to have a platform that e.g. gives you 2's complement arithmetic or defines the result of shifts, or allows unaligned access, because mainstream computer hardware does provide those guarantee. And at least half of C programmers are not even aware those kind of stuff are technically forbidden, (historically often because of other architectures, some obscure, and not compiler optimizations, btw). More knowledgeable people who know about the spirit of the standard (1) can also be bitten by compiler revelation of bugs because compilers devs are doing literal and insane interpretation. Bug ensues, including security holes.
(1): Let me remind you about the definition of undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data,
for which this International Standard imposes no requirements"
One keyword is nonportable and it is clearly not considered enough. So yes, there are numerous constructs that are reasonable to expect on a given architecture when we do not learn the compiler manual by heart, but some people don't give a fuck about the principle of least astonishment when they can gain 1% on a synthetic benchmark.
> It's actually reasonable to have a platform that e.g. gives you 2's complement arithmetic or defines the result of shifts, or allows unaligned access, because mainstream computer hardware does provide those guarantee.
ARM, PPC, and x86 all define the results of some shifts differently (for shift amounts >32UL) and behave differently with unaligned pointers (only x86 tends to accept them). Which mainstream hardware are you using?
Adding magical nice semantics to your language is another way to hide bugs in your program. Try building with ubsan and doing runtime testing instead.
1) Challenges in understanding undefined behavior are exactly why tools like ubsan[1] exist. In a way they help /everyone/ understand when 'undefined' constructs are encountered, and having this emitted by the compilers responsible for translating your source code into binary code is the ideal place.
2) Compilers leveraging undefined behavior knowledge get a lot of hate for breaking this implicit contract defining what they "should" do as expected by programmers. Unfortunately these feelings are rather misguided, compilers are just doing what they can to produce the best code for you using the explicit contract that exists: the language specification(s). Are these specifications perfect? No, but they're what we have. If someone were to define a new language that actually captures the assumptions "real" programmers "expect" when they write code then compilers could choose to support it (and probably would!) and everyone would be happier. Until then this is a big case of "how come you're not doing that thing you never said you'd do and no one is willing to define properly?" which is a bit silly. Imagine you're a compiler developer: what assumptions CAN you make to satisfy everyone? In short, compiler developers aren't your enemy, they're just doing everything they can to produce the best code they can for you.
2a) Use of -f* is in a way used to define that a special variant of a language is desired, for example -fwrapv builds the code using an unnamed variant of C where signed integer overflow is well-defined.
> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.
I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a priority of my compiler any more, with Clang. All the more reason to switch to Rust I suppose.
That the Clang people are trying to deflect this bug as "fine, no fix necessary" is kind of disgusting software engineering behavior. Compilers spontaneously introducing undefined behavior in another layer (to the point of causing codegen warnings from perfectly correct C code, is as clearly cut a BUG as any, independent of current API implementations outside of the ISO spec.
If you want to make undefined API calls the norm, then make a new API where the behavior you want is defined. Invoking undefined behavior of a decoupled library that users can swap out is flat out indefensible.
That they are reluctant to fix this makes me shy away from ever using Clang/LLVM, which is sad because before I had always held them with very high respect.
Understand that some of the folks who don't think this is a bug own one of the libc implementations being used. So for their platform, it really is 100% correct and valid.
The folks working on other platforms seem universally unhappy with that bug.
It is not the particular situation people are upset about (well, at least me). It is the reasoning, especially in the context of "you shall not write undefined behavior according to the standard even if it should makes sens given the computer you are actually programming" that is usually sent on us regular programmer by compiler authors... :/
Yet they do exactly the same thing, and pretty much casually. And I actually don't really blame them for doing it. Everybody makes mistakes. But I blame them for the double standard, the risks, and the attitude. The problem in the first place: engineering is also about mitigating the fact that mistakes will happen, often mitigating even the mistake of others, and the modern approach in regard with undefined behavior does not help and is even ultra-hurtful for that. I don't see how stability will emerge with that kind of thought process.
You are tarring all compiler authors with a very broad brush here.
Myself and other compiler authors I work with are much more specific in their requests. We do actually define some behavior that the standard doesn't require, and we do document this and tell our users its fine to rely on it.
Unfortunately, we also have a lot of really clear experience with this stuff that lead us to two conclusions:
1) When we identify UB in code and go look at that code, for very large bodies of code what we find are overwhelmingly bugs. Bad, complete bugs. Defining the behavior wouldn't fix the bug, it was just one of the many symptoms.
2) We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. As an example, shifting left by more bits than the size of the value gives different results on several different platforms.
None of this means that we don't need to help programmers cope with this. We do indeed also make mistakes, and in fact it was these compiler authors that built a tool to help programmers find UB in their code:
http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
If it's not 100% valid on 100% platforms and 100% of all future Libc revisions (which is impossible if it relies on undefined behavior and can link with different Libc versions), then it's not 100% correct and valid.
Well, electrograv was not distinguishing between the llvm project and the clang project. So it seemed reasonable to point out that rust was using llvm.
> I have suddenly lost a great deal of respect and confidence for _LLVM_/Clang after reading this bug thread;
However, do you believe that there is little overlap between llvm developers and clang developers?
The bug report contains a quote from the C++ standard: "An rvalue or lvalue t is swappable if and only if t is swappable with any rvalue or lvalue, respectively, of type T"
What does this statement mean and why is it not tautological?
If swap doesn't work for a self swap, this code would break in the case of an odd length string.
It's an easy fix in this case, but being able to swap an element with itself is convenient.
It's quite common actually. Usually in the context of something like "swap element 'n' with the last element in this vector" without any check to see if 'n' is already the last element, because checking for that condition just to avoid a harmless no-op in the uncommon case is ugly and slower in the common case.
Before shrink to fit, you could easily have produced this with a few references and a "use std::swap unconditionally".
The std::swap thing is only one pitfall, the reliance on "our memcpy will do the right thing with any overlap" is a bit risky (and doesn't hold with LD_PRELOAD, but I think it may have been switched to using the LLVM based librt project anyway, that is I'm not sure these calls are supposed to become naked memcpy).
IIRC, there are passes that also detect memcpy-"equivalent" loops and replace them with the builtin to memcpy (mostly to get the vectorization benefit). And of course there's SimplifyLibCalls (or whatever it's named) that undoes this or any other "you called a standard library function we know how to inline in this case".
The second one sounds like big fun when you LD_PRELOAD a different memcpy (say, for debugging purposes) and spend hours and hours trying to figure out why it is not being called.
Yes, it's a nightmare, particularly on OS X with its funny loader semantics that make replacing malloc and free correctly very painful. For example, you can easily LD_PRELOAD your way out of malloc() but did you remember that asprintf also calls an allocation routine and won't use your new malloc? Enjoy.
Hmm, why won't it? Maybe in the past when asprintf and malloc were located in the same dylib, since OS X dylibs generate direct calls to other public functions in the same image (rather than going through the PLT as GNU does by default), but for many years /usr/lib/libSystem.B.dylib has been split into several reexported libraries in /usr/lib/system, so there shouldn't be an issue there. Probably better to use this technique though (should still work in the latest version but needs mprotect):
Reading most thread of llvm bugs so far about there "undefined behaviours" I see a pattern.
Llvm goal is performance over correction. stop.
If old code breaks, it is you played with fire, correct your code to eliminate these undefined behaviours. Stop.
We made wonderful coloured warning to help you. Full Stop.
Well on one hand I get the point, people played with fire. Code should be "correct" by standards that are emerging, but I can predict one thing : before the new "correctness pattern" that enables lots of awesome robust high performing code spreads, there will be chaos.
My first self interrogation is in how much time llvm team can build a completly correct stable compiler without any surprises? It it takes 2 decades, it will be a lot of chaos involved...
My second interrogation is can the code that "is incorrect" per LLVM standard be obtained without rewriting the source code (I guess not), and how much will it costs before all the code is being "corrected". Do we even have the man power to do it?
You see, some C code may be used everyday but are so complex hardly any developers even well paid like to look at them. (openssl, gnutls, libpng, zlib, dhcpd, bind...). And with platform specific ASM, we may discover that code has lost portability, and it may take a lot of time to discover that it is already happening.
Don't we risk another strike of bad news one day like discovering old software/libraries are behaving differently and need to be rewritten and that no one want to do the same effort as libressl especially since some software have a LOT of dependencies on C code (I do not think to loud of PHP or python, or Perl).
Of course there are unit test, but there are so much of them, some tests (like for python on freeBSD with curses) have been disabled, because "no one has the time to fix them". It is ofc a bad example in this precise case, but in the context, it feels like a growing ghettoisation between the CPU/OS/langage with lots of eyeballs/investments, and the old/poor market. Especially non company backed open source software. A smell of obsolescence by attrition of resource to report bugs/rewrite code.
As a normal former experimental physicist I can feel a transition coming, it smells of chaos and instability, and the question is always, how long, how smooth before the new equilibrium if ever we reach it, and what will break in the process? And well since transitions are transitions, we may reach a new equilibrium, but nothing guarantees it will be reversible, hence some stuff may disappear in the process forever.
I don't know why, but I guess the most vulnerable ecosystem to the changes to come is the non commercial open source.
Much of the transition has already happened - most applications have already moved on from C to safer languages. What's coming is either the replacement of C libraries (Rust, very careful C rewrites), or perhaps bypassing them entirely (OCaml unikernel work).
I would fear more for proprietary systems. Open source is relatively innovative and adaptable.
We are kind of crowning the right to rule of the x86 computer industry. llvm is very tied to apple, maybe it can extend to ARM. But it means that some HW platform that may be to old may be difficult to maintain in the future (HPPA, Dec alpha, MIPS, PDP, zseries?).
And some computers have life cycle that are more than 10 years. (Expensive industry robots, telco switches, some medical devices, aeronautic/space CPU, radars, some very old mainframe used for accounting, embedded automation devices)...
Is this push to obsolescence really cool?
Okay critical system already handle the problem, but what about the stuff in the grey zone? Stuff that were not critical but get adopted nonetheless because they just worked and good be updated?
Huh? Higher-level languages are much more portable than C - you only have to port the language runtime, and often someone's done that already. LLVM has a lot of different backends available.
hum.... well. If you imagine that they are 100% idempotent on every platform and 100% fully supported. It would work.
But as I stated in my example python/Perl/C# and probably others are in the language support dropping either some platform or supporting only x% of the feature on some platform...
I dare you to access you services using C#/mono on openBSD.
I'm running FreeBSD as it happens and C# works absolutely fine - I have even just downloaded random .net exes and run them and had them work.
If you had a race where you picked a random C program and a random python/perl/C# program off github/sourceforge/whatever and tried to run them on OpenBSD, I'm pretty sure 9/10 times the python/perl/C# would win.
Remember 90% of programming is internal tools for businesses, not consumer applications.
(Also in a lot of cases you wouldn't necessarily notice. If you've played a few recent games you've likely run at least one written in C#. I've seen commercial drivers written in Python that you only notice if you look in the internals)
What it can be is "nonconforming".
A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming.
An implementation is not bound by the standard in its own use of the standard library functions. The implementation can use a function like memcpy in ways that would be formally undefined if they occured in user code; its implementation of memcpy just has to harmonize with that use, so that the intended behavior is ensured. This is because implementations can add their own requirements to areas that the standard leaves undefined. For instance, an implementation can add the requirement to its memcpy that the copy may overlap, if the destination has a lower address. Then, the implementation can generate code which uses memcpy that way, or make internal uses of memcpy from other library functions which use it that way. It's just using its own (possibly not publicly documented) extension.