I see a couple of Rust developers working on LLVM almost full-time (like nikic), but there should be more. The Rust language needs to become a more important stakeholder, and for that it needs more paid full-time LLVM developers, infinite loops are UB in LLVM-IR...
People with deep pockets are typically advised not to depend on unsupported infrastructure. It is hard and, often, unwise to argue with such advice.
It is still early days. Give it ten years: Rust will either have taken off or sunk, by then. Maybe somewhere in those years there will be some new hotness to jump on instead.
Either way it's a short time in which to gain acceptance anywhere near the level of C++ so it's not surprising it hasn't. Although companies (including Microsoft and some Google teams) are just starting to take it seriously.
If your bug happens to hit a use case important to flagships like XCode or Android -- or any of the internal projects at Apple or Google, you have a much better chance of seeing it fixed.
Unless your bug shows a clear regression and is interesting enough to be a release blocker, it probably won't get prioritized. OTOH if you supply a patch with your bug report, it probably has a decent chance of getting accepted.
LLVM doesn't not care about rust. It's just not a pet project of a heavy hitter yet. So rust devs dig down and find llvm bugs. But a best case scenario would be a "llvm developer" working for Google/Apple/Mozilla who will prioritize features and fixes impacting rust. But that's not necessarily better than another "rust Dev" willing to dig down and find llvm bugs.
I'm not knocking LLVM, I think it's a great community. Just strikes me as more of a BYO-bugfix kinda gang.
Edit: Another possibility is difference between how often Rust and Swift update to LLVM trunk. Both have LLVM fork pinned to specific version updated from time to time and I guess the first to update gets all the bugs.
 (I have thousands of lines in both)
> They haven't fixed noalias in years,
I think you fail to understand the complexities of TBAA, which are and have been working on for a long time. It's a lot of work frankly.
> there isn't still a freeze intrinsic in the IR, the LLVM-IR semantics are often not documented enough for the Rust compiler to know whether it is generating IR that has UB or not...
It's important to have more languages target LLVM to flush these out. Things aren't intentionally undocumented; more like no one has really thought that hard about edge cases and interactions. And if there's only a few front ends generating similar IR, who will expose those dark corners?
But I think the limitation is on developers willing to write a blog post rather than "serious" development. Excluding "LLVM Weekly" which has since moved to http://llvmweekly.org/, there's been less than 1 post per month, even though LLVM conferences have grown significantly in size.
> We quickly learned, however, that the LLVM linker plugin crashes with a segmentation fault when trying to perform another round of ThinLTO on a module that had already gone through the process.
It sounds like they worked around this, rather than fixing the segfault and putting some error handling in place? Might make it easier for the next bunch of people working in this part of clang.
True, but they should still focus on oxidizing whole modules and subsystems in their entirety whenever possible.
This isn't Rust or Clang's fault, of course, it's just a consequence of using LLVM IR as the data for LTO, that LLVM IR has no backwards compatibility guarantees, and that Rust is out-of-tree for LLVM.
In theory using a stable format for LTO would avoid issues like this. Wasm is one option that has working cross-language inlining already today, but on the other hand it has less rich an IR, and the optimizers are less powerful than LLVM.
In other words a newer optimizer running on older IR may emit broken code.
For example, imagine that LLVM has a known bug with some pass, and it has a workaround somewhere else that disables generating IR that would hit that bug. A different version of LLVM may fix that bug, and remove the workaround - but then optimizing bitcode from another version could be vulnerable.
Another example is undefined behavior in LLVM IR. It may be handled differently in different versions, and it's hard to know what might happen from mixing them.
In general, LLVM is heavily tested - on each revision. I'm not aware of any large-scale project that tests all LLVM versions on LLVM IR from all other versions. That's untested. I'd be afraid to rely on that.
I don't know what Apple does with user-supplied bitcode, but if I were them I'd be recompiling old bitcode with the old LLVM that matches it, or something else (like subset the bitcode to remove undefined behavior, etc.).
However, the way I found it is that it affected the random version of LLVM trunk stable rustc was using at the time... That's one of the reasons why stable rust should stay away from LLVM trunk.
Instead of fixing the crash, they landed a workaround.
> We learned that all LLVM versions involved really have to be a close match in order for things to work out. The Rust compiler's documentation now offers a compatibility table for the various versions of Rust and Clang.
It's cool they got it working, but it sounds like this is currently proof-of-concept quality and not very productionized yet. To me, the overall tone of the article sounds like they ran into a bunch of issues and opted for duct tape instead of systemic fixes. Which is fine to get things off the ground, of course! But I hope they take the time to go back and fix the underlying issues they ran into too.
That said, they don't have infinite time, and if, as in this case, the upstream fix would: (1) be pretty involved and (2) be very likely to get regressed because upstream doesn't have the capability to run tests that would prevent that (e.g. because upstream only runs C++ compilation tests and there is no way to exercise the relevant bugs via C++ code), then investing in fixing upstream may not be the right tradeoff.
In theory, one could first change upstream's test harness to allow Rust code, but that involves upstream tests depending on the Rust compiler frontend, which apart from being a technical problem is probably a political one.
Maybe it would have been possible to do upstream tests via bitcode source instead of Rust or C++; I don't know about LLVM to say offhand. But in either case this is not as easy as just "fix a simple upstream bug"...
As far as I can tell, MIR inlining currently happens with -Z mir-opt-level=2 or higher, and that is not implied by -O. But I have no idea what future plans exist in that area.
I admit I have a bias here: it feels to me like everyone (not just Rust) is running away from doing things at LLVM IR level, and the resulting duplication seems inelegant. But on reflection, part of the reason I have that feeling is that I've recently been spending time fixing up Clang's control-flow graph functionality... which itself is 12 years old, so it's not a new trend at all!
This, along with const generics and simd, would make rust the perfect language for me.
I stopped working on LLVM about 2 years ago (give or take), as i now have way too many reports to be able to do any effective patch or design review, or honestly, keep up with the mailing list. I'm also too far divorced from work being done.
(I unsubscribed late last year)
I specifically reviewed and approved the llvm.noalias patches before i stopped, which is why they are marked as accepted by hal, back in 2016.
More than that, i was one of the people who basically showed that !noalias/etc is fundamentally broken and can't be fixed.
Nothing should be blocked on me at this point, and my reviews account is deliberately disabled so that people can't assign/add me to things.
If something is blocked on me, hal certainly hasn't let me know :)
Another is that compiler developers often don't have infinite amounts of time to sort out shenanigans like this when they come up. Users generally prefer the compiler to work, even if suboptimally, when compared to "not working", so there is some tension between things and how long they take. So landing workarounds in various ways -- sending patches upstream, using custom builds with bespoke patches, code generation workarounds -- all have to happen on a case by case basis. Many LLVM clients do this to varying degrees, and a lot of features like LTO start off fragile due to these kinds of things. Over time things will get tightened up, hopefully.
When I worked on GHC (admittedly several years ago now), the #1 class of problems for the LLVM backend were toolchain compatibility issues above all else, because we relied on the users to provision it. At the time it was nothing short of a nightmare -- various incompatible versions between various platforms causing compilation failures or outright miscompilation, requiring custom patches or backports at times (complicated by distros backporting -- or not backporting -- their own fixes), some platforms needed version X while others were better served by Y, flat-out code generation buts in GHC and LLVM, etc. It's all much better these days, and many features/fixes got landed upstream to make it all happen, but that's just how it works. Rust made several design choices with their packaging/LLVM use that we didn't that I think were the right ones, but I'm not surprised they've had a host of challenges of their own to address. TINSTAAFL.
However, libstd is by necessity tied quite closely to the compiler, it's one of the oldest and most fundamental parts of the stack, and there are tons of little complications around making it "just" a crate, so changes to libstd/rustc/Cargo necessary to make that possible will take longer.
In the meantime, changing one problematic compiler flag seems like a sensible solution.
A big reason C++ took off was backward compatibility with C. Network effects. Today C++ has the role that C had in the 80s.
No one uses any other compiler but LLVM for Rust anyway, so who cares about compatibility with MSVC and others. This will also force adoption of LLVM, which can be a good incentive for LLVM to support it.
This seems doable for simple cases, but libraries like Boost push templates to their limits, so they're not the easy cases.
Basically it would be a "extern C++". The FFI to C++ would hopefully have as low overhead as "extern C" while making it easier for C++ codebases to gradually adopt Rust.
This is already supported (via Bindgen).
However, it would also be extremely hard; I don't think any programming language has ever created such a tight bridge to C++. The existing approaches I've seen are:
- Swig, rust-bindgen, etc.: Pass through auto-generated C ABI wrappers; support for generics is limited and requires declaring up front what types you want to instantiate them with.
- D: You can bind directly to C++ if you rewrite your C++ header in D, generics and all... including the implementations of all inline functions.
Both very limited, especially in template-happy modern C++.
The whole point of UWP was to improve COM to make it even better for language interop, increasing the kind of language features that can get exposed as COM libraries.
Also the number one reason that if Rust wants to succeed as system language on Windows it needs to have first class support for COM/UWP.
Just make Rust use the C++ ABI.
C++ code is compiled in separate translation units as C++ by the C++ compiler. The linker statically links calls to C++ code from object files created by the C++ compiler. That's it. Rust doesn't need to know how to deal with templates because templates are all instantiated by the time this happens. This means you have to write a lot of your fancy C++ logic in separate C++ source files (not headers), but that's the whole point, to use already written code.
The only thing Rust has to do is pass pointers to static code and follow the calling and name mangling convention. Zero calling overhead.
Keeping C++ software while making sure important parts are bug free sounds awesome...