> I kinda thought something like this was already being done
It is. When building a crate, Cargo doesn't need to wait for all of that crate's dependencies to be completely finished compiling, instead it only needs to wait for each dependency to produce its metadata. The prototype here builds upon that existing machinery by causing metadata to be written earlier than it would otherwise be, prior to complete typechecking. It has the potential to increase the level of effective parallelism, so whether or not this has a benefit for a particular crate graph will depend on whether or not all of your cores are already effectively occupied during compilation or whether Cargo is letting cores lie idle while waiting on metadata to be produced.
Yes, if you mostly have no extra cores already or if you have very wide unrelated deps, this mostly does nothing. Typically, this is not the case though so there is some good perf gains.
Rust has been taking steps in this direction for several years now, having first added pipelining via eager metadata emission in 2019: https://internals.rust-lang.org/t/evaluating-pipelined-rustc... . There have been several other steps in this process since then, e.g. working to make metadata less verbose, experimenting with different approaches to compression, etc. People have been eyeing making pipelining even more eager for a while now, as the submitter here noted a few days ago: https://news.ycombinator.com/item?id=49924257 , but there remain some decisions to be made regarding what to do when encountering errors in crates that were speculatively approved.
If there's a project that takes 45 minutes to produce a debug build from scratch, that's not a Rust problem, that's a them problem.
Let's compare. I just checked out ripgrep and built it and all its dependencies from scratch. Total time to produce a debug build: 6.92 seconds. This is inside a 16GB VM on a random Linux laptop (on battery); no beefy hardware here.
Let's try something beefier. I just checked out uutils (an implementation of coreutils) and built it from scratch. Total time to produce debug builds, for eighty binaries and all their dependencies: 30.26 seconds.
Let's try something beefier. I just checked out Servo, an entire browser engine, and built it and all its dependencies from scratch. Total time to build and link its one-thousand one-hundred and ten compilation units (along with its infamously huge and serial servo-script bottleneck) into a binary: 4 minutes and 38 seconds.
If Codex takes 45 minutes to build from scratch, that's an indictment of OpenAI and their development processes.
Arguably it is a rust problem. Rust can be improved to reduce the need to do as much manual maintenance of crates dynamics to allow for faster build times. The "you're holding it wrong" answer is not a good perspective to apply and I am surprised to see a prominent member of the rust community using it.
No, one can write pathological code in any language. I welcome efforts to improve the performance of the Rust compiler, but nobody is served by optimizing for degenerate use cases to the detriment of legitimate use cases. It is, in fact, possible to hold it wrong; this isn't blithe dismissiveness, but rather an accurate statement of reality, and pretending otherwise doesn't lead to a better language.
Compilation times were a consideration, but they were a subservient consideration to the prime considerations of 1) being as memory-safe as GC'd languages, 2) exploring the limits of statically-guaranteed thread safety, and 3) being as fast and memory-efficient as C and C++. If Rust had been willing to compromise on those goals and instead had just, for example, used a virtual machine with pervasive garbage collection and been designed for dynamically-dispatched generics then you'd get a lot of compilation time reductions for free, but the world didn't need another Java, it needed a more secure systems language to stand up against C++ where every previous challenger had failed.
While it has found success on mainstream, it basically proven Cyclone ideas were right, and now we have other languages adopting similar ideas into their type systems, which helps closing the gap with 3).
Since the 2000's we have been on the wrong path, many developers kept reaching for C and C++, not because they needed the actual features of a systems programming language, rather they didn't knew anything else that would ahead of time compile to native code, or even if they did the mindshare wasn't relevant for them.
Now thankfully we are circling back to how it should have been all along.
However all of this might be meaningless in the age of AI driven programming, where developers have no clue if the agent is driving automatic or stick, while delivering on the Markdown file plan.
No, Rust is not at the paretto frontier for your mentioned 1, 2, 3 and also 4 compilation speed. It could have all of those things and also just have faster compilation. Rust devs have talked about how they have some regrets about not optimizing more for compilation, but instead another attribute got optimized in terms of paretto efficiency - the language's development time. Arguably I think that is the single worst trait you can optimize for in language development, because by saving a little time developing the language you cost humanity, let's say hundreds of millions of hours of development time downstream from you, with millions of users being less productive.
> It could have all of those things and also just have faster compilation.
Rust was originally a research language. It wasn't even clear that their primary three goals--"safe, concurrent, fast"--were simultaneously achievable to their desired degree. That it turned out to be possible was one of the most surprising and impactful results in the history of language design; even the early formulations of Rust assumed that both a garbage collector and a green thread runtime would be necessary, neither of which turned out to be true. That Rust could have faster compilation is both true and vacuous; there's always more optimizations to be eked out. The main question is whether or not significant performance wins could be gained by having made different, fundamentally-backwards-incompatible design decisions, and the answer is no, not really. Performance-related features like referentially-transparent procmacros can easily be made the default over an edition, and polymorphization can be implemented via an alloca-based virtual-dispatch ABI vis-a-vis Swift.
> Rust devs have talked about how they have some regrets about not optimizing more for compilation, but instead another attribute got optimized in terms of paretto efficiency - the language's development time.
No, the early Rust devs got this right. They were in fact highly incentivized to care about compiler performance, because rustc has been bootstrapped since 2011, making them the most serious users of the language in earnest (with the second-most serious users being the Servo devs one cubicle over). Rust spent years getting its basic design nailed down, then it did the hardest thing for any language: understanding the importance of shipping, resisting the urge to endlessly pursue perfection, and future-proofing what they could. Without shipping a useful language you can't get in the hands of users, and without getting in the hands of users you can't discover your most important flaws in practice. And because Rust shipped in 2015, it had ten years to hammer itself into shape to weather the agentic era; it could have easily still been on version 0.113 by now, puttering around in an ivory tower with no commitment to stability and no real users to give feedback.
Trait solving isn't a bottleneck for Rust compilation unless you're doing some extremely cursed things like trying to implement Doom in the type system. At the end of the day it's still mostly that Rust just generates a ton of IR for LLVM to chew on (because of monomorphization), then LLVM takes a while to process all of it (because LLVM is designed primarily to produce high-performance code, often resulting in reasonable tradeoffs against compilation speed), and then the linker takes a while to connect it all together. With an alternative backend to LLVM (e.g. Cranelift) you could choose to design it with a greater emphasis on compilation speed (likely trading off generated code quality in the process). And with a more radical vertically-integrated model where Rust controls the linker you could do in-place linking and nearly completely skip the final step (at least for debug builds), but that's a big change compared to the classic C-style build model.
Chromium being open-source is a red herring here. The web is not an application, it is a protocol between clients and servers. Google has immense influence on both sides of that protocol, and regularly attempts to use that influence in an anticompetitive way. Chromium being open source just means you can fork the client, but that gives you essentially no influence over the web as a protocol; if Google wants to take the web in a direction that your fork doesn't agree with (e.g. adding new APIs that you don't want to support or getting rid of APIs that you do), then you're out of luck regardless of your ability to compile the code.
and? How does this all matter when you choose a web browser engine?
You can write your own, but you still have this problem.
So, this is not an argument against chromium at all.
I think they already answered this but their explanation simply bounced off you. When your market position and control over development is dominant you can steer the web in self-interested ways that hurt users and hurt the independence of the web as an ecosystem. That's a perfectly legitimate answer to your Q and it's basically stuff they already said the first time around.
You can't really replicate that unless you're a trillion dollar company that chooses software defaults on billions of devices and have the world's most visited website.
Copyright law is about finding a balance between, on one side, incentivizing creators via granting them a legal monopoly on publishing, and on the other side, the philosophical notion of freedom of speech, which would allow anyone to republish, remix, and repurpose any preexisting work. The original formulation of this legal monopoly was that it was supposed to be short enough that it would expire well within the average person's lifetime, a notion which has long been gutted via regulatory capture.
To be fair to your latter point, Massachusetts is arguably the biomedical research capital of the world, those MRI machines are being put to research purposes.
I’m reminded of a quote from my biomed days: “If there’s ever a zombie apocalypse, you can bet it started in Cambridge.”
I just thought it an amusing anecdote until COVID, and then realized it’s less that said virus would be developed here and more that we have the business idiots to let it loose.
Careful which Cambridge you are on about. I take you are referring to Cambridge MA, US and not Cambridge, Cambridgeshire off of England, UK.
I'll note that COVID-19 "broke out" in Wuhan a city which has a virology laboratory nearby - the Wuhan Institute of Virology, who perform research on viruses.
I'm sure that is completely incidental to the pandemic of 2020-21. Clearly, some rather nasty live food markets are to blame.
It wasn't your numpties wot did it then, it was another lot that time!
reply