Hacker News new | past | comments | ask | show | jobs | submit login
QEMU should move from C to Rust (vmsplice.net)
208 points by rwmj 5 months ago | hide | past | favorite | 209 comments



Is it worth the effort to write a "X should switch to Rust" post that ends with "but I personally don't have the appetite to do it"?

Migrations of this size aren't not done because they're technically bad ideas, but exactly because they're expensive, messy, and few people have the appetite or funding to do it. Proposing a solution to that is more interesting that just saying "switch to Rust!".

Also, I don't know much Rust (still on my PL bucket list), but usually when I dive into a more detailed analysis of such a migration the most relevant parts are around how much unsafe code will be required and whether or not integrating with the existing codebase has to be fine-grained or can be done in larger modules like Rust recommends.

I would naively guess that a VM would have lots of unsafe code. Are there any investigations on whether the recent vulnerabilities would have migrated to safe or unsafe code?

edit: mea culpa. I didn't read the second to last-paragraph closely enough, which at least proposes a modularized way of doing this with device emulation being written in Rust in separate modules.


Given that this person has apparently worked on QEMU for ten years, I do think that it's interesting, regardless of the answer.


In addition to the author being a long term contributor, the piece seems to have been a leader for a discussion[1] on the qemu-devel mailing list (where it seems to have been positively received).

[1] https://lists.nongnu.org/archive/html/qemu-devel/2020-08/msg...


Much of this is written in the context of the new vfio-user proposal he linked at the end. This is a new, more flexible mechanism for implementing device emulation in a separate process. That emulation code can certainly be written in rust, providing a gradual path toward conversion.

As background, vfio-user was at least partially born of some discussions and a proof of concept last summer for using SPDK to provide virtual NVMe devices into QEMU. These are intended to be high performance devices.

Except SPDK is written in C. So maybe the title should be rewrite SPDK in Rust!


Forget the last paragraph. Literally opened answering your question:

> My KVM Forum 2018 presentation titled Security in QEMU: How Virtual Machines provide Isolation (pdf) (video) reviewed security bugs in QEMU and found the most common causes were C programming bugs. This includes buffer overflows, use-after-free, uninitialized memory, and more. In this post I will argue for using Rust as a safer language that prevents these classes of bugs.

__found the most common causes were C programming bugs. This includes buffer overflows, use-after-free, uninitialized memory, and more__


Sure, but the alternative is investing in static analysis for C code, of which there are plenty of tools for.

My usage of rust for now has indicated to me that for most low level tasks you are better of using C or C++. It is more suited to it. There appears to be a lot of boilerplate in rust.


> but the alternative is investing in static analysis for C code,

Yeah, the author explicitly acknowledges that:

> Eliminating C programming bugs does not necessarily require switching programming languages. Other approaches to reducing bug rates in software include:

> Coding style rules that forbid unsafe language features.

> Building safe abstractions and prohibiting unsafe language features or library APIs.

> Static checkers that scan source code for bugs.

> Dynamic sanitizers that run software with instrumentation to identify bugs.

> Unit testing and fuzzing.

> The problem is, the QEMU community has been doing these things for years but new bugs are still introduced despite these efforts. It is certainly possible to spend more energy on these efforts but the evidence shows that bugs continue to slip through.


Yes? Half of those are classes of bugs that would probably still exist with the usage of Rust, that is, they're simply a diversion from what you expected and what actually happens that the borrow checker can't save you from. You need to separate all bugs from the bugs Rust can't introduce.

It's safer, in theory, to just do it in Rust first, but it is also important to consider the amount of time it actually takes to implement either.

Why is this garnering downvotes...?


First off, I upvoted you, and I wish people wouldn't just downvote.

That said,

> Half of those are classes of bugs that would probably still exist with the usage of Rust, that is, they're simply a diversion from what you expected and what actually happens that the borrow checker can't save you from.

It's unclear what you're referring to. The linked article is talking about vulnerabilities in QEMU drivers, which are almost exclusively memory safety issues from my own experience and memory. The post you're replying to is just about tools and techniques for avoiding them.

> but it is also important to consider the amount of time it actually takes to implement either.

No one has said otherwise and the article even explicitly addresses this.

So I find your point really unclear.


>>> but it is also important to consider the amount of time it actually takes to implement either. >No one has said otherwise and the article even explicitly addresses this.

What do you mean by this? The article might reference it (I don't see any reference to this), but if it does it doesn't actually explore what factors would influence costs and their magnitude.

That's what I'm looking for.


I mean that your statement implies that people are advising to perform expensive work without considering the cost, specifically the cost in terms of time, and that's not true. No one has said that. The article addresses cost in a number of places - maybe not explicitly the cost of time, but it seems weird to focus on that.

The article explicitly addresses that it's not so simple to just rewrite a codebase:

> Moving a large codebase to a new programming language is extremely difficult. If people want to convert QEMU to Rust that would be great, but I personally don't have the appetite to do it because I think the integration will be messy, result in a lot of duplication, and there is too much un(der)maintained code that is hard to convert.


That's just a token mention of it instead of actually evaluating whether it will be problematic or not. I bring this up as he is probably one of the best people to evaluate this in detail and he didn't; lots of people here are acting like he did.


It's garnering downvotes because a lot of HN readers have never worked with a very large and mature codebase before and understand just how crazy a migration is. They seem to think that writing in Rust magically eliminates most bugs. No... it eliminates certain classes of bugs but instead you get plenty of logical bugs.

The number of new bugs introduced in a migration would be enormous. It's not just "translating" the C into Rust-- you ideally need to write "idiomatic" Rust.

Rust is a fantastic language but what I despise about its community is how they seem to think every C/C++ codebase should be rewritten in Rust. Either it's a monumental effort or it would decrease the developer base in many cases. In the case of C++ template libraries then it's in some cases even impossible to replicate functionality.


Having been significantly involved in the work to integrate Rust into Firefox I can say that while it's definitely nontrivial it's absolutely worthwhile. Every C++ developer I knew at Mozilla that took the time to learn Rust wound up being impressed with it and preferring to write Rust over C++.


> In the case of C++ template libraries then it's in some cases even impossible to replicate functionality.

Rust has procedural macros that can replicate anything C++ templates can do, often more cleanly. And C/C++ are hardly immune from logical bugs.


> "Rust has procedural macros that can replicate anything C++ templates can do"

This is an entirely false statement. Wishful thinking makes a poor substitute for knowledge. Promoting wishful thinking against facts does all readers a disservice.


Welcome to the Rust world order domination brigade. /s


>> The problem is, the QEMU community has been doing these things for years but new bugs are still introduced despite these efforts. It is certainly possible to spend more energy on these efforts but the evidence shows that bugs continue to slip through.

While personally being a proponent of Rust, I’ve been wondering if some of the projects that have codebases too big to realistically RIIR any time soon might benefit from running statical analysis as a git hook server side rejecting any commits that don’t pass some set of rules.

Though, only having second-hand knowledge of statical analysis of C code through reading blog posts on the net, I am also of the impression that false positives are a common occurrence in statical analysis tools. So based on that I am thinking that it might either not be feasible, or at the very least it would be a major undertaking and you might need to continuously tune the rules and also bugs of this sort would perhaps still slip through.


> Is it worth the effort to write a "X should switch to Rust" post that ends with "but I personally don't have the appetite to do it"?

Could it be that he agrees in principle that Rust is the way to go but for some reasons (time, interest, etc) he does not want to do it.


  Is it worth the effort to write a "X should switch to Rust" post that ends with "but I personally don't have the appetite to do it"?
he made it to the front page of HN, so I guess the answer is "Yes"


Entertainment value is separate from practical value and feasibility.


Maybe the practical value is traffic?


A GPT-3 written nonsensical article made it to the front page of HN[0], so I don't think it's a good measure of value

[0] https://news.ycombinator.com/item?id=23893817


How would I go about confirming that your statement is accurate ?

I see several comments on the article saying it looks like it might have been written by GPT-3, but there's no confirmation.

There's also a comment on the article itself that is certainly suggestive but doesn't seem to come out and openly state that this was written by GPT-3.

Finally I clicked on the author's name and see a list of several articles but no disclaimer that this author is a bot.



The answer is "Maybe" but this developer won't do it. Someone else has to make it a reality.

Once again and eventually the response is still PoC||GTFO.


Given the author is a long time contributor, this post is actually a bad indicator.


> Proposing a solution to that is more interesting that just saying "switch to Rust!".

The firefox way would be more interesting, start by introducing Rust to the codebase, than write first components in it and switch it over time.


The blog post proposes an incremental migration.

From the next to last section "How can we change programming language?":

> The reason I am writing this post is because device emulation, the main security attack surface for VMMs, can be done in a separate program. That program can be written in any language and this is where Rust comes in. For vhost devices it is possible to write Rust device backends today and I hope this will become the default approach to writing new devices.

For all those criticizing the knee-jerk impulse to rewrite, you're arguing against the wrong post!


It seems that the controversies in the comments come from the inexact title. It should have been ,,Why QEMU should incrementally move from C to Rust'', or something similar, that reflects the content of the article better.


Or maybe people should read the post instead of reading too much into the title?


but the title is important, it claims that the article will give sufficient reasons for QEMU to stop using C in some natural sense and move to Rust.

if this is not the reflective of the content or it is a tongue in check exaggeration following a c common template (similarly to the "considered harmful" titles) it is reasonable to criticize the title

(it is less reasonable to criticize the article under the assumed content)


I'd expect a technical audience to be already aware that any such efforts can only ever be incremental -- so your elaboration would be redundant.


Whats with the treand of why X should move to Rust these days?

Not saying Rust is flavour of the week but this happens with a new language every couple of years. Everything doesn't need to be rewritten. You switch languages, you might fix some existing bugs but you open lots of other ones.


This blog post discusses specifically what would be gained by moving to Rust, and why Rust. The word "re-write" does not appear. It is actually suggesting that new work be started in Rust. It even suggests that re-writing what's there is a bad idea!


You'd never know that the blog post describes such an incremental strategy from all the top-level posts on this page rushing to argue against the straw man of a complete rewrite. :\


Like almost even those that claimed to read the article did so haphazardly.


It suggests two things at the same time - that a re-write would be great - "If people want to convert QEMU to Rust that would be great" - and that the author has other thoughts too.


The story for C/C++ software is a bit different. It has an order of magnitude more security issues than (similarly well written) software in other languages. Mostly because of memory issues. Partly due to undefined behaviour.

There's a lot of this software. And until recently C and C++ were more or less the only viable options for writing it. Now there's Rust, and it has the potential to bring rates of security issues down to the levels of other memory safe languages (or even lower - Rust actually has protections against things like thread-safety issues that languages like Java don't).

If the language didn't matter, then we'd all be writing in assembly code. It can make a big difference.


> C/C++ software [...] has an order of magnitude more security issues than (similarly well written) software in other languages.

I think that's very debatable. Certainly there are classes of bugs with C code that don't exist in other environments. And addressing that is worthwhile.

But "security issues" is a VERY big bucket, and most of them (and frankly most of the most dangerous) aren't about memory protection or heap management at all. Java doesn't prevent you from exposing a database with a default password. Node apps can have XSS vulnerabilities. Access to a Rust app can always be phished.

I know those all seem like specious examples, but the point is that you're using a very narrow definition of "security" (to mean The Stuff Rust Addresses) in one sense, while using a very different definition (Stuff That Is Really Important For Software To Do) in the premise to your argument.

The space between "well-written C++ code" and "well-written Rust code" is a very different number in those two spaces.

And in the one that actually matters... I just don't think it's nearly as big as people like to imagine.


> has an order of magnitude more security issues

But then there is a LOT more C/C++ software as well, so it makes sense that when you have a lot more code you'll also have a lot more bugs.

> than (similarly well written) software in other languages.

Why isn't that similarly well written software being used instead of the C/C++ software when the latter ends up with more bugs?

> And until recently C and C++ were more or less the only viable options for writing it.

I do not think that this is the case, there have been alternatives for pretty much as long as C existed. For example Object Pascal existed for decades before Rust and while it doesn't have as many compile-time guarantees as Rust does, it still is safer than C and C++, provides a similar amount of functionality while not losing in terms of low level functionality. Yet outside of a few exceptions (mainly during Borland's heyday) people still went with C or C++ despite those languages not providing anything special compared to it.

And there have been other languages too - i've seen Ada being brought up often as a much safer language (safer than Pascal too) with a strong type system, yet still being usable even on tiny devices.

What makes people stick with C and C++ when other solutions existed for years and how would Rust change that when other languages apparently failed?


This correct viewpoint is always down-voted because it is uncomfortable to the clique of something like 20 HN members that fanatically control every Rust thread.


> But then there is a LOT more C/C++ software as well, so it makes sense that when you have a lot more code you'll also have a lot more bugs.

70% of C/C++ security bugs are memory management bugs. Those don't (for the most part) exist in GC'd or other non-manually managed memory languages.

> Why isn't that similarly well written software being used instead of the C/C++ software when the latter ends up with more bugs?

Performance and low level control. Familiarity. etc.


(The statistic you're citing did not separate by language, it was "70% of security bugs are memory safety bugs" only, no language break-out.)


Thanks Steve. I'm going to have to do more research I see :)


It's because c or c++ hasn't had a decent alternative for a long time.

Rust has the ability to compete with them. Other languages don't. So it's ripe for distrupting projects that are normally written in c.


There have been alternatives to those languages pretty much as long as they existed. Pascal was a common alternative to C for example, as was Object Pascal for C++ (both appearing around the same time) and Ada also existed for years and provided a much safer alternative to any of those while still being usable for low level programming.

Yet C and C++ still persisted. What makes Rust different this time?


They don't provide as much advantage over c to make the transition worth it.

Rust has more advantages than Pascal. A single implementation. A decent package manager, lifetimes, sum types, all the infrastructure around sum types etc


Pascal had a single defacto implementation (UCSD Pascal at first, Borland later) for each of the platforms so that wasn't really much of an issue. Ada is a standardized language.

And C and C++ (especially C++) were not standardized for a long time and even after they were, most programs relied a lot on compiler-specific behavior.

Pascal does have sum types (called variant records) and AFAIK had those since the beginning. Not sure about how exactly Ada does its memory management, but AFAIK it isn't a free-for-all like in C or C++, most dynamic objects are stored in containers which handle memory and any custom memory management must be done in explicitly designated ways (which it doesn't sound very different than Rust's unsafe code).

So while Rust does add some stuff, i do not think that they are that much of an improvement over other existing languages (remember that many thought C was a step backwards compared to what other languages provided even at the time). Rust may add a lot compared to C, but very little when compared to other languages.

Also note that i brought up Pascal and Ada as examples here, but those aren't the only alternatives to C and C++ that existed, just a couple that came to my mind. Of course there have been other languages.


Ada has no equivalent to Rust's borrow checker. It uses C-like memory management (malloc+free) plus idiomatic patterns somewhat like C++.


I know, however unlike C++ in Ada releasing memory doesn't seem (from the little i've seen) as straightforward as in C or C++ (or Pascal for that matter) so it is a bit safer than those, even if not as safe as in Rust.

Which is really what i refer to above - compared to C or even C++, Rust does add some things but other languages that existed for years (decades in some cases) added to C and C++ and compared to those Rust doesn't add that much (which isn't the same as adding nothing). Yet these languages weren't really adopted and people remained with C/C++.

Do you really think that what Rust provides will succeed in making people adopt it when other languages failed (and let's be honest - while Rust adds some stuff it isn't like it has all the features other languages have either)? Is it even a matter of features (be it about memory safety or anything else) in the first place?


> Do you really think that what Rust provides will succeed in making people adopt it when other languages failed

Given the ever broadening adoption of Rust, I'd say it'll definitely have a large foothold. I don't know that anything, no matter how good it is, will every fully replace C or C++. But I think a lot of projects that don't need a specialty compiler (embedded or stuff like game consoles) will start picking up Rust instead of C or C++.


Can you pattern match using variant and use it as a result type, or option type with associated bind, map and other functions?


You can use the variant as a result, yes. It is somewhat like a struct with a tagged union in C.

The only functionality standard Pascal offers is the simplest "case" statement where you can match against the variances with only little more functionality than C's select statement (you can have multiple values per case and you can use ranges), but nothing more than that.

In modern Object Pascal it might be possible to implement the functionality using generics and anonymous functions (currently only available in Delphi but soon also in Free Pascal), but AFAIK there isn't any functional programming-like functionality out of the box. In general Pascal is largely an imperative programming language and even in the more advanced dialects there is almost no functional programming functionality.


Pascal is a neat little language, but are you really trying to claim that its features are on par with Rust? The grandparent poster might not have picked the best examples (I’d have chosen traits, zero-cost lambdas and syntax macros), but the point is, Rust’s abstractions are inspired by decades of PL theory research that haven’t made their way into any Pascal dialect that I know of.


No, i only mentioned that sum types exist in Pascal - and existed in other languages too before Rust. But my main point of bringing up Pascal was that it existed for decades before Rust, it provides at least the same functionality as C (in fact, even late 80s Turbo Pascal has more functionality than C), is safer than C while still providing as much low level functionality as C, yet it never supplanted C.

Which makes me wonder how much programmers who use C really care about those features in the first place.


> A single implementation

I consider this one my biggest concerns about adding a Rust dependency to a core package. Especially when in https://forge.rust-lang.org/release/platform-support.html "Tier 1" is so small.


The existence of HN and a way to get developer attention?


Rust competes on performance and portability far better than Ada did at the time. It's also at parity in features and price, whereas Ada lacked proper OO and cost money until the Air Force paid for GNAT in the '90s. Modern Ada is a nice language, but pre-1995 Ada lost to C++ for a reason. Soon after that, Java showed up and offered a new, easier way to write safe programs, while also locking in the expectation that "modern languages look like C".

Pascal, by contrast, simply didn't offer any compelling advantages.


I've used a decent amount of Rust, that's not the case this time. Rust is a game changer. The effect of borrow checker and automatic RAII everywhere is profound. I'm convinced its going to fundamentally change low level language design. Borrowing semantics will be mainstream someday, whether its through Rust or otherwise.

It has all of the safety characteristics of a VM language but its low level with no runtime. Its a direct replacement for C with all the performance and modern design of languages like Java and Go. And it uses LLVM so FFI with other languages that compile to native is simple and zero overhead. You can even do whole program optimization (LTO) on the combined binary, that's huge. Trust me the hype about Rust is real.

Much more real than the hype around Go, which to me just feels like yet another GC language. The hyped AOT compilation is nothing profound. Since the GC forces Go to have a runtime anyways IMO they should have gone all the way and just used JIT. And Go doesn't use existing C compilers so FFI has overhead, can't be optimized together at a bytecode level


I think this is a bit overstated, because Rust does not yet address all use cases of a language with tracing GC (such as general graphs, or some specialized concurrent data structures), so something like Go will still be useful in many cases. Though this might change in the future if Rust itself gains better support for "pluggable", local allocators, whicn might subsume tracing GC when needed.


It definitely doesn't. Rust needs a GC for certain use-cases. I've been hoping for a while that somebody will do a port of Shenandoah or ZGC to Rust. It would be a lot of work but give Rust the best of both worlds


Because sometimes a software project has specific problems which Rust specifically address.

(Not that I'm personally a big fan of all these call to rewrite in Rust posts, there's a reason the Rust subreddit discourages ”X should be rewritten in Rust” posts)


> (C++) programming is hard, let's go (language) shopping!

There isn't really a trend, those who can are doing it in C++, those who can't are teaching us about the wonders of rewriting in a more fashionable language.


All bullet points under "Why Rust?" actually suggest that it would be much better to gradually rewrite all critical parts of the project in ATS, thanks to it being a frontend to GCC. Firstly, it's a much more powerful frontend than the current Rust. Secondly, I'd expect it to be less disruptive than a complete compiler switch.


If a rewrite is warranted, and despite Rust zealotry being annoying at times I think it has fair points, I think going for something even more hipster than Rust would be a mistake.

I wouldn't even call Rust "hipster" anymore. It's already been embraced and rolled out to production by big companies working on hard techinical problems (Cloudflare, AWS, Discord), as well as the crypto community.

I think the reasons from steering away from hipster languages for these critical projects are roughly the same ones we use to justify not using hipster crypto for critical, real-world applications


I also would call Rust the new normal for fast secure code.


I still do not understand. Ada is not exactly new. Have you ever seriously considered Ada/SPARK? Might want to give https://www.adacore.com/uploads/books/pdf/AdaCore-Tech-Cyber... a read, or even https://blog.adacore.com/, or my previous comments on Ada. I called Ada the programming language for writing "fast secure code" for quite a long time now, a "better C" in that regard.


Ada just doesn't have the steam behind it.


It certainly lacks the hype, and judging by the comments on here, many people still have loads and loads of misconceptions of the language, negative ones, unlike say Rust's "unsafe code does not affect the rest of the program", for example, which I would call a positive misconception. And of course thus, it also lacks libraries. This is really my biggest problem with it. :/


Rust isn't nearly popular enough to be referred to as "the new normal" in any field or context. One would have to be in a very closed off bubble to think it is. If they haven't made some big strides with having it certified for use there are entire industries where you can't even use it and get certification, if I understand correctly.


I wouldn't really call Discord a hard problem. It's web chat.


You might have missed the "technical" qualifier.

Twitter isn't an especially hard problem either.


I didn't miss the qualifier. Discord is a chat client with very restricted servers. It's really not that hard as technical problems go.


Please, do tell us what kind of exotic problems you work on, that you think massive scale real-time communications is not a hard technical problem

We're eager to know!


No, that's stupid. I couldn't build Discord. My point is that as relative problems go, I don't see Discord as a big one. I might be wrong but as far as I can tell it's a solved problem and has been for over a decade. I'd wager you could build it with an incompetent team that had domain expertise.

AWS is orders of magnitude more complicated than Discord. Frankly, so is World of Warcraft.

What does Discord do that wasn't solved in 2005?


> AWS is orders of magnitude more complicated than Discord. Frankly, so is World of Warcraft.

AWS maybe, insomuch as "orders of magnitude" can mean as little as "100x". But still, unconvinced. There's something like 200 AWS services, and Discord on the whole is certainly more than twice as complex as the average AWS service.

World of Warcraft being >100x more complex than Discord doesn't track.

> What does Discord do that wasn't solved in 2005?

Perhaps you are unaware that Discord has (pretty high quality!) voicechat and livestreaming? Certainly much better than anything that existed in 2005.


> What does Discord do that wasn't solved in 2005?

Discord was pretty much unrivaled in voice-chat quality for a long time, and to this day it's only "real" competitors in technical prowess in this space that come to mind are Apple's Facetime and Google's Duo.

Both Facetime and Duo have bespoke algorithms for this kind of thing: Duo for example leverages SOTA ML to correct audio gaps in the call[0], while Facetime leverages ECN on their custom adaptive bitrate algorithm so that the call is more resilient to noisy networks (couldn't find a good reference for this second one).

Keep in mind that both Google and Apple have highly-qualified teams working exclusively on networking protocols, and yet still there was a time a while ago where Discord pretty much had the best call quality experience around (GMeet is still wonky to this day).

The only thing about this that was "solved" in 2005 is shoveling packets over UDP, or at most, if you wanted to be really fancy, just shoveling them over SCTP instead.

I think you're severely underestimating how much knowledge of networks it takes to build something to par that level of quality. The only thing you can point at back in 2005 is Skype, and they do not come even close. You also seem to be equating difficulty with complexity, which even though they are at times correlated, is still a false equivalency.

[0] https://ai.googleblog.com/2020/04/improving-audio-quality-in...


From a cursory search, it looks like ATS is implemented as a "compile to C code" language. This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.

Also, from what I can tell of its mailing list, ATS is a pretty dead project: https://sourceforge.net/p/ats-lang/mailman/ats-lang-users/ shows only announcements post-2015.


A language that compiles to C does not inherit all of C's problems, and doesn't "punt" on it for undefined behavior. I mean, after all, Rust compiles to Assembly, which is even unsafer than C, but Rust is still considered safe.

A compiler has greater ability to enforce invariants that make the resulting code safe. The issues with C result from faulty humans; running through a program gives an opportunity for those issues never to occur to begin with.


> A language that compiles to C does not inherit all of C's problems, and doesn't "punt" on it for undefined behavior.

ATS avoids some things; you shouldn't be able to do the silly memory-unsafe parts.

Stuff like dividing by zero is permissible by default but can be avoided.


How is assembly less safe than C? C is full of undefined behavior which might be interpreted differently by different compilers, whereas (maybe with the exception of some unusual circumstances) the behavior of assembly language for major instruction sets is 100% documented.


I see that you have not coded in assembly language.


I haven’t; I’ve only read it a bit. I could be wrong about a relative lack of UB compared to C - do you have examples?


When you code in assembly language, you discover how incompletely documented every chip is. Any detail of behavior not documented might change in a later chip, and is thus UB.

Makers of compatible chips are obliged to keep a collection of systems with all the extant chip designs to run experiments against, and to maintain their own documentation of discovered but unspecified commonalities that users' code may have come to depend on.

Each released chip, and each stepping of each release, has its own unique set of bugs-- deviations from documentation, from observed behavior of previous chips, and of chips yet to be released.

Writers of assembly code need a compendium of behaviors, documented or not, that cannot be relied on for the set of release platforms. Many such bugs depend on an ill-characterized history of processor states, making them, often, (more or less) reliably reproducible only with a particular program that was discovered to evoke each. Often the behavior noted is seen only once in a million runs.

An example on Intel is the popcount instruction that runs very slowly, sometimes, in chips released up to a few years ago. The instruction appeared along with ...SSE3?, and had the bug until it was noticed and fixed in a subsequent chip. The bug is the reason Gcc uses the same register for source and destination when it emits this instruction. Intel's ISA manual does not mention this behavior.


Thank you for the details. I guess I had an overly rose-tinted view of the quality of Intel documentation.


It's far from dead, a new version of the compiler is being developed on top of ATS2 (https://github.com/githwxi/ATS-Xanadu). ATS2 is being updated too (https://github.com/githwxi/ATS-Postiats), the latest stable release happened last December:

https://github.com/githwxi/ATS-Postiats/blob/master/CHANGES-...

https://sourceforge.net/projects/ats2-lang/files/ats2-lang/


> This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.

I think it's worth pointing out that you can't always get away from those undefined behavior situations even if you skip C all together. Rust, for instance, has some internal/implementation details I've run across that are just C undefined behavior details leaking out of the LLVM IR instructions they use to implement the language, since those instructions have the same undefined behavior semantics as the corresponding C code they're used for.


Not always, but a lot of the time. There are also a whole bootload of things that are UB in C but have defined semantics in Rust.


Sure, I was mostly responding to the "compiles to C vs. directly to GCC/LLVM". I don't think the difference is actually all that relevant in terms of avoiding undefined behavior, a lot of the undefined behavior situations still need to either be worked around (or accepted) when generating the IR - it's not as simple as "skip C, UB is gone".


It’s not more concerning than a language that is compiled to assembler. It all depends on the quality of the compiler/transcoder.


> This kind of implementation is actually quite concerning, because it means you're punting through a language which is notorious for its heavy reliance on undefined behavior, and there is no way to opt out of undefined behavior in some cases.

if it were a new project I'd agree, but if there's an existing C codebase, it already contains all these UBs and the goal of the rewrite is to grow a formally-verified critical core functionality in ATS that would still inline the rest of the project intact. This way the formally verified part of the project can grow naturally through small iterative steps without major disruptive changes to the tooling and supportive infrastructure.

People who are familiar with C can acquire basic ATS patterns pretty quickly (within a month or so), and the same is applicable to those who are familiar with ML-family languages.


Does Rust ultimately not compile to machine code? Is the underlying hardware proven? You are ultimately going to get your code punted through untyped data lines and unproven logic units.


Rust has the good sense to keep all its Undefined Behavior behind unsafe blocks.


It's a google group! Not dead at all, though it is... ATS :)


> ATS

For anyone else who was unfamiliar: https://en.wikipedia.org/wiki/ATS_(programming_language)


Is ATS really suited to being an industrial language? How do its standard library, build tools, and library ecosystem compare to Rust's?

Admittedly, if you're switching from C, you start with a pretty poor story about all that. But at least that story is widely and thoroughly understood.

ATS has an optional garbage collector. It seems the standard library has two implementations of each container type, one which uses GC, and one which uses linear types, so does not need GC, but is more demanding to use. That is pretty cool! What is the situation with other libraries? Is there a convention of assuming GC can be used, or that it can't, or of supporting both?


> How do its standard library, build tools, and library ecosystem compare to Rust's?

as ATS is a frontend to GCC, it means that the standard library can be any battle-tested C standard library. On top of that there can be a formally-verified interface that calls underlying C API. Here's an example - https://bluishcoder.co.nz/2012/08/30/safer-handling-of-c-mem...

There's also a safe Prelude and common data structures, implemented in ATS:

http://www.ats-lang.org/Libraries.html

> Is there a convention of assuming GC can be used, or that it can't, or of supporting both?

afaik there's no officially defined convention, but you can have a look how this switch may look like in practice in this video - https://youtu.be/Xkg_EmERYRE?t=1742


Back when it was on the PL shootout, ATS was consistently in the top five for performance. However, it was also consistently the worst in program size, comparable to Ada (but even more loquacious). Between that and the lack of existing uptake it would probably be difficult for QEMU to adopt.


Article author here. Thanks for sharing ATS. I will take a look at it.


Hey, no worries!

Personally I think this playlist provides a nice set of real-world examples implemented in ATS:

https://www.youtube.com/watch?v=dsILVOqYCU0&list=PL6BIXG1a4e...

And this series of blogposts is a pure gold:

https://bluishcoder.co.nz/tags/ats/

And here's the official language introductory resources:

Language Tutorial - http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...

Language Features Overview - http://ats-lang.sourceforge.net/DOCUMENT/ATS2TUTORIAL/HTML/A...


What’s the status of the Rust project after recent Mozilla layoffs? How much does Rust depend on Mozilla?


Mozilla employs a small number of people to work on Rust fulltime, and many of the Servo people contributed to Rust too, even if it wasn't their job. They also pay for the hosting bill for crates.io. They also own a trademark on Rust, Cargo, and the logos of both.

Two people from the Rust team have posted about their situation, one was laid off and one was not. Unsure about the others. Many of the Servo folks (and possibly all, it's not 100% clear yet but it doesn't look good) have been laid off.

The vast majority of contributors and team members are non-Mozilla, though the Mozilla folk are very important. Rust will survive. This situation is very painful, and it has the possibility of being more so, but (thanks to a lot of work from people (not speaking of myself here) who used to be at Mozilla!) Rust is bigger than Mozilla, and has been on a long road towards diversification. Some other companies have stepped up to pay a bunch of the bills that Mozilla used to pay, namely Amazon and Microsoft. Here's hoping that we get even more involvement in the future.


If Servo is a dead end (by the sound of it), what is the new "halo project" that showcases Rust's use in something non-trivial and widely-used/public? (Not trying to rub salt in the wound, geniunly interested.)


Tons of stuff. We've got everyone in FAANG but Netflix, Microsoft, Cloudflare... different people have different definitions of "widely-used/public."

Anyway, two big projects in a similar space to QEMU are Firecracker (the underlying virtualization tech for AWS Lambda + Fargate), and Crostini (the underlying virtualization tech for ChromeOS). Firecracker started life as a fork of Crostini and now there's a rich ecosystem going on in this space.

Don't forget the Rust code from Servo that lives in Firefox, too.


Thanks. With "widely-used/public" I was hoping for something open source that runs on many people's machines (like a web browser) or at least on many company's servers (like qemu) as opposed to running on a single company's servers in a controlled and uniform environment.


Both of those things are open source, incidentally :)

Yeah, it's tough to get the combo of all of those things, for other reasons too. I would probably want to first reach for VS: Code, which uses ripgrep for search. But that's a "important component written in Rust", rather than a whole-Rust project. There's tons of tools like ripgrep, which is the most well-known, but it's hard to say how popular they are. And there are things like 1password, which do run on people's machines, but aren't open source.

I actually have recently seen a bug report that implies something really, really huge in this space, but I don't want to jinx it, so I'm not gonna link just yet. We'll see :)


The crosvm project which powers crostini is both open source ( https://chromium.googlesource.com/chromiumos/platform/crosvm... ) and on many people's machine (most Chromebooks) which might fit some definition of not controlled and not uniform.


Firecracker is fantastic, but it's also --- to its great credit --- substantially smaller than Servo.


100%. I picked it because we're in a thread about QEMU, and it's an important part of a well-known product. I am also glad that it's a lot smaller, as it should be. I figured that, when discussing a "halo project," popularity is more important than sheer size.


A lot of the code from Servo is now in Firefox, and will live on.

If widely-used doesn't imply open source then there's a lot of options. Dropbox use Rust in their sync client, as well as in their storage backend. Cloudflare are using it for their VPN. Microsoft are rewriting parts of windows in it. Amazon are using it for their Firecracker micro-vms that power lambda.


To be clear here, while Servo is a wonderful project and was definitely pivotal in making sure that Rust was useful for building real-world software, calling it "widely-used" is a bit of a stretch. The individual crates that were written to implement pieces of Servo have found widespread adoption, but Servo itself was not something that had millions of users. (n.b.: we did ship Servo's style engine in Firefox as part of Quantum, but that was just one piece of Servo.)


I imagine AWS wouldn’t abandon Rust after the investments they’ve made in using it to speed up their VMs/Lambda. And they are a sponsor.

https://aws.amazon.com/blogs/opensource/aws-sponsorship-of-t...


Also Microsoft/GitHub are sponsoring their build servers, so that's not a cost for Mozilla https://blog.rust-lang.org/inside-rust/2020/07/23/rust-ci-is...


There were talks about creating Rust's own Foundation, to secure financing independent of the mess that Mozilla has become. Not sure if that went anywhere.


We’ve been working on it, and be “we” I mean “not me personally but the core team generally.”

It’s not so much about finance as it is holding the trademark, imho.


Cool, hope it works out and soon, I have little trust left for Mozilla under current management. Wish you all the best (and by "you" I mean you and also the entire Rust team!)


The message in Mozilla laying off the Servo team, for me, isn't so much that Rust is dying as it's about a team making their programming language a priority can't at the same time focus on their actual task ie developing a browser. What a missed opportunity to specify web layout in a constraint formalism (and I hate to always be right).


The Servo team and Rust team were two different teams, so this doesn’t follow.


Evidently they can. The Servo parts are the only things that are fast in Firefox.


The Servo parts in FF is a CSS "engine" right? My expectation is that at some point it gets rewritten to C++ like the rest (layout, RT, API, fonts, parsing, JS, UI) because having a minor part written in Rust (with an extra build system, test framework, API bindings to maintain) is only a hindrance to building a team able to pickup work of other members. Especially when that other language has failed to deliver, and not even demonstrated the feasibility to develop a competitive JS VM, nor fast-enough binding to eg FreeType.


Stylo, the component you're talking about, was written in Rust after multiple attempts to write it in C++, which all failed. So, I would hope that that isn't what happens. It sounds like a waste of time.

> having a minor part written in Rust

"every single CSS bit touches this code" isn't a minor part, and it's not the only part. https://4e6.github.io/firefox-lang-stats/ is a sorta flawed metric but it reports nearly 10% being Rust.


WebRender is the biggest part in FF.


The following justifications are given:

-Use-after-free, double-free, memory leaks, and other lifetime bugs are prevented at compile-time by the borrow checker where the compiler checks ownership of data.

-Buffer overflows and other memory corruptions are prevented by compile-time and runtime bounds-checking.

-Pointer deference bugs are prevented by the absense of NULL pointers and strict ownership rules.

-Uninitialized memory is prevented because all variables and fields must be initialized.

These are all only true in safe rust. A hypervisor has to do low level things like translate memory addresses, move stack pointers around, etc. Nearly all the buggy C code, if implemented in Rust, would be wrapped in 'unsafe' and would be just as buggy.

I recommend everyone interested in Rust to take a look at how the standard library Vec is implemented[1]. The code is littered with `unsafe` - There is just no way to avoid it and do something useful at the same time.

This isn't really a shortcoming of the language, but of modern computers. It is fundamentally an unsafe operation to read or write memory, to communicate with a disk, etc. Every computer program is fundamentally unsafe. Rust does a fantastic job of making this explicit and controlling it as much as possible, but the idea that Rust code is bug free for free is just wrong.

[1] https://doc.rust-lang.org/src/alloc/vec.rs.html#300-303


I think the Vec example makes the opposite argument from the one you put forth. Vec uses unsafe internally in order to have the desiref API and performance characteristics, but it exposes a safe API. If there are bugs in the unsafe blocks that can cause UB all over the place, but it also makes it easier to audit.

> There is just no way to avoid it and do something useful at the same time.

I don't think that's true. Your code might have to rely on code down the abstraction layers that uses unsafe, as interacting with the hardware is inhere you unsafe, but that doesn't mean that any code you want to write won't be able to leverage the safe abstractions. The argument you're making sounds to me, and please correct me if I'm misunderstanding, that trying to create safe abstractions is sheer folly so we might as well not do so.

I've been writing Rust code for years now and I've never had to write unsafe code myself other than when interacting with C code and when doing so I've found issues with the way the API the C libraries were exposing having fundamental issues.


> Your code might have to rely on code down the abstraction layers that uses unsafe, as interacting with the hardware is inhere you unsafe, but that doesn't mean that any code you want to write won't be able to leverage the safe abstractions.

This is the point I was trying to make by linking to the vec code. Any useful Rust program will run unsafe code. That code might just be hidden in a library. Ultimately, you are trusting that the code author did not write any bugs - in this regard you are not much safer than using the C++ stl.

> The argument you're making sounds to me, and please correct me if I'm misunderstanding, that trying to create safe abstractions is sheer folly so we might as well not do so.

On the contrary! I think what Rust is doing is a big step forward. But it is very important to keep in mind the limitations of a system.

The authors attitude of "This is written in Rust, so it must be free of memory bugs!" is dangerous. The key benefit of Rust is to isolate and explicate the dangerous parts of your program (because there are dangerous parts!!). This should make you more vigilant of bugs, but the "Eh, the borrow checker will debug for me" attitude can often lead to programmers being less critical of their own code.

Of course, none of this really applies if you are doing higher level stuff. But QEMU would definitely have to write their own 'unsafe' code.


Here's the thing these arguments always miss: writing C/C++ is like writing your entire Rust program using unsafe. Sure, unsafe Rust code will always exist and can cause memory safety bugs. It's a lot easier to verify a few unsafe blocks in a Rust program than it is to keep all of your C++ program's invariants in your head and manually make sure you don't violate them.


> These are all only true in safe rust. A hypervisor has to do low level things like translate memory addresses, move stack pointers around, etc. Nearly all the buggy C code, if implemented in Rust, would be wrapped in 'unsafe' and would be just as buggy.

> I recommend everyone interested in Rust to take a look at how the standard library Vec is implemented[1]. The code is littered with `unsafe` - There is just no way to avoid it and do something useful at the same time.

I think Vec is a somewhat extreme example in terms of unsafe. It requires to be able to realloc, or allocate a new buffer and move all elements; those operations do not fit into Rusts safe memory and lifetime model. If you are writing some form of container like Vec or Hashmap, I would agree with you; Rust doesn't offer much help in preventing those memory bugs.

However, there is tons of other code than can be written in safe rust. Parsing binary data formats in particular is an area that can result in buggy C code and can be solved in Safe Rust. If you were referring only to the buggy code in qemu, that may very well be true. However, I would suspect there large swaths of code that don't do anything particularly interesting, but do allocate and free which Safe Rust might help with.

> This isn't really a shortcoming of the language, but of modern computers. It is fundamentally an unsafe operation to read or write memory, to communicate with a disk, etc. Every computer program is fundamentally unsafe. Rust does a fantastic job of making this explicit and controlling it as much as possible, but the idea that Rust code is bug free for free is just wrong.

If you trust the standard library to abide by Rust's memory invariants, you totally do unsafe operations without letting program state go into an undefined state. I don't think I have ever had a Rust program using only safe rust (in the main source code) segfault and that is simply no where close to true for C programs I have worked on and other tools I have used.

The feature that I like best about the safe vs unsafe Rust is that if I only use well established libraries and only write safe rust, I can almost completely ensure my code won't segfault (even if I am incompetent). Panics are still a problem, but they are more manageable.


Thanks for spelling this out. When I read the post I was actually wondering if there isn't a fundamental trade-off between fast and safe. A hypervisor's basic function is to add the minimum number of CPU cycles to emulate a real machine, its memory and its devices. Using safe operations, which would add a lot of cycles to do checks, would make QEMU a worse hypervisor.


For the Rustaceans in the thread, I've been wondering - What's the current state of affairs of Rust for non-x86_64 targets? I know it's in flux, and that documentation isn't always up to date.

What architectures can rustc target? Can it cross-compile? Can I cross-compile rustc itself, or do a Canadian-cross (where I cross-compile a rustc that wants to target a different architecture than the one that hosts the compiler)?

Speaking as somebody who does a ton of system programming for embedded Linux systems, this is basically C/C++'s biggest advantage over Rust - the extreme flexibility of the available toolchains.


Every Rust compiler is a cross compiler. There can be some issues on some combinations of cross compiling, that is like, you can't run link.exe on a Mac so cross compiling from a Mac to Windows only works if you don't use the msvc target, stuff like that. I personally work on embedded ARM stuff on Windows, so I'm cross compiling from x86_64-pc-windows-gnu to thumbv7em-none-eabihf all the time. No need for a special build or setup, just a "rustup target add thumbv7em-none-eabihf && cargo build --target=thumbv7em-none-eabihf" and that's it.

ARM themselves are committing to getting the various ARM targets to Tier 1 support. AVR support landed in nightly recently. Other targets have less support, but that's mostly due to lack of people power, not a lack of interest from us.


I know you know this, but just to clarify for other people, "Every Rust compiler is a cross compiler" is only true for pure Rust code. You'll still need a C cross-compiler toolchain to link your final binary, and any Rust libraries that are bindings to external libraries and expect those libraries' C headers / link objects to be pre-installed [1] will need said external libraries' C headers / link objects to be available for the target architecture.

So the availability of a C cross-compiler toolchain can affect your ability to compile Rust code for foreign architectures. As a concrete example, at $dayjob we need to compile our code for CentOS 7 armv7 and aarch64, but CentOS 7 does not have a usable cross-compiler toolchain in its repos [2], so we have to run the build process in a centos7 arm container running under qemu.

[1]: Eg openssl-sys is bindings to openssl and expects you to have installed the equivalent of your distro's openssl-devel package. In cross-compilation scenarios, this means installing the openssl-devel package for the foreign arch. For debian you'd do something like `dpkg --add-architecture armhf; apt install libssl-dev:armhf`

[2]: What it has is only for cross-compiling the kernel, so there's no cross glibc available to link the final binary to.


Ah yes, thank you. I should make sure to clarify that better in the future :)


The biggest limitation for embedded Rust development tends to be "does LLVM support your CPU?" If not you may be able to hack something together but you're not going to have a great time. If so, there's still a broad range of experiences between "people are actively writing Rust for this board and there are tons of crates exposing the hardware with nice APIs" and "you can get Rust code running on this board". I'm not an embedded developer but I've dabbled enough in the space to believe that Rust is going to do very well there. The development experience is so bad for most existing cases and there aren't a lot of other viable options (unlike for folks writing desktop or server software).


You can find the target support levels at https://forge.rust-lang.org/release/platform-support.html



ARM is very well supported. I have had zero issues using a Raspberry Pi. As for cross development there is an excellent tutorial targeting the STM32F3DISCOVERY dev board, https://docs.rust-embedded.org/discovery/

I attended a hack session where the attendees had zero Rust experience and they were able to get blinken led running in under an hour. Basically 10 fullstack web developers were able to open the package on the devkit, install rustup on windows/mac/linux and start writing firmware and updating the board.

Pretty solid I'd say.


I’ve been cross compiling for both Android (x64, i686, arm and arm64) and it works great. Rustup allows you to install additional targets with a simple terminal command. I haven’t tried anything more exotic than that, though.



I think the real benchmark is a large "C" project writen fully in Rust and to compare the issues on the long term, not sure it's safe to asume that Rust would fix all the security issues with all the unsafe code that would be?

What a 1.5M LoC Rust low level kind of project looks like?


> not sure it's safe to asume that Rust would fix all the security issues with all the unsafe code that would be?

Hi, Article author here. I am not assuming Rust will fix all security issues. Bugs are still possible even in safe code, plus there will be unsafe code as you mentioned.

Unsafe code also worried me when evaluating Rust. Given the nature of Virtual Machine Monitors and emulators, isn't a lot of the program going to be unsafe? After studying existing Rust codebases and writing programs myself I found that it's still a significant improvement over C.

Whether Rust is safe enough depends on your requirements. For the QEMU, I think the answer is yes. For writing the firmware for a pacemaker implant or control software for a nuclear powerplant, I don't know :).


My favorite project to point people towards to understand how unsafety can be highly isolated is the work here: https://os.phil-opp.com/

That very low level educational project can show how little unsafe is needed even at things running on bare metal.


> I think the real benchmark is a large "C" project writen fully in Rust and to compare the issues on the long term, not sure it's safe to asume that Rust would fix all the security issues with all the unsafe code that would be?

I think you overestimate how much unsafe would be required. Most code, even low-level code can be safe.


Honestly, I think anyone who has written Rust, even low level Rust, intuitively understands that it wouldn't even be close. It's not really worth measuring - like measuring C vs Java, no one really needs to do it because it's so obvious that C will have many times more memory safety issues.


Bad science.

Rust code does have unsafe at times, so it can still have memory safety issues.

It may not matter that your program is memory safe if it was written by people who don't understand the problem domain and doesn't work correctly. E.g. for qemu, it's perfectly possible to be absurdly vulnerable -- giving target code control of the host -- without any memory safety bug at all.


In fact it's not science at all. I didn't claim it was science, I claimed it was so obvious and intuitive that science was unnecessary.

> without any memory safety bug at all.

Most QEMU escapes I have seen are memory safety issues in emulated devices.


Well, actix-web is a pretty large project written fully in Rust and it never really had any issues with lack of safety.


The irony of this statement when the author was hounded tirelessly by the Memory Safety Mafia until he plain didn't want to work on it anymore...


It was meant sarcastically.


Telling QEMU to move from C to Rust is bad idea imo. QEMU Devs are probably well versed in Source code of QEMU and C. Telling them to learn new language and rewrite QEMU would be a joke. It would give lot of new problems than solving the current issues.

Probably, RUST Devs who are well versed in emulation can start another project in RUST which has same goals of QEMU but telling existing software to rewrite is a big JOKE to be laughed at.

This has nothing to do with RUST but "Rewrite in New language" syndrome.


A major QEMU contributor suggesting that future work on a specific subset of components (that are prone to vulnerabilities that are difficult to avoid in C) be done in Rust is not naïve. It's the beginning of a community discussion.


Hello.

I'm not a QEMU developer. I used to be pretty active in Apache HTTPD, APR and related projects. They are large user space code bases written in C - with, overall, a relatively OK security record.

I think Apache should be on a path to be rewritten in Rust. Here's why...

Every new line of C has a chance of a remotely exploitable vulnerability. No matter the precautions taken, no matter the developer who writes it. HTTPD has a long history, precautions like memory pools from APR and uses "safe" string functions, but it still has issues.

Apache's core C code doesn't change much, so yes, many remotely exploitable memory corruption bugs have been fixed over 20+ years, but new code, new code still has bugs. And they can be bad.

Look at the recent known vulnerabilities in httpd: https://httpd.apache.org/security/vulnerabilities_24.html

Vulns that are largely fixed by Rust are concentrated areas like HTTP/2 support and the Proxy - areas under active development. Vulns around logical errors are still there, and more widespread, but largely have a lower impact.

So, I emphasize with QEMU developers. I'm in the same boat. I have a full time job, and that job is not to work on Apache HTTPD. It would take a full time job to incrementally adopt Rust. The outcome would be in 24+ months, no more remote memory corruption bugs. Is it worth it to fund this? Can you put a price tag on this for the dozens of "core infrastructure" software projects written in C?


No offense, but rewriting httpd in rust won't make it any better. There's no excuse for thread-per-connection in 2020.


Good thing Apache has mpm_event https://httpd.apache.org/docs/2.4/mod/event.html


in a hilarious turn of events (to me at least), I was one of the primary developers of the event mpm.


Alright, so please correct me if I'm wrong then.

I cannot perform a non-blocking operation (querying redis for example) from within my module's handler function because as soon as my handler returns httpd sends a response. i.e. there's no 'NOT_DONE_YET' return code?

And while I'm anonymously ranting on the internet...

- being able to control response headers completely would be nice. In our case, the Date and Server headers cost money to send and our clients don't care about them. It's not for security reasons, it's about traffic egress costs on the cloud

- having a way to hook a worker thread's init would be useful

- there must be an easier way to get a request body other than the bucket brigade stuff?


1) You return SUSPENDED instead of OK from your handler. See also http://svn.apache.org/viewvc?view=revision&revision=1593860

2) You can control all response filters at the byte level with an Output Filter.

3) Hooks already exist for this: https://ci.apache.org/projects/httpd/trunk/doxygen/group__ho...

4) No. Bucket Brigades is the API for this. It's an efficient linked-list of future-data. It's a good API IMO, just needs some understanding, and because its in C, its full of macros (there are more alternatives to making it cleaner if we weren't using C)


1) potential gamechanger. Thanks for pointing it out

2) yes, but there's a config directive for the Server header too that could just allow people to turn it off

3) for child_init, but nothing obvious that maps to a worker_thread_init

Are there plans for httpd to avail of io_uring if the kernel provides it?

I will admit, I've struggled with configuring httpd for years. A single threaded redis server can handle more connections than a full httpd install.


LOL


That's not at all the issue they said that Rust would help with.


Yes but porting httpd to another language would just be putting lipstick on a pig!


First up, the person writing the blog post is actively involved in QEMU development. It's not some random person evangelising use of Rust. It's most definitely not naive for them to suggest such a thing, but instead exactly how such things ought to be done.

Secondly, you may be started to find out that the QEMU developers seem to be in agreement, with the interest related to Rust for vhost-user, and considering ways interfaces etc. might make that feasible.

OPs post to the QEMU development list, thread makes for interesting reading: https://lore.kernel.org/qemu-devel/CAJSP0QWF8g7r5VqU_PRbskWZ...


> This raises the bar to learning the language, but I look at it this way: learning Rust is humanly possible, writing bug-free C code is not.

It's simply impossible to write completely safe c code. Lots of bug like heart bleed, dirty cow, whatever, you name it was write by people that have dozens years of experiment and they still make accidents.

Switch the language is expensive of course. But it might worth it eventually if this project continue to run for a long time (or the project got abandoned before it pays off, who knows?)


Yeah, there's no discussion on this blog post of what a lot of people are doing in writing project-specific linters, static analysis passes, code generators and so on that just ensure higher code quality which to me is more bleeding edge when it comes to actually helping. There's also no mention of the fact QEMU is tightly bound to other C based projects like libvirt and KVM and having developers be able to switch back and forth is extraordinarily useful. It wouldn't just be replacing QEMU but also the ecosystem around QEMU. It's not like QEMU is the application at the end of a dependency chain, it's in the middle for a lot of users.


And maybe you should read the conclusion of the article before commenting on it?


Rust is the new silver bullet...


And an easy way to tell is there is no section in the post on what the downsides are to picking Rust.


There may not be an explicit section, but the post certainly talks about how it would be a mistake to rewrite the code, and gives reasons.


IMHO that is just a soft touch, it's just saying there might be some areas where Rust isn't a good fit. What the article doesn't address is what the intrinsic costs of switching are, as opposed to what the article does address is that there are probably places where it doesn't have a good fit.

In short the article doesn't address what new costs Rust might bring that are intrinsic to using Rust (as opposed to costs that are inherent in making a change to _any_ new language).


I find it odd that for the solution proposed for solving problems with C that are explicitly solved by C++ is not to move the code-base to C++ but to an incompatible language, like Rust. Nothing against Rust, but sounds more like someone who wants to work on Rust.


Wrong thread, fyi.


We've moved it. Comment was originally posted to https://news.ycombinator.com/item?id=24133146.

I always wonder about the specifics of how these happen!


> I always wonder about the specifics of how these happen!

Once I read someone saying they where blind and had forgotten the context or navigated back or something without noticing. I think there were some interesting follow up questions as well.


Exactly—I bet the reasons are often surprising.


I think you're in the wrong thread


What's the current state of compilation times of Rust? How many times slower than C does it compile? Any links to statistics? I know it's hard to compare, but I am interested in developing a gut feeling of what to expect.


> What's the current state of compilation times of Rust?

Still slower than we'd like. Some folks are putting in a heroic amount of effort to improve this significantly, but it's going to be a long time before it really happens. Smaller stuff is helping all the time, but has less dramatic impact.

> How many times slower than C does it compile?

Really impossible to say, because it varies along so many different factors. C will almost always compile faster though, it's true.

> Any links to statistics? I know it's hard to compare, but I am interested in developing a gut feeling of what to expect.

The closest we have is a few bits of basically anecdotal evidence; we have projects demonstrating that rustc got about 3x faster since 1.0, and we have had at least one (in my mind) pretty reasonable C++/Rust comparison that showed them as being roughly equal. But it's just really hard to get good data here.

You should expect something roughly C++-like.


Well, that would be a nice benchmark ;)

I would be very curious to compare speed, size and compilation time of the two versions.


"Talk is cheap, show me the code" - Linus Torvalds

The author applies the same standard reasons for moving from C to Rust for the QEMU project which are valid reasons. However, I would take the author more seriously if they have done a proof of concept first and then later made their case for this.

Otherwise, it is another "rewrite it in rust" post which is yet to materialise into a real project.


> Otherwise, it is another "rewrite it in rust" post which is yet to materialise into a real project.

I'm not sure you can reasonably said that "it is another 'rewrite it in rust' post..." if it's coming from a QEMU maintainer.

Besides, before starting a large engineering project like this, it's worthwhile to write a high-level proposal _then_ write code. Saves you a lot of wasted effort!


Haha, good luck porting QEMU to Rust.

For those who don't know, the original author of QEMU is Fabrice Bellard. This guy has several winning entries in the IOCCC. One of them is a C compiler, that once de-obfuscated, forms the basis of TinyCC.

QEMU C code makes heavy use of macros and gcc extensions. It really pushes C to the limit, it even broke my syntax analysis tools.

Now, if the Rust community wants to write a new emulator in Rust, why not, but starting with QEMU may not be the easiest way.


> Eliminating C programming bugs does not necessarily require switching programming languages. Other approaches to reducing bug rates in software include:

> - Static checkers that scan source code for bugs.

Does QEMU use a sound static analyzer (like IKOS [1])?

[1] https://github.com/NASA-SW-VnV/ikos


No. Coverity scans QEMU regularly but it is not sound.

The challenge with static analyzers is that false positives make people not want to deal with the analyzer reports.

I will check out IKOS, thanks!


One of the cool things about that particular analyzer is that you can choose how accurate it is in exchange for how long it takes.


Could the effort be bootstrapped by using a C to Rust translator, like https://github.com/immunant/c2rust, preserving the existing semantics prior to updating the code to remove the "unsafe" tags.


C11 has safer version of those bad boys(strcpy_s etc) and gcc/clang is more powerful about static checking and runtime checking these days, make existing c code better/safer might be more realistic than rewriting code size of qemu in rust.


Seems a bit silly to rewrite working software in a relatively new language. I like Rust, but it has its own trade-offs. The big one for QEMU being that QEMU developers and users tend to know C already and not be familiar with Rust.


Rewriting the device drivers would be huge. Having been tasked with "Make a QEMU environment safe against advanced attackers" memory safety issues in device drivers were by far the most concerning issue, and the hardest to mitigate.


Rust definitely has that advantage. The downside is most device driver programmers know C, not Rust. So if you want other people to work on your driver you either have to expect them to want to work on it badly enough to learn Rust or to already know Rust.


This blog post agrees with you, and says that re-writing the code in Rust is a bad idea.


I've heard from different sources that QEMU's code is one of the best C code base available in terms of quality and clarity (and smart tricks!). Tough call.


Too bad the C beer-goggles have lasted so long that other great languages didn't take hold. Don't know about that theorem proving stuff in ATS, but with some application of hygiene like "if denom <> 0 then" and compile time prechecks on structures that like every Wirthian language ever has offered, maybe the computational arts would be further along right now.

Nah, too much trouble. I'd rather not check if denom = 0. Back to writing autonomous car code!


How big is a device emulation mentioned in the article? How many of them are necessary?


If people are really serious about writing virtualization in Rust, they should just contribute to firecracker: https://github.com/firecracker-microvm/firecracker


Firecracker is a really nice project, but the scope really isn't the same as QEMU, and they are not mutually exclusive projects


The project goals of Firecracker - lightweight VMs (literally "microvm" in the name) with the bare minimum device emulation - are definitely removed from QEMU at large.

There may be some overlap, but just "contributing to Firecracker" ignores the goals of these projects.


maybe they can recruit the remacs team


Maybe remacs can recruit the remacs team, frankly


is remacs looking for itself these days ?


Start porting some device emulators, otherwise this is just a waste of time.


[flagged]


We've banned this account for breaking the site guidelines and ignoring our requests to stop.

If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future.

https://news.ycombinator.com/newsguidelines.html


> Put up or shut up fool.

Please don't use personal attacks.

> by the people who actually matter to these projects.

According to the author's "About me" section, they are one of the people that matters to the project:

> I am an open source developer and contribute to QEMU and KVM. I work at Red Hat. The postings on this site are my own and don’t necessarily represent Red Hat’s positions, strategies or opinions.


a) The post is written by a QEMU dev

b) It argues against rewriting QEMU in Rust

c) It was the start of a positive discussion on the QEMU dev mailing list


The best way for QEMU (or anything) to keep improving is to have it written using the languages and tools that the actual maintainers are the most comfortable with.

People suggesting rewrites in X/Y/Z are usually not people maintaining these projects. This is evangelism, but it doesn't help with anything.


Article author here. Thanks for mentioning it, I have updated the article to clarify that I'm a QEMU maintainer. The article was intended for the QEMU community but now that it's being discussed more widely it helps to be explicit about my involvement in QEMU.


Be the change you wish to see in the world. Otherwise, posts like that are mostly people asking others to use same toys as they like.


Article author here. I am already working on Rust code for QEMU :).

The goal of the article is to start a wider discussion in the QEMU community and encourage others to do the same. I will post more about the things I have been working on when they are ready.


Happy to hear that, and I support your effort. If you can show people that Rust actually works and is better, it will help advance QEMU. It'll be much more productive discussion, when you have actual data and code to back you up.


Yes. Let's make things less accessible to everyone. People with less resources have had it too good for the last decade or so, and our double digit core, double digit gigabyte memory computers no longer make us feel special, so let's move software to something even bigger and more bloated than anything we've ever seen before.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: