Hacker News new | past | comments | ask | show | jobs | submit login
Mitigating Memory Safety Issues in Open Source Software (googleblog.com)
376 points by littlestymaar 18 days ago | hide | past | favorite | 236 comments



For what it's worth, I reached out to the Open Source Security Foundation (OpenSSF) after their press release on "Securing Critical Open Source Projects" [0], because my open source project is on their "critical list" (Python top #200) [1].

After some clicking around, I was directed to the OpenSSF mailing list [2].

…where my request was left to rot, without any response.

Made me wonder whether this is just a PR stunt / corporate power grab.

[0] https://opensource.googleblog.com/2020/12/finding-critical-o...

[1] https://news.ycombinator.com/item?id=25381397

[2] https://groups.google.com/g/wg-securing-critical-projects


> …where my request was left to rot, without any response.

Hey, I am involved in the OpenSSF and happy to take at least a look. I maintain a popular python OSS project in the security area so we have that in common. Did your email actually go through, I just searched my list archives and there is nothing for gensim. EDIT: see your email now, have msg'ed the WG in slack.

> Made me wonder whether this is just a PR stunt / corporate power grab

I would not say it is, we have folks from OWASP on the technical advisory panel and many other non profit orgs / individual developers. One thing I would keep in mind, is that being a newly founded org, a lot of time has been spent working out and agreeing how we will operate, so the working groups are all still quite fresh.


I couldn't find "Gensim" in the Python top 200 list:

https://www.googleapis.com/download/storage/v1/b/ossf-critic...

In the list of all projects, it is 2909th.

https://www.googleapis.com/download/storage/v1/b/ossf-critic...

You posted in December that it was #119 for Python. That suggests to me that this list might be quite volatile, which calls into question its usefulness.

Regardless of the actual position in the list, your email should have got a response!


Unfortunately, the openssf members haven’t come to consensus on the process for handling funding requests and the process. We’ll get there (hopefully!) but it’s looking like it’s going to take more time. -Sincerely, Google’s OpenSSF governing board rep


I think it is, the maintainers of curl even talked about how C is a secure language and how curl is on almost every system possible which is why it has flaws, not because C is insecure. Debugging your code on almost every hardware out there is not possible which is why it is open source and why they intended you to adapt it not just run it


What is left out of the title is that they do not provide funding to the current authors/maintainers of these open source projects. Instead, they fund an organization that will rewrite these tools.

Correct me if I’m wrong, but that’s my understanding of the rather terse article.

I would rather have the original maintainers funded and still in control of the many pieces that form the basis of the major Linux distributions.


The article says that Google is committing funding to the ISRG (https://abetterinternet.org). It isn't really clear whether that is one-off funding, or the start of an ongoing relationship.

The ISRG seems to have two projects of this type mentioned on its website: an improvement to curl (https://www.abetterinternet.org/post/memory-safe-curl/), in which the ISRG funded the curl author directly; and an improvement to httpd (https://www.abetterinternet.org/post/memory-safe-tls-apache/), in which Google (via the ISRG) are funding an httpd committer.


lets encrypt are under the ISRG


(Executive Director of ISRG here)

> What is left out of the title is that they do not provide funding to the current authors/maintainers of these open source projects. Instead, they fund an organization that will rewrite these tools.

This's not what's happening. Here's what is:

ISRG plans and coordinates investments in moving open source software to memory safe languages. We have a strong preference for working on these plans with maintainers and developers, and then funding them to execute the plan.

Once we have a plan prepared, and, ideally, maintainers/developers on board, then we pitch it for funding. Once someone (e.g. Google) funds the project, then ISRG handles contracts with the project developers/maintainers to get the work done. In some cases maintainers may be on board but want us to find a contractor to actually do the work.

In both of the first two projects, and in most future cases, the money that Google (or another company) provides will largely go to project maintainers / developers.


This sounds great, thanks for clarifying!

Also, thanks for Let’s Encrypt. I didn’t make the connection between the two names. This initiative looks much better to me now.

Is rust the only language that you are considering? Would go make sense as well?


so potentially the result is a complete rewrite not involving the original authors in a language the original authors may not be current with, by a contractor code publisher / producer.

given the potential for security theater and general fud and the potency that appeals to authorities such as sponsors like Google can create in the minds of the public, how has this been evaluated in terms of the possibility for widespread disintermediation of the open source software community at large?


> The ISRG's approach of working directly with maintainers to support rewriting tools and libraries incrementally falls directly in line with our perspective here at Google.

Incrementally – that means in-place. You can't fork and rewrite incrementally since there will be no adoption/users of the fork.

Also they talk about maintainers of the projects. I think that largely means original authors and maintainers, not forks.

I don't see how you come to that conclusion. Could you clarify? Thanks :)


At least for curl the project leadership is closely involved, as evidenced by a related article on Stenbergs blog [1] and the ISRG annoucement [2], which mentions funding Stenberg directly and is actually linked from this anouncement...

Please don't make statements like that without doing a minimal amount of effort to verify.

[1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

[2] https://www.abetterinternet.org/post/memory-safe-curl/


Maybe I got it backwards, but it’s a PR post, and the burden is on the author to make the point clear. I’ve read it 3 times and still can’t find any sentence where they state clearly what the development strategy is.

And from link #2 that you posted, I see they talk about the curl stuff, but also show a bunch of repos with tools rewritten from scratch, e.g. https://github.com/ctz/rustls

So I really cannot draw a conclusion one way or the other.

Normally I’m an optimist, but after the recent chromium misunderstandings, I am going to err on the cynical side.


The example from the ISRG blog [1] directly contradicts the assumption you're making:

> Memory safety vulnerabilities represent one of the biggest threats to Internet security. As such, we at ISRG are interested in finding ways to make the most heavily relied-upon software on the Internet memory safe. Today we’re excited to announce that we’re working with Daniel Stenberg, author of ubiquitous curl software, and WolfSSL, to make critical parts of the curl codebase memory safe.

> ISRG is funding Daniel to work on adding support for Hyper as an HTTP back-end for curl. Hyper is a fast and safe HTTP implementation written in Rust.

I too found the post terse and the lack of quantification of the financial commitment makes me suspicious but I think they deserve the benefit of the doubt for now.

[1] https://www.abetterinternet.org/post/memory-safe-curl/


Yeah, it sounds a lot like Google is going to "hijack" popular open source projects for the sake of "security".

It'll be interesting to watch how this plays out. But I pity the projects where Google's gaze falls upon.


I’m not sure “hijack” is the right word. They are using money to entice projects to rewrite in memory safe languages.


Your missing what the two parent comments are driving at. Google says that's what it is doing, but this money is going to "an organization" that will seemingly get it's money from Google, giving Google control over whatever open source projects they target.


> control over whatever open source projects they target.

but if they rewrite the project in rust, why _shouldn't_ they control that project? If other people choose to switch, then it's not wrong for them to move, and not wrong for to google gain control of those users.

The original project maintainers is a third party to this whole process, and i would argue, is uninvolved tbh. It is up to the users of the project to decide to trust google's rewrite (and their tendencies to abandon projects...), vs the original maintainer's version, balanced against the security aspect.


If you translate an existing project into Rust, it is a derivative work and should retain the original license.

If the rewritten project gets more successful than the original (perhaps due to corporate promotion), you have morally stolen the work of the original authors.

If you write a project from scratch without looking, this of course does not apply. But I doubt that is how Rust rewrites actually happen, the temptation is too great.


> If the rewritten project gets more successful than the original (perhaps due to corporate promotion), you have morally stolen the work of the original authors.

Ownership is by "moral" definition something completely arbitrary and made up.

If you work in open-source you most likely have different views on ownership than others.


Now you only need a lawyer army bigger than Google's to win your case.


> If you translate an existing project into Rust, it is a derivative work and should retain the original license.

By this logic, WINE is a derivative of Windows and should retain their license.


WINE goes to some length to avoid any reverse engineering of windows components and certainly any glimpse of the source code (such as the leaks which have happened over the years: if you have seen any of them then WINE does not want your code). They do this to avoid any accusations of their code being a derivative work. If you re-implement an open-source project on the same terms then it does not qualify as a derivative work. However if you look at the source code and use that to develop your new implementation, then that qualifies.


>> control over whatever open source projects they target.

> but if they rewrite the project in rust, why _shouldn't_ they control that project? If other people choose to switch, then it's not wrong for them to move, and not wrong for to google gain control of those users.

I think the issue is the more critical projects Google controls, the more they can make decisions by fiat. Those decisions will favor Google's interests, and could sacrifice everyone else's interests in some way.

It's sort of like web browsers. My understanding is that Chrome is so dominant that Google can basically dictate HTML standards at this point, or at least effectively push them in directions that are favorable to its ad-based business.


You're scare-quoting "an organization", which suggests you don't know what the ISRG is. You might look them up. They're pretty well-liked.


I was quoting OP.


And? Google can't decide to fund organization that rewrites open source projects, or what? How is that different from forking the projects - apart from Google not taking anything at all?


[flagged]


This is like the archetypical lowbrow dismissal, since the ED of ISRG is literally on this thread saying that their M.O. is to fund project maintainers directly to get this work done. These workers control the means of production.


They're literally funding an organization that will take software to the next level and share it with the world, what more do you want?


A rewrite is not taking it "to the next level". It provides a new version in a new language that does the same things, and if things go according to plan, will have fewer safety issues. There's nothing that binds them to maintain any of these projects. Given Google's history, it's irresponsible to not consider long-term maintenance and other issues.


No, C and Rust definitely don't do the same things. If the project is open source, I don't see how any maintenance concerns are relevant - everybody can judge for themselves.


It's not that smart to mistake the purpose or specification of a program, with its implementation.

Anyway, if everybody can judge for themselves, then I guess it's cool to judge that giving Google an inch over many critical open source projects is a horrible idea.


You speak as if it's impossible for someone outside Google to maintain a project that isn't written in C++ or C...


Sure, nobody says you have to use their software. But don't claim that they don't give back or that it's purposeless.


*to entice projects to rewrite other peoples projects


If they pay for the rewrite, they have control over it.


-


Since they don't do that (it's right there in the announcement, which no one in this thread seems to have bothered to read) that whole point is moot:

> The ISRG's approach of working directly with maintainers to support rewriting tools and libraries incrementally falls directly in line with our perspective here at Google.

And a link to: https://www.abetterinternet.org/post/memory-safe-curl/


Then be satisfied that your project is a good enough idea that someone else is willing to put up money to rewrite it using a safe language.

But if you're not the one doing the rewrite, why would google pay you? And if you would be capable of such a rewrite, may be they would pay you to do so.


Wouldn't feel great, but what in the article gave you the idea they're doing that?


If such changes had been planned in the project all along and the money injection can help speeding this process up that's fine. But I can't see Google security people getting involved being a good thing otherwise. They come across as prioritising security problems over anything else (e.g. "if all you have is a hammer...") and I'd be surprised if this sort of extremism will be healthy for a project in the long run. Memory safety is just one of many aspects to juggle in a software project.


Could be something great, but could end up with some "entity" will rewrite, say, ls or cp and suddenly it will turn out, that ls is still free and open source, but sends all user listed files to that entity (obviously for the sake of the noble purpose of making ls even better and all those stats are anonymous).

And there will be some pro version that does not grab that stats, but you need to purchase subscription.


Most of those project authors refuse to rewrite or adopt memory safe languages in their projects.


No reason no exclude them in the first place.


Sure, but the internet shouldn't be too upset when a fork happens with a version that has feature parity and is memory safe.


Any why shouldn't they?


I don't know, reinventing the wheel with the latest buzzword compliant language sounds like a fun time. Think of all the exciting new bugs you get to fix!


They specifically mention rust based backends for curl where the author of curl was funded to integrate as a starting point. I don't see any mention of funding being exclusively for organisations other than the author(s)?


Agreed 100%.

Sadly everyone expressing concerns like this will be dismissed and everyone is going to once again hail google for being community friendly and supportive of FOSS.


Replying to myself as I cannot edit anymore.

As others pointed out, it’s not clear who will do the rewrites. Based on the curl example, it might be the maintainers. It would have been great if they could state that concretely.

I’m also wondering why there’s a middle man instead of funding the projects directly, but that’s another story.


I'm the head of ISRG, the "middle man" entity you're referring to.

We have a memory safety initiative in which we plan and fund projects to move popular open source software to memory safe languages. For each plan we make we seek a funder - in a couple of cases now that has been Google, but it may be others in the future.

We are in the "middle" of the financing in the sense that funding passes through us, but the memory safety initiative is coordinated primarily by us. We select projects and work with maintainers to make plans. Part of that coordination we perform is matching up projects with funders like Google, and then managing the contracts and overseeing the work to completion.


Thank you for your reply.

If I understand correctly, this is an initiative that might have multiple sponsors, so it makes sense to have a central entity organizing it, and following up with all the projects involved. This makes perfect sense to me now.

Also, I was not aware that your org is behind Let’s Encrypt. This puts things into a very positive light.

It might make sense to post a sibling blog post/announcement to explain what this project is about and how it would work. I feel that Google’s post left a lot of questions to be answered.

If you don’t mind me asking, is rust the only language that you have in mind? Would go work as well?


Can someone please fix the title to the original in the blog ('Mitigating Memory Safety Issues in Open Source Software'), because the current title is not at all representative of what the blog is actually saying.


Also "for rewriting popular systems in Rust" is just false. The blog is about improving memory safety and gives two pieces of software rewritten in Rust as an example.

There's nothing about this effort being focused on rewriting things in Rust.


You're going to see a lot of Rust, because by and large the programs ISRG and Google are talking about remain in memory-unsafe languages because they have constituencies where dynamic-runtime memory-safe languages are problematic. That's why security people are so excited about Rust: it's hard to come up with modern settings where Rust is unworkable, unlike Go (a language I prefer to Rust), which, for instance, isn't a good fit for the Linux kernel or for the guts of Chromium.


There's a ton of platforms that Rust doesn't not target, or targets but without many guarantees of support. The can be problematic as its closest competitors (C++, but especially C) generally have much more widespread support.


That's a problem, but it's one that's more likely to be worked around (most likely: by maintaining unsafe forks for those platforms) than "creating entire new communities of support for another language" or "redesigning whole projects to make GC tenable".

I think you might see some full-system rewrites for serverside components where Go will work out. But I think the projects we're talking about, like I said, have ended up in this position mostly because they can't easily be written in Go, or they already would have been (if only be a competitor).


Oh, I'm not suggesting that they will use Go–they will use C and I'm just responding to you saying that it is difficult to find situations where Rust is unworkable.


The premise of this story is that ISRG is going to secure funding to essentially bribe people (my words, and I think it's a very good project) to not keep critical software in C anymore.

My guess is what will happen in the cases where Rust is unworkable is that there will be less-secure forks for the obscure platforms.


Rust is not the only option.

F-Secure has been using TamaGo for bare metal on their secure USB keys.

MeadowLabs is using .NET with a tiny C++ hardware glue for IoT.

Aicas and PTC are still in business delivering real time bare metal Java stacks.

Astrobe is still in business selling Oberon compilers for embedded development.

MikroElektronika is still in business selling Pascal and Basic compilers for embedded development.

And then there are those 5 surviving Ada compiler vendors, one of them open source.

Finally, if there is no way around having to use C, there is no reason not to use C++ instead other that anti-C++ hater religion.

Unless we are talking about UNIX clones, thanks to them C will still be around for several decades to come anyway.


Yeah none of that is going to happen. People aren't going to rewrite major open source projects in Oberon, or specially hosted .NET, or custom JVMs, or Ada. ISRG isn't going to arrange funding for things to be rewritten in another memory-unsafe language, so C++ is right out. Stuff is going to get rewritten in Rust. Sorry!

That sounds snarky, I know. I promise, I'm not being snarky, and, again: it ain't my first choice (it'll be fine though).


In what concerns Microsoft, the official security guidelines are .NET, Rust and C++ (with Core Guidelines).

As for the other platforms, and what Google is paying for, lets see where the bribe drives people into, after all from your point of view Google should rewrite gVisor and everything k8s as well.


No, I don't see how you get there from what I said.


> Stuff is going to get rewritten in Rust.

Yes it definitely will, but not only, specially among groups where not using a GC is more of a religious issue than an actual technical impediment.


I'm not making a value judgement. I prefer GC'd languages. I'm saying: if they're not GC'd now, when there are multiple excellent choices for managed-runtime languages, it ain't gonna be GC'd. And obviously, it's not going to be C++.


Ok there I agree with you, however in some domains the choice is made by who is calling the shots and not due the technical merits of some sorts.

Windows is a good example of this politics game actually.

Joe Duffy mentioned on his Rustconf keynote that even with the system running in production, the Windows team couldn't be argued against.

A system that even powered parts of Asian Bing infrastructure.

Likewise they clearly sabotaged the Longhorn efforts, to replace them with COM, and yet UWP, mostly written in C++, happens to usually run slower than traditional .NET applications on Win32, or any other compiled GC'd language on Windows for that matter.


Somehow I think ISRG is not going to try to secure funding to rewrite core Windows components.


Are you sure?

> That’s why we’re expanding our collaboration with the Internet Security Research Group to support the reimplementation of critical open-source software in memory-safe languages.

Though the rest of the article doesn't go into detail about this effort beyond http/TLS libs for curl.


Rust is going to be used often, but we are open to other choices when they make sense.

Since most unsafe code is C or C++, Rust usually makes the most sense for rewriting one component at a time. It integrates with C and C++ very cleanly because it has good FFI and no runtime.

If we were rewriting more projects from scratch and integration with existing C and C++ code was not important, we might use other languages more often. I don't expect we'll attempt to rewrite many pieces of software from scratch.

Source: I run the memory safety initiative at ISRG.


The quote you provide above doesn't specify that these rewrites are to be in Rust and I don't think you've contradicted the parent in any way. The examples have been in Rust but the foundation doesn't seem to insist on rewrites being done in a specific language. As an aside, the foundation doesn't seem to be much more than a collection of papers at this point.


The "memory-safe languages" may mean Go, Java and a bunch of other less popular languages.


In the end of the day only Rust will be chosen because we are talking mostly about system infrastructure.


HN is like 15% pro-Rust propaganda these days.


Have you considered the possibility that people might just like it?


I'm a native english speaker, and I've never seen the word "fundings" in my life.

Edit: the submission title is fixed.


Might be reinventing the wheel ;D


While rust certainly does a lot of things right, it is still a moving target with a relatively high velocity.

There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.

This also means that very long-lived platforms will probably be forced to upgrade that library every half year or so in line with the development of rust.

I'd say such an initiative needs a long-term supported version of rust, similar to C99 or so.

And this does not even account for cargo. I know you don't have to use cargo, but is that practical?

How are we going to maintain stable software for, say, ten years or more? Do we have anyone committed to providing a stable rust environment for these cases? Especially considering that rust has no ABI for dynamic libraries.

Oh and by the way: We need a way to bootstrap rust as well. The current distribution downloads binaries of the last one to build itself. And it ships with a vendored version of llvm.

So in conclusion: I really like the rust language. But I am concerned that it is still way too immature and to "hip" for long-running, stable projects.


> There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.

there comes a point where we can't let dependencies on Amiga or DEC Alpha (companies/platforms that haven't existed in literal decades) hold the rest of us back. If someone wants to port rust/llvm to those platforms more power to them, but the rest of us don't need to suffer. The only platform that actually matters that was negatively effected by that change is musl/alpine because there aren't musl wheels available and you have to build the rust part from source.


> dependencies on Amiga or DEC Alpha

What about embedded platforms, which tend to remain relevant (i.e. present in large numbers out in the real world) much longer than desktop or server platforms? Don't they matter? Don't they need updates, including for security, as much as any other?


Is it not a bit of a paradox that said embedded platforms matter enormously right up to, but not including the point of having toolchains ported to them and maintained? And yes while gcc can be built to produce binaries for an alpha it's not producing any bugs right?


The interests of people writing new language toolchains sadly does not seem to extend to supporting older platforms. This doesn't mean that they are not important, it just means that that they are not important to the people writing the new language toolchains.


It's not a matter of interest; if an embedded platform matters they should be willing to pay someone to port Rust.


…or they can just continue using C.


And not be surprised when open source maintainers don't go out of their way to continue supporting their poorly supported platform...


There are a few niche embedded targets that Rust might not have a toolchain for, but the vast majority are actually well covered. MIPS, PowerPC, the relevant flavours of ARM, MSP430, RISC-V.

> This also means that very long-lived platforms will probably be forced to upgrade that library every half year or so in line with the development of rust.

As a security library... you really ought to be updating it every half year of so.

This has nothing to do with rust though. Rust maintains full backwards compatibility, so even if you upgrade the compiler (which is unlikely on long lived platforms) and not the library nothing breaks.

Nor is it necessarily the case that upgrading the library will require updating rust (though this isn't what you appear to be complaining about). Not only is it a build time dependency (that you don't even need to install if you are on common platforms which ave prebuilt binaries), but they are explicitly testing against a particular fixed old version of rust as well as recent ones: https://github.com/pyca/cryptography/blob/main/.github/workf...

> I'd say such an initiative needs a long-term supported version of rust, similar to C99 or so.

This describes every version of rust since 1.0. Rust is fully backwards compatible, unlike new C standards.

> Do we have anyone committed to providing a stable rust environment for these cases?

Yes, the entire rust team...

> Especially considering that rust has no ABI for dynamic libraries.

This is... rather irrelevant? Technically it does have support for the C abi for dynamic libraries though.


I think you should reconsider the actual problem, if you do not understand it.

The fact that rustc is backwards compatible is great, sure. But it does not solve the problem of distributors.

Say you have libfoo-1.0 that uses rust and cargo 1.2 . A distributor that now releases their linux version and wants to ship libfoo-1.0 must consider the requirements for supporting it. This support usually outlives the upstream support.

If libfoo-1.0 uses C, for instance, it is highly likely that a distributor can backport any security-relevant changes from libfoo-1.1, 1.2, and later into 1.0.

In the case of rust there are two uncertainties creeping in: One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.

So our distributor now must consider if changes in libfoo-1.2 pulled in changes via cargo that cannot be easily backported because they are feature releases or have a dependency on a newer version of rust.

The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust. It is great that they want to be backwards compatible but now our distributor must consider upgrading a substantial piece of software during the support period (and testing the backwards compatibility, of course) just to backport a security fix.

Please don't get me wrong, I really like rust and I think these problems can be solved. But I also like stable software and all these languages with their package repositories seem to simply ignore everything that's older than one or two years. If rust wants to become a new C, which I would very much like, it needs to demonstrate a certain stability.


I think you are missing the fact that the solutions are already in place for all of this.

> One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.

crates.io is designed so that dependencies put up on it remain available forever, and that the only other cargo related dependencies also have to be hosted on crates.io - solving the availability problem.

Of course if you're trying to backport a security fix to an old version of a transitive dependency that (like the non-transitive dependency) is no longer supported, you do have to make that backport. Once you make that backport it's easy to update the version of the transitive dependency with a [patch] section in the top level Cargo.toml.

> The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust.

This is already mitigated in multiple ways. As discussed before, people do make sure to support old versions of compilers, you don't need to have a compiler on your target platform anyways. But moreover, the amount new language features make backporting difficult is dwarfed by the amount that internal library changes do.


What I don't really like in cargo is its' overcentralized and author-centric approach. What if some of your dependencies relies on the crate, which author doesn't support anymore and doesn't even accept patches? You will basically have to maintain a fork either with a different name (because you can't push it to crates.io) or have a patched version which you also can't push and have to manually update every Cargo.toml of every dependency in your project to use your fork. And what if you have not only one Rust project in your complex system? You will have to update or vendor everything. This is a real maintenance hell.

While in C/C++ I can just build a fork as dynamic library and put it to the system image or to the own package manager repo.

I mean for sure Cargo is great for small hobby projects, but for the big complicated enterprise projects it's not really suitable. Even Google doesn't use Cargo in Fuchsia and vendors everything.


I'm not entirely thrilled with the cargo solutions here, but when I compare them to the options in other languages I think they're adequate.

If you're making a small change, you use a [patch] section in the Cargo.toml for your workspace if you're using one, or the Cargo.toml for your binary/.so/.a if you're not, and it will cause your entire dependency tree to use your forked library. You don't need to fork the entire ecosystem as you suggest.

The [patch] works at the "workspace" level (and you generally should stick your entire complex system under one workspace), so if you use that feature properly for your "complex system" (keeping everything under one workspace), it is a one line change to replace the dependency for your entire system. If you use multiple, well, one per workspace that produces a binary/.so/.a.

Also, for big enterprise projects, I would assume you would use a private registry in the first place, which is apparently well supported (though I haven't used it). That's how the enterprise places I work at have dealt with python dependencies for instance (I haven't used rust in a enterprise).

I can sort of understand people trying to maintain LTS linux distros complaining that for their purposes this is a bit of a pain compared to C style shared libraries (because they need to modify a lot of rust workspaces)... but to be honest I think that's a question of their package management tooling not rust's. Rust provided the tools they need to make their custom tooling work. As a user the number of times shared libraries have caused me pain makes me think that this static linking is the superior method of packaging.


When I started programming, static linking was the only way of packaging software in mainstream computing, if it was so superior we wouldn't have moved away from it.

Also although we are talking about open source here, in many OSes, and enterprises, distribution of binary libraries is a very important subject, which cargo doesn't allow for.


> Any change in libfoo-1.2 might require a newer version of rust.

In theory perhaps. But in theory this could happen with C or C++ too. In practice, security patches try to be minimal.

Sure, if you're several versions behind you might have more work to do. But you knew you were gonna own that problem when you decided to lag behind.


We have a way to bootstrap Rust: https://github.com/thepowersgang/mrustc


Was this an encryption library by any chance? Generating pgp keys to yubikey with an offline master involved having a system with rust toolchain in order to build a Python lib I required for a Yubikey utility.

I could have got around it, but it took me by surprise that a Python dep required it. Not being negative or criticising the fact, just first time I had observed it.


It was, "cryptography".


>There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.

Progress should not be held back for the rest of us because a few will get left behind. If you are running something so old that LLVM doesn't support it, upgrade. It's 2021.


I guarantee you that there is a device in your house that Rust does not target.


LLVM does not support every proprietary embedded device. And even if it did, rust support does not come for free. (One must add support for the target ABI and other stuff)


At some point, they had to switch from Python 2 to Python 3 no?


Executive Director of ISRG here. To provides some more clarity on how this partnership works...

ISRG plans and coordinates investments in moving open source software to memory safe languages. We have a strong preference for working on these plans with maintainers and developers, and then funding them to execute the plan.

Once we have a plan prepared, and, ideally, maintainers/developers on board, then we pitch it for funding. Once someone (e.g. Google) funds the project, then ISRG handles contracts with the project developers/maintainers to get the work done. In some cases maintainers may be on board but want us to find a contractor to actually do the work.

In both of the first two projects, and in most future cases, the money that Google (or another company) provides will largely go to project maintainers / developers.


Have you considered the downsides of this shift? For example c++, with these initiatives I'm thoroughly expecting the compile times to bloat up along with increasingly bloated code in the effort to make the code safer.


It's buried in there, but it doesn't specifically say Rust:

> That’s why we’re expanding our collaboration with the Internet Security Research Group to support the reimplementation of critical open-source software in memory-safe languages.


> The new Rust-based HTTP and TLS backends for curl and now this new TLS library for Apache httpd are an important starting point in this overall effort.


This item had a different title specifically mentioning rust which had been changed to remove it, before you made your comment


Nowhere does it call out Rust by name.

Go / Zig / Pony / Haskell are all valid contenders.


Go, Pony and Haskell all have complex runtimes and a GC, which makes them unsuitable for embedding into other applications or exposing them as widely usable libraries.

Rust, Zig, and to a certain extent D or Nim without GC, can easily expose a C API and have a minimal runtime.

But the others do not provide the same guarantees as (safe) Rust with the borrow checker. All very cool languages in their own right, but memory safety in a no-GC environment is not one of their defining attributes.

Note: a dependency on any of those compilers is still far from trivial for many projects.


D now has a prototype borrow checker implemented.


That's very cool to hear!

As the ceator of D, how do you see the place of borrow checking in the ecosystem long term?

Would you imagine most D code to be written against the borrow checker in the future, with transparent convenience for GC types?

Or would it remain a somewhat niche feature for domains that can benefit?


The borrow checker in D can be used incrementally, like functional programming, on a function-by-function basis. It can be used where it makes sense to use it. (If only the GC is being used, the borrow checker isn't needed.) Being incremental, it won't require redesigning the program to make use of it.

I don't know how pervasive its use will become, but I expect its use will steadily increase.


One thing that I think we'll have to deal with at some point in the next two years or so is that although using the borrow checker will be incremental on (say) functions, the actual design of said borrow checker needs to be more integrated than incremental i.e. currently we can catch lifetime related bugs but you can't properly express (say) invalidating a pointer into a container by calling popFront.


i just write all my projects in Haskell on any platform so the RTS is a given (or the RTS isn't present at execution time since the executable is generated from a Haskell eDSL)

it's truly superior to any other programming language (5+ years of pro experience talking), so why waste my time with the manual labor of imperative programming?


I write 90% of my stuff in Haskell. When I need speed, I used to use C, now I use Rust.

> why waste my time with the manual labor of imperative programming

Someone has to. https://github.com/ghc/ghc/tree/master/rts


I'd say about the same. C has its place, and the GHC RTS is a great example.


Nim is not using GC anymore [1], is memory safe and support formal methods [2]

It's also introducing isolation [3] to provide safe multithreading similar to Rust's sendable.

[1] https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc...

[2] https://nim-lang.org/docs/drnim.html

[3] https://github.com/nim-lang/RFCs/issues/244


From [1]:

> ORC is Nim’s all-new cycle collector based on ARC. It can be considered a full-blown GC since it includes a local tracing phase (contrary to most other tracing GCs which do global tracing). ORC is what you should use when working with Nim’s async because it contains cycles that need to be dealt with.

Sooooo unless you

> disable the GC and do manual memory management, but then you lose access to most of the stdlib

you still have to run a memory-managing runtime


No. It's an inlined reference counter, not a runtime with dedicated thread and so on. Most people use "GC" to refer to the latter.


like swift you say


IIRC Zig isn't memory safe though, is it? If not it seems like a bad choice if the explicit purpose is to mitigate memory safety issues...


I think this is the issue tracking memory safety in Zig: https://github.com/ziglang/zig/issues/2301


Has any language successfully added memory safety as a "feature" after the initial release? To me memory safety seems like a fundamental aspect of a language that is very hard if not impossible to change afterwards, without so many breaking changes that you essentially in the end have created a new programming language.


Depends what you consider an "initial release", is the point where it is available to the public or the 1.0 release?


Let's say 1.0. Languages can change significantly between the first publicly released build and the first stable release.


I suppose D would also fall into this category.

Although I find it interesting that Common Lisp gets left off these lists. It's memory-safe (with GC of course), has great integration with existing C libraries via CFFI, has a formal spec, has a solid library ecosystem, supports a "mostly-functional-but-imperative-if-you-need-it" programming style, has optional/gradual compile-time type checking, and at least one open/libre implementation generates fast native code (SBCL).

Might as well throw OCaml in the mix too, since it has all of the above minus the formal spec.


Oh yes, SBCL would also apply, I guess!


Edit because I can't edit my post anymore: Kotlin.


Pony isn't that well known, here's their home page, for the curious. [0]

I remember a recent HN comment said that if you create a website for your new programming language, you should always ensure there's a meaningful example right there on the landing page. A pity that Pony's page fails to do this. Here's one of their example programs. [1]

[0] https://www.ponylang.io/

[1] https://github.com/ponylang/ponyc/blob/main/examples/timers/...


Also, to my recollection Pony is meant to be somewhat specialized for high-concurrency server applications, not for general-purpose tools/libraries.

It seems like it's best grouped together with Go, Erlang, Elixir, and Crystal.


> The new Rust-based HTTP and TLS backends for curl and now this new TLS library for Apache httpd are an important starting point in this overall effort.


Or Ada or Wuffs

But I have always used Pascal for that. It has memory safe strings and arrays with refcounting, so long as one avoids using other features like pointers, classes, inline assembly, it is perfect.


Wuffs? Never heard of that one before.

Edit: here's the link, interesting idea. https://github.com/google/wuffs


And it is supposed to be safer than a language that is only memory-safe

The memory-safe languages can still panic or have integer overflows, Wuffs should prevent even that


This.


Not really, no:

* Languages with significant runtimes are going to be rejected by project maintainers; nobody is going to rube-goldberg a GC into an existing project just to replace a risky component when the option not to do that exists.

* Languages with insignificant adoption are going to be rejected by project maintainers because why take a flyer on an unknown quantity (not just technically but in terms of where the community will be 5 years from now) on a project you've been maintaining for decades? Say what you will about the Rust community and its long-term financial viability but the industry is pretty much already committed to it.

I'm fine with Rust but, like, not it's biggest fan (strongly prefer Go) but Rust pretty clearly hits the sweet spot being targeted here. I think you can assume safely that most of this work will be done in Rust.


F-Secure has decided otherwise for bare metal secure firmware.


semi-conspiracy theory: the main goal of this rewriting is to have less and less infrastructure depending on GPL and AGPL-licensed code


If all Google wanted was GPL- and AGPL-free code, they're quite capable of assigning their own engineers / contractors to work on it rather than the more tortuous route of going via a 3rd party


Their accountants probably prefer this route though. They don't get a competitive advantage by have a proprietary license, so open source is a good choice, but the GPL has issues they want to avoid. By going this route they can use tax breaks that are not available for doing the work in house. Sure it is more complex, but the results matter, and the result they are hoping for is someone else does work that they don't pay for.


Also it's probably cheaper to fund an OS developer for a year than paying for lunch alone for their own engineers.


What kinds of tax breaks are available for the case of using consultants instead of employees?


It isn't consultants it is 501 charities doing the work. The charity hires the consultants.

There are a lot of tricks needed to make the above work, and details that you need to consult a lawyer and accountant for.


> they're quite capable of assigning their own engineers

but if they develop it for internal usage only, then they don't get the added benefit of destroying the existing copylefted version!


Apache and Curl are not GPL or AGPL to start with.


Of course any such endeavor must have a patina of plausible deniability. That's evildoing 101 :)


Sure, but what if the person/team willing to rewrite wants to do so under GPL? Or what happens next when the next project to be rewritten is GPL? I think its fair to keep the original license, but if re-licensing is allowed it should be permissive.


They removed licensing from their FAQ, but I was able to find this issue: https://github.com/ossf/tac/issues/26

I'm also curious if they will accept GPL or not


It will take quite a lot more effort to rewrite something so that the (A)GPL doesn't stick. To do that you would need to rebuild it from its documentation and/or a clean-room reverse engineering, as opposed to a port of the source code.


My semi-conspiracy theory: "safe" languages also make it harder for things like jailbreaks and other defeats of user-hostile "protections", allowing Big Tech to maintain control, so making them popular is very much in their interest. It's like how every utopian vision for society turns into a dystopian one.

"Insecurity is freedom."


What if any requirements are there on the licensing of these reinvented wheels? Specifically would GPL or AGPL be acceptable?


The more I look at Rust the more I realize it’s going to become the next standard language.


You mean standard for systems programming?

I don't doubt that it will be a popular choice in that area, but even there it has competition from new languages.

And above all, I think it might take longer than a century to phase out all the C/C++ legacy code!


It'll be interesting to see how it plays out.

I could absolutely see C++ devs jumping ship to Rust. I think const generics just stabilized or is about to. That was one of the first shortcomings I noticed when I learned Rust. I think custom allocators is pretty much the next big piece that would give C++ devs pause today.

But I don't see Rust picking up a lot of C devs. Rust is significantly more complex than C. You certainly don't have to use traits and Futures, etc, but you also don't have to use a bunch of features of C++ either and C devs still don't jump to C++.

Zig certainly gets a lot of buzz here, but I haven't personally looked into it. It does certainly sound like it's designed to appeal to the C crowd.


I know I'm only one person, but I'm a C programmer at my day job and I do all my hobby projects in Rust.

For me cargo, rust-analyzer, and lifetimes/borrow checker just makes programming so much more pleasant.

edit: I dont go overboard on the more advanced features in the rust language, and probably write more C idiomatic code sometimes, but I still prefer using rust over C


For sure. I didn't mean to suggest that nobody from C land would enjoy Rust. Just that I think the trend will be that more C++ people move to Rust than C people.

But my point of view is quite biased. I am still baffled that people would start new projects in C instead of just using a strict subset of C++, at least: classes, only trivial templates for generics, references, maybe smart pointers depending on what you're doing, but no STL, no exceptions, no inheritance, etc.


The security information about C and curl is a bit outdated in the post, and recent research shows Rust does not solve the memory safety issue.

The "recent study" quoted in the article was published at the beginning of 2019, using older data.

Current vulnerability data shows that curl has very much limited the risk of memory safety issues. How many reported security vulnerabilities in the last two releases of curl? Zero so far. You have to go back 9 months before you find one expired-pointer derefence issue resulting in potential data being sent to the wrong connection in rare circumstances and configurations. Which is a logic error that could happen in Rust too.

To quote from a Oct 2020 study on Rust safety - "Our study results reveal that while Rust successfully limits the risks of memory-safety issues in the realm of unsafe code, it also introduces some side effects. In particular, most of the use-after-free and double-free bugs in our data set are related to the automatic drop scheme associated with the ownership-based memory management model." -- "Memory-Safety Challenge Considered Solved? An In-Depth Study with All Rust CVEs"

They study 60 Rust memory safety vulnerabilities.

As you can see not only does Rust not solve the memory safety problem, it has other issues. Additionally the old research quoted misleads people about the current status of reported memory safety issues in curl.


Rust does not solve the memory safety issue. It does mitigate it, and the post is about mitigation. That study finds that Rust "successfully limits memory-safety risks to the realm of unsafe code".

It also finds that Rust has novel patterns of unsafety in unsafe code. That's important! But it's fully compatible with the claim that Rust is much safer than C overall.

I don't think it ever says that the sum of safety in safe code and novel unsafety in unsafe code adds up to as much unsafety as C. The paper's overarching claims aren't quantitative.

I think you're overstating the claims of both the blog post and the study.


The recent track record of curl shows it has zero reported memory safety issues recently. Reading the article and the old linked research you'd be mislead.

It also states that Rust completely prevents them - it does not. The article talks about mitigation, but also says: "would have been completely prevented by using a memory-safe language". The "completely prevented" claim in the article is the one not supported by current research. If you only read this article, you'd be mislead about memory safety in Rust.


> The recent track record of curl shows it has zero reported memory safety issues recently.

Only if you look very recently. Earlier you said:

> You have to go back 9 months before you find one expired-pointer derefence issue resulting in potential data being sent to the wrong connection in rare circumstances and configurations. Which is a logic error that could happen in Rust too.

That bug is 6 months old and could not happen in safe Rust because references (pointers) cannot outlive their referents.

> It also states that Rust completely prevents them - it does not. The article talks about mitigation, but also says: "would have been completely prevented by using a memory-safe language".

It is literally true that they "would have been completely prevented by using a memory-safe language." The complication is that Rust is only memory-safe if you don't use unsafe.

rustls (the new curl component in question) uses no unsafe itself. I find some unsafe in its dependencies, but most of it seems to be for FFI, which is inherently unsafe. I'm also not sure that those should count—do OpenSSL vulnerabilities count as libcurl vulnerabilities?

> The "completely prevented" claim in the article is the one not supported by current research.

I will grant you that it's slightly misleading because it's possible to write unsafe Rust code. But that's not news, "current research" has nothing to do with it.


No memory safe language in existence can meet your standards, since they're all written on a bedrock of unsafe code.


Safe Rust solves the problem of memory safety (assuming that any unsafe code is correct). Unsafe Rust continues to be potentially memory unsafe. That is equivalent to other memory safe languages like Java (which is memory safe under the assumption that the JVM has no memory safety bugs and any JNI code is correct), and a very large improvement over C and C++.


Would be nice to see an approved list of "memory safe programming languages" as RTFA just mentions a Rust example.

I'm assuming Go will also be an option, what are the others options for core OSS system tools?


I wonder what implications this has for C/C++ in big tech backends. Startups, from what I've seen, have already moved away.

No doubt, it will still be around in low latency or embedded. However, this is a significant cut of their market share, especially if Google, Microsoft, and Amazon retires its usage in its systems.


A big chunk of the C++ development is done in game studios. I doubt they would be willing to move away from it just for memory safety's sake.


Buggy AAA releases are currently the norm, but that takes a reputational and financial toll and doesn't make for happy developers either. Cyberpunk 2077 is the latest example.


But aren't most user-visible bugs in games caused by logic errors, not memory errors?


Is there a reason why some developers prefer Rust over C++? I often hear that it’s because of Cargo, but in my experience, universal build systems and package managers end up not really solving the problem they are designed to solve, and are a huge attack vector. Anyone who’s tried to deploy their own complex Maven project knows that it’s a true nightmare.

C++, on the other hand, has a variety of choices for build systems. CMake is prevalent, you can use simple makefiles just like C, or you could use countless other custom solutions. I’ve found that languages which support a singular build system are really easy to build simple demo apps, but once you need to build something that the default build system doesn’t support, you’re in big trouble.


I like cargo because I don't like having to even think about build systems.

I often renounce to contribute to C/C++ projects because I know I will spend more time figuring out how to install all of their dependencies than making my two-line fix.

On the contrary, in the rust ecosystem, I know I can just make my two-line fix, run cargo test, and if there is no red line in the output, I'm done.

> I often hear that it’s because of Cargo

I don't think the main reason is cargo. The main reason is that you don't have to worry that the very simple pointer dereference you just wrote may allow anyone on the internet to take full control of the machine of anyone that runs your software.


C++'s build systems are miserable. Almost any other language is preferable (yes, even a complex maven project). CMake and co are just as susceptable to issues when you want to do something off the beaten path, and have a few landmines on the beaten path just to spice it up. Cargo is a breath of fresh air in comparison (and as package managers go it's very good: they took some care to learn from the mistakes of others from the get-go). And rust isn't wedded to cargo: you can build your code using make if you wish. In fact rustc doesn't use cargo.


Cargo is the least of reasons I prefer Rust. Rust APIs are much more ergonomic than C++'s, its type system is better (built-in ADTs and type classes, no null), it's obviously safer by default, it's expression-based which is just sooo ergonomic, its move semantics are enforced by the compiler, and it just doesn't have all the sharp edges of C++ (returning dangling references, implicit constructors, 57 different kinds of lvalue, rvalue, xvalue, lrxhvalue, whatever), etc.


Variety of choices for C++ end up causing fragmentation, and all choices are flawed in some way. If you choose Cmake, someone will tell you how much it sucks, and you should have used Meson. You choose Meson, people will complain that it needs Python, or Bazel needs Java, or why bother with any of them if Makefile is good enough, and so on in circles forever.

OTOH Cargo is the only option for Rust projects. There's nothing to bikeshed about. Everyone has it and knows how to use it.

Apart from that the project is about memory safety. Companies like Google, Microsoft, and Mozilla already have huge C++ codebases, top engineers with lots of C++ experience, spent tons of effort on making it safe, and still weren't happy with the results.


>Anyone who’s tried to deploy their own complex Maven project knows that it’s a true nightmare.

Maven != Cargo

> C++, on the other hand, has a variety of choices for build systems.

None of which even remotely approach the ease of use and maintainability of the Cargo build system.

> once you need to build something that the default build system doesn’t support, you’re in big trouble.

Cargo can build arbitrary dependencies using the build.rs build mechanism.


> Maven != Cargo

Indeeed, Maven supports binary libraries.


At this point if you stick to a limited subset of C++ and don't go too deep you can get a perfectly safe language which is powerful and performant.

If you want a safer language with in my opinion a far better type system than anything Rust could possibly achieve at this point you can use Ada.

But why not those two options? Old, smelly and most importantly, difficult to deal with dependencies. You have to actually think about how your API will work in your library because if your library ever ends up used anywhere people are going to be running different versions and you don't want to ruin their day by having them change their code every time they update their machine.

In comes cargo (and friends, lots of other languages have picked up the same idea) where everything is managed by the programming language's choice of build system and API stability and dealing with dependency hell no longer exists when you statically link everything.

I think people don't care or want to care about why we got to where we are (distributions with dependencies and API stability and an aversion to micro-modules). A lot of people just want things to work when you run one command no matter how much complexity gets hidden under the carpet, how obscure the issues end up being and how windows-esque the debugging process ends up being when things do go wrong.

All this pushed by lots of big companies who don't care about the origins of the open source model and how distributions do things and instead only care about lining their own pockets.

Mark my words, distributions will disappear in a few year's time, people will cheer this on as some great technological advancement, things like snap will become the de-facto standard and debugging a complex problem will just never happen again. The internet will become filled with forum posts of how people "fixed" a problem by rebooting their machine or deleting a random file or re-installing.

Anyway, maybe I'm just a crazy person thinking up crazy scenarios and being irrationally scared of "progress".


Is Ada memory safe in the presence of malloc/free?


Here is a good summary of how Ada deals with safety in the context of memory management: https://news.ycombinator.com/item?id=14216128

It's not rust but at the same time I think rust has sacrificed general safety in favor of memory safety which has produced a tendency towards a false sense of security.

After all, it seems rust still has memory safety issues despite the borrow checker: https://arxiv.org/pdf/2003.03296.pdf


Rust is a nice language for greenfield system software project. But everyone in the software industry knows that you can't just put a project on hold and rewrite completely in new language. I can't see anything good coming from this initiative.


Which is why the word 'incremental' is key here. They didn't rewrite the whole of cURL - they rewrote a key component.

Yes, ideally the whole program would be rewritten but security is not about absolutes. For a manageable amount of effort they have reduced the attack surface significantly.


Rust is designed to integrate into legacy codebases so security critical code can be rewritten in small steps. See the URL parser I Firefox. Moreover, the initiative this post is talking about uses rust to rewrite small parts of a C codebase.

Honestly, I'm getting more and more disappointed by the quality of discussion here on HN.


Thanks for the kind words :)

There is no such thing as security critical code. If it is out there - it can be expoloited. There are a lot of stories with minor exploit chaining leading to huge compromise.


Some solutions aren't that well publicized. Here is an example of an open source png encoder/decoder written in C (mostly) being auto-translated to a memory-safe subset of C++:

https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...


Rust is a nice language for gradual rewrites too, especially when the original language was C. Some examples are Mercurial, librsvg, and Mozilla Firefox.


Why not contribute to MesaLock Linux, which is already an entire memory-safe Linux distribution instead?

Or encourage other memory safe-languages as well?


Maybe I am missing something: public-facing systems generally don't build on a shell-like environment and I don't think memory-safety is a big problem anymore in 20 years-old core-utils people use individually on their CLI.

So what's the point of reimplementing one-off-tooling with under the same name (like sh?!) in Rust, except adding confusion and new bugs? And in a container, you already can run only your rust/Go-program, which effectively means that the main memory-unsafe attack-surface is actually the kernel (which they don't swap out either).


Someone probably should look at the kernel, too.


Most memory safety issues are buffer overflows, which is easily solved in D by using pointer/length pairs as a builtin type.


From the title my first thought was "RUSTLANG!". Then I saw it was a google article and thought "well, that's not going to be rust-lang at all". I guess the answer was somewhere in between.

It would be nice if there was more meat on the article though, like what types of things to do, or how each memory error happens, etc.


hm


and abandon it right after they become technically uninteresting. seriously aside from not supporting current GPL licence on these open source projects I dont see a reason to not support them.


Do they also have a study what the cost is of reimplementing an existing project in a new project regarding feature parity existing bugs people might depend on?


Rewrite vs Transpile

I see references to "fixing" C and rewriting. An alternative would be to transpile a memory safe, easy to program language such as python3 and generate code in a memory safe language.

Since I don't know which one of these is the best, I have implemented basic (as in fibonacci and a simple class) in several languages.

I hope more effort is spent on writing code in small statically typed languages and generating memory safe code.

https://github.com/adsharma/py2many


Would not be cheaper to fix those unsafe languages, such as C, and offer safe mechanisms that could be used upon need?

Why spend thousands, if not millions of dollars, on rewriting existing codebases when they could help fixing the existing toolset and make it safer?

I don't get it.


To me this idea sounds a lot like "why do we need IPV6? Why not keep IPV4 and add a few bytes?" except it's effectively almost the same amount of breakage. Either you're C-compatible or you're not.

Making C memory-safe by adding extra annotations to allow for static zoning/borrow checking is possible, but the churn involved is probably not much different than rewriting in a different language. See Cyclone for instance if you want to see what such an approach looks like.

On top of that modern languages have facilities that are seriously lacking from C, memory safe or not. C's error handling is atrocious, C doesn't have (standard) destructors, C doesn't have type inference, C doesn't have algebraic types, C doesn't have proper string/collection support etc...

I like C quite a lot, it's probably the language I know best, but it's definitely very outdated. I think in the not so far future it'll be almost entirely replaced outside of legacy applications, hopefully with Rust. I think the watershed moment will be when the Linux kernel is rewritten in an other language, which definitely won't happen overnight but at this point I'm fairly certain that it will happen within a couple of decades.



> To me this idea sounds a lot like "why do we need IPV6? Why not keep IPV4 and add a few bytes?" except it's effectively almost the same amount of breakage.

[DJB disagrees](https://cr.yp.to/djbdns/ipv6mess.html). Quote:

>The IPv6 designers made a fundamental conceptual mistake: they designed the IPv6 address space as an alternative to the IPv4 address space, rather than an extension to the IPv4 address space.

> Either you're C-compatible or you're not.

If we take this black and white approach then we must say that C++ is not C compatible just like Ada isn't C compatible.

The reality is that C++ being broadly a superset of C was integral to its adoption, even though that compatibility isn't perfect. It enabled easy incremental porting of projects.

This is really just a nit though. I do agree with your conclusions.


With all due respect I disagree with DJB here. I can't see how his proposal would significantly improve the current status quo. Note step one of his self-described "straightforward" transition plan (emphasis mine):

>0123456789abcdef0123456789abcdef to 192.5.6.30: The client sends a UDP packet to the .com DNS server asking for the address of www.google.com. The client software, intermediate computers, and server software have all been upgraded to handle the client's extended address.

A lot of work in this one "straightforward" step. I know that DJB is a great network engineer, so I'm willing to consider that I'm missing the point here, but at the very least I don't find this particular exposé very convincing.

>If we take this black and white approach then we must say that C++ is not C compatible just like Ada isn't C compatible.

Agreed, in a parent comment I mention C++ being effectively mostly compatible with C. But I can't really imagine how you could do something like that with a memory-safe subset of C. The changes involved are pretty thorough (you basically either need a garbage collector or a borrow checker). Things like raw pointers (especially nullable ones) can't really exist in their current form in a memory safe language.


The main strength of a memory-safe language is that the safety mechanisms are opt-out, not opt-in: you can confine the unsafe behavior to a small, manageable portion of the codebase. And trying to "fix" C to become a memory-safe language would probably just result in something like a Rust dialect that merely bears more superficial resemblance to C. In any case, thorough re-writing of existing codebases would be required. You might as well re-write in a language that currently exists, rather than add creating a new language to the top of the to-do list.


I would call C++ a memory safe language that has more than a superficial resemblance to C. Rust makes is more obvious when/where you are intentionally opting out, but most C++ memory bugs are where someone opted out of the modern C++ way to use the C way. (including new/delete as the C way).


> I would call C++ a memory safe language

C++ is not, in any sense, a memory safe language.


Then you don't know anything about C++, only the C legacy it is built on. Modern C++ has good data structures and good memory management.

To be fair actually using those features of C++ instead of the C legacy is still catching on. Most instructors aren't even teaching it.

Using C in C++ is like using unsafe everywhere in rust - you can do it, but you lose all the advantages.


Memory safety is not about what a language has. It's about all the things a language doesn't have, doesn't allow.


Even if you stick to "modern C++" and never use C-style pointers, arrays, enums or unions, it is still trivial to have memory errors in C++.


This is a common viewpoint but also wrong. There's plenty of ways to invoke memory unsafety in 'modern' C++: iterator invalidation is probably the biggest source.


While you are technically correct, those who study C++ security note that those are a tiny minority of all security errors. Still an issue, but not nearly as common an issue as the ones that come from the C roots of C++.


No true Scotsman huh?

https://en.wikipedia.org/wiki/No_true_Scotsman

Also can you link to an example of this purported memory safe C++ compiler? Please note for an apples to apples comparison it must reject all memory unsafe non-modern C++ constructs.


Iterator invalidation was the cause of a Firefox zero day in 2016. https://bugzilla.mozilla.org/show_bug.cgi?id=1321066


    std::string_view xx = something_that_returns_a_std_string();
    use(xx); //oops


If C++ compilers had a default mode of rejecting code that was unsafe (any code using raw pointers or unchecked array access, I guess?), then sure- you could call it that. But as it is, no way.


> would call C++ a memory safe language

LOL.


You can't fix what is fundamentally broken.

In the same idea of "Any sufficiently complicated C program contains an ad hoc, bug-ridden, slow implementation of half of Common Lisp." let's paraphrase

Any sufficiently safe C program contains an ad-hoc (etc.) implementation of the infrastructure needed to make it safe:

- custom string implementation

- custom object/memory management

- custom memory slices and other data structures

Not forgetting the compiler tricks added to try and catch when the program does something stupid


> Would not be cheaper to fix those unsafe languages, such as C

I don't think C can be "fixed" without turning it into a radically different language. At that point, why not simply start from scratch?

This is about language design that makes some languages fundamentally unsafe, not about fixing some bugs or hardening the toolset.


FWIW, with my contrarian hat firmly affixed, I suspect that C plus tools makes more economic sense than Rust.

https://compcert.org/ > The CompCert project investigates the formal verification of realistic compilers usable for critical embedded software. Such verified compilers come with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program.

https://frama-c.com/ > Frama-C gathers several static and dynamic analysis techniques in a single collaborative framework. The collaborative approach of Frama-C allows static analyzers to build upon the results already computed by other analyzers in the framework.

https://www.cprover.org/cbmc/ > CBMC is a Bounded Model Checker for C and C++ programs. ... CBMC verifies memory safety (which includes array bounds checks and checks for the safe use of pointers), checks for ex­cep­tions, checks for various variants of undefined behavior, and user-specified as­ser­tions.


I don't know how well that would work. For example, my understanding of Rust's bounding checking for slice indexing is that often the compiler can prove it's not needed and the check is optimised out because of the type system. This woulnd't work in C because of C's lacking type system


The people "in charge" of C are just not interested in radical changes - they have not even bothered to add better strings. There was a recent post here describing how the C standard bodies even oppose adding warnings, because it would increase the size of building logs.

So the only chance for advancement are those who propose radical changes, like Rust, or Zig if you want something less complex and "spiritually" closer to C


How do you "fix" C?


Rewrite it in Rust!


Zig is a “fixed” version of C.


No, it's another language.


I based my comment on Jens Gustedt's article [1] about adding a "defer" mechanism in standard C.

Clearly, the C committee works hard on improving the language and they are in the process to standardize important features, such as lambda functions, auto, typeof (like C++'s decltype more or less), and so forth.

In my humble opinion, "defer" can mild the buffer / memory overflow, but of course not permanently remedied it 100%; to me though it's better to have such mechanism than have nothing at all to protect you.

https://gustedt.wordpress.com/2020/12/14/a-defer-mechanism-f...


Defer seems more like a solution to making cleanup easier and less error prone, preventing memory/fd/mutex leaks. It wouldn't significantly improve memory safety IMO. Adding a "slice" type with checked bounds (something basically every C codebase in existence ends up reimplementing somehow) would be a much greater improvement IMO.

But overall even though I still use C almost daily, I'm a bit skeptical of the committee's direction with these changes. They added half assed generics through macros, now a defer mechanism and a bunch of other stuff that's, IMO, out of scope for C. People have already tried to make a "better C" with all these bells and whistles, it's called C++, and it's still mostly backward compatible with C.

IMO C should be considered to be in "maintenance mode" these days. And again, my IRL job involves writing a lot of it, so it's not like I don't care about it.


Yup, there is a group of people who think that we should add brakes and lcd displays to horses because cars have these nifty things. For some reason, they don't understand it's ok that horses are not used for everyday travel anymore.

I hate the mess that c++ is. It started out with a object based programming paradigm with c like syntax. Then generics/templates were added. Then we have modern c++. It's 3 different languages crammed into one. The c++ committee has a serious case of nifty-itis.


grrr why are they introducing guard {} blocks? Would not want an extra level of {block} just for this.

__attribute__((cleanup)) gcc/clang extension already works without extra blocks. GLib has been using it for a while https://blogs.gnome.org/desrt/2015/01/30/g_autoptr/


Defer doesn't even replace destructors, let alone resolve memory safety in C.


> Another analysis on security issues in the ubiquitous `curl` command line tool showed that 53 out of 95 bugs would have been completely prevented by using a memory-safe language.

That's not just a command line tool but a library, libcurl, with a command line wrapper. That library is easily used from a myriad of memory-safe languages, which is possible in part because it doesn't foist highly opinionated memory representations onto the caller.


A worthwhile goal.

But not in a memory unsafe language, please. There's hype and there are lies: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Ao...


This is flamebait, ignoring that "nearly perfect detection" and "no detection whatsoever" are very different things.

But also, stack overflows aren't memory unsafe in modern rust. The issue search you want to find memory safety issues is: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Ao...


No flame sit. It is either safe or not. Safer doesn't qualify as safe.

Your query searches for type unsafeties. A third query would be for concurrency unsafety: deadlock. All three unsafeties are builtin by design. There's no way to deny it. But they still do repeat the same lies again and again.

Stack overflows being memory safe you can tell your grandma or upper management. Problem was rust was once memory safe, but they ripped out the GC. Now it's just better than C++, but not better than Java or really safe languages. Overhype rarely reflects well on a community.


> No flame sit. It is either safe or not. Safer doesn't qualify as safe.

I agree and I often have to try to make this clear to proponents of "modern" C++/Go.

> Your query searches for type unsafeties.

His query searches for all unsoundness in Rust (including both type- and memory unsoundness). It does not search for unsafeties. Unsoundness can enable unsafety in code written in save Rust. However, unsoundness is fixable w/o redesigning the language (though may be very hard in practice) and usually doesn't cause significant breakage when fixed (in Rust at least). Unsafety in code enabled by fixing unsoundness often disappears when the unoundness is fixed but will result in a compiler error or -lint when the unsoundness is fixed if it doesn't disappear. Unsafety in the language is not fixable w/o major redesign and ecosystem wide breakage. It should be noted that most remaining unsoundness in rustc is suspected to have very low impact in practice and are very hard to fix but they are being steadily fixed.

> A third query would be for concurrency unsafety: deadlock. All three unsafeties are builtin by design. There's no way to deny it. But they still do repeat the same lies again and again.

Deadlocks are theoretically impossible to eliminate in any Turing complete language. In Rust there are often ways to prove your program doesn't contain any deadlocks but indeed some academic research languages (mostly dealing with effect types) makes this easier in more cases. At least in Rust a deadlock results in the lack of stuff happening that you would've preferred happen (which is bad enough), not in UB like in C++ (which is considerably worse).

Memory leaks in Rust are a similar situation to deadlocks although admittedly authors of unsafe Rust need to take some care to ensure it remains similar and not a way that worse memory safety issue could occur. The Rust designers wanted to make it very clear that they consider memory leaks bad but not part of Rust's memory safety guarantees and even went so far as to add a safe function to std that leaks memory (i.e. not as a side-effect, leaking memory is what it is for).

On this note. No language can ever be save in all respects. But save doesn't mean no bugs. It means you have explicit guarantees and enforced contracts. Save rust explicitly guarantees concurrency safety except that deadlocks can occur and the compiler enforces the guarantee unless an unsoundness bug causes the compiler to fail in its explicit contract to enforce it. Similar with memory safety, which is guaranteed (and enforced except sometimes in the precence of unsoundness bugs) but the guarantees explicitly exclude memory leaks. That still leaves a lot less to worry about than in "modern" C++ where absolutely no concurrency- or memory safety is in any way guaranteed nor enforced. Rust is all about eliminating all compromises it can and where the cost/benefit ratio makes sense but to still be pragmatic when and where appropriate.

> Stack overflows being memory safe you can tell your grandma or upper management. Problem was rust was once memory safe, but they ripped out the GC. Now it's just better than C++, but not better than Java or really safe languages. Overhype rarely reflects well on a community.

Stack overflows are memory safe in Rust. Rust's safety guarantees don't say stack overflows will not occur only that they would not cause specific issues if they do occur (like double free or UAF or ...). Again. Sometimes it is possible to prove they cannot occur but in general this is an impossible problem to solve in any Turing complete language.

EDIT: Also: GC's don't prevent stack overflows either. Therefore I don't get your point.


This is flamebait, and you know it. Stack overflows in Rust are perfectly safe because they hit a guard page.


"bugs would have been completely prevented by using a memory-safe language"

Relying on safe languages isn't going to work as the defect resides in the Memory Management Unit. You would think by now these 'integrated' 'innovators' could come up with a MMU that provides total process isolation.


Have you confused the kind of memory safety that Rust, etc. are about with the Spectre and Meltdown vulnerabilities? Or do you think that each individual array that's potentially subject to a buffer overflow in C should have been allocated by an entirely separate process?


Plan9 sort of does the latter (crypto in its own process and so on)


The security issue with memory-unsafe languages is that programs can overwrite their own memory, not the memory of other processes.


Software written by a party with a proven record of surveillance of its users in the service of governments, and of selling user tracking data en masse to advertisers as its business model - cannot, in general, be trusted.

Now, if the codebase is small enough to allow for scrutiny by 3rd-parties, that point may be moot, but even then, Google may have some ulterior motive which will not immediately become apparent.

So, I'm kind of suspicious of such an initiative.


> The ISRG's approach of working directly with maintainers to support REWRITING tools and libraries incrementally falls directly in line with our perspective here at Google.

So, its not enough that anybody that don't want to rewrite millions of lines of code in Rust because its just pure insanity, need to be constantly harassed and forced to feel ashamed (or feel even feel guilt) because "how in the world you did not did this code in Rust"

Now there's a whole INSTITUTION to funnel the scarce resources, not by fixing the security bugs but by forcing others to Rewrite Everything In Rust® or else they will have their popular projects hijacked by highly paid engineers from companies with unlimited financial resources to rewrite the whole thing in Rust leaving the maintainers of the original project eating dust if the don't conceive?

If this is not going to support the source code maintainers to patch the bugs in whatever technology they are using, this is the Rust bullying that used to be just annoying social dynamics, being take into another level.

Daniel Stenberg and libcurl will be exemplary nailed so others feel more inclined to follow or else see their projects being taken out of their hands if they do not consent.

Imagine if you are a open source maintainer with scarce resources, being approached by an institution that promise you to give you some funds, but only if you show a plan to rewrite the whole thing in Rust?

Edit: Just follow the Stenberg tweeter if you are curious https://twitter.com/bagder to see all the pressure this guy have being taken lately.


The more pressing issue is the fact that critical tools that we depend upon are not properly funded or the incentive to properly maintain them is simply not there. Many of these projects were started as pet projects in peoples spare time. Sure, Google gives a lot of money and contributes to some of these projects but their contributions do not match how much they depend upon them. This is all compounded by the fact that Google is actively working to undermine many of these same projects because they can't get a license change or projects don't want to bend to their will.

Rewriting the world in Rust or other "memory safe" languages is not going to be the silver bullet that is being claimed. In the case of Rust, I am tired of people evangelizing it when it's plagued with issues that undermine its goals and rewrites will bring about more nasty bugs. "Unsafe" code will still need to be written and not being able to use or have stable toolchains to work with is a non-starter for a lot of projects. Until all of these languages have set standards and stable toolchains, hopefully the Rust Foundation will push, these languages are a non-starter.


> In the case of Rust, I am tired of people evangelizing it when it's plagued with issues that undermine its goals

Do you mind explaining what you mean here? You seem to be alluding to something different than an unstable toolchain which you mention after.




Applications are open for YC Summer 2021

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: