Hacker News new | past | comments | ask | show | jobs | submit login
Preparing Rustls for Wider Adoption (abetterinternet.org)
230 points by jaas 9 months ago | hide | past | favorite | 132 comments

It took me a minute to work out who 'a better internet' aka ISRG are. It's the folk behind Let's Encrypt. That's cool.

Anyway, this sounds great to me. I'm not a huge fan of memory unsafe languages, especially for critical code.

I find it deeply worrying that Google controls letsencrypt and an increasing amount of "open" source software.

I find it shameful that letsencrypt issues statements like this one:

"we hope to replace the use of OpenSSL and other unsafe TLS libraries in use at Let’s Encrypt with Rustls."

OpenSSL has made companies billions (trillions?) of dollars, now they drop it like a hot potato.

Have you not followed the discussion on the state of OpenSSL? It desperately needs to be replaced. Google pushing for it and providing funding for it is not somehow shameful, it's being a responsible participant that gives back to the community.

Google had more than 20 years for auditing OpenSSL and submitting patches. It didn't.

Instead NIH solutions are now popping up after having exploited OpenSSL for more than two decades.

Well, yes. It's still better to start doing the right thing late than never?

> Instead NIH solutions are now popping up after having exploited OpenSSL for more than two decades.

Building new crypto libraries to replace OpenSSL is not caused by NIH, it's caused by rationally examining the existing codebase and coming to the conclusion that sometimes starting from scratch is the sane choice.

Google is doing the incremental fixing approach (in BoringSSL) too, but it's very clearly a stopgap.

Google priorities:

1. Making money.

2. Making backwards incompatible changes to products and services in the name of best practices.

And the submission is now sinking after critical comments, as expected.

How does this compare to the verified TLS implementation from Project Everest?



From a layman's perspective:

1. The EverCrypt primitives are formally proven, whereas ring has no such formal proofs. Also it seems that all the EverCrypt primitives have portable (non-assembly) fallbacks, while ring has several primitives that are assembly only and have no portable fallback.

2. MiTLS is written in F#, which is harder to integrate with other languages than Rust.

> The EverCrypt primitives are formally proven, whereas ring has no such formal proofs.

We do use some of the Fiat Crypto stuff for elliptic curve computations. I am not opposed to switching some stuff to use EverCrypt or other things that might be better, as long as the performance is the same or better, and as long as there's a clear path towards that code being in Rust.

> ring has several primitives that are assembly only and have no portable fallback.

Either in the latest release (0.16.20) or the upcoming release, there is a portable non-assembly implementation of everything in ring.

Thanks for the correction!

> Also it seems that all the EverCrypt primitives have portable (non-assembly) fallbacks, while ring has several primitives that are assembly only and have no portable fallback.

Ring uses assembly to make sure it can do constant-time operations, to avoid leaking information through computation time. What does EverCrypt use for constant-time operations?

This is an important point. The C standard does not define the time it takes for an operation to complete as "observable behavior", so compilers are *always* free to change timings, even with optimization disabled. It is fundamentally impossible to guarantee constant-time execution in Standard portable C. It's possible to use non-standard intrinsics and compiler directives to get such a guarantee, but that's not portable.

AFAIK Rust also doesn't guarantee timing. That's part of why Ring uses assembly for the constant-time bits (really Ring uses code from BoringSSL for that, and BoringSSL uses assembly for that reason and for performance.)

Rust doesn't guarantee timing, but there are crates that carefully construct constant-time primitives that should work with current Rust, such as https://crates.io/crates/subtle-ng .

Personally, I would love to see enhancements to Rust and LLVM that would make it possible to provide such guarantees in pure Rust.

Word on the block is that there is a RFC to add support for secret types in LLVM, and Rust is waiting for them to also provide it in Rust [0].

[0]: https://github.com/rust-lang/rfcs/pull/2859

It doesn't seem to stop people from trying though: https://www.bearssl.org/constanttime.html

(This sort of thing is not my area of expertise, but has always made me uneasy...)

Yeah. That sort of thing works, right up until some compiler gets released that optimizes it out. It's "portable C", but the behavior it's trying to produce is non-portable.

Especially note his guideline to look at the assembly output. The best guarantee is to simply check that the compiler did what you wanted.

There are also some aspects that a compiler is unlikely to alter. EG if you write code with no secret-dependent branches, no normal compiler will insert any secret-dependent branches during optimization. It's generally safe to rely on the compiler not being actively malicious or de-optimizing code.

I don't think Intel/AMD make any guarantees about instructions being constant time either.

Correct, and don't provide any guarantees about electromagnetic side channel emissions or power side channels or…

The lack of a guarantee in assembly is less of a concern than the lack of a guarantee in C, since the CPU behavior is less likely to change unexpectedly than a C compiler.

Though Intel at least does provide a list of instruction latencies and throughputs, in their "Intel 64 and IA-32 Architectures Optimization Reference Manual"[1].

[1] https://www.intel.com/content/dam/www/public/us/en/documents...

Normally, rather than counting on two different codepaths to take the same number of cycles, assembly code written for constant execution time will always run all the same instructions unconditionally.

So, you don't have to know how many cycles every instruction takes; you just have to make sure you use instructions that don't take a data-dependent number of cycles.

This is only possible with resource aware types, it would be nice to import a function with resource constraints such as constant time, etc.

It should be possible to create constant-time C code, if the code avoids branching (if, case, goto), and all loops have a constant number of operations.

Expressing most algorithms this way is not impossible, but does nor feel natural for many.

> It should be possible to create constant-time C code, if the code avoids branching (if, case, goto), and all loops have a constant number of operations.

It fundamentally isn't, because there is no way (in standard C) to prevent the compiler from introducing performance differences. Even if you avoid control flow constructions, what if the compiler e.g. generates a separate codepath for when one particular variable is 0 (perhaps as part of arithmetic optimizations)? There's just no (standard) way to stop that happening, no matter how cleverly you write your code.

Sad! It's as if we might need some kind of simple portable assembly, one which straightforwardly translates simple constructs and formulas to the target machine code. Like, well, C used to be in 1975.

Jokes aside, I wonder if some kind of -Ox option could be standardized to switch off all optimizations, like -O0, and also guarantee no code rewriting for whatever other purposes.

Hand-crafting constant-time algorithms in assembly is even more error-prone.

In fairness assembly isn't what it was in 1975 either - with superscalar architectures and the dominance of caches these days even assembly is not as constant-time as you might expect.

And assembly makes the code harder to audit: https://cryptologie.net/article/520/cryptography-and-assembl...

But assembly is unavoidable.

for the guaranteeing const execution time aspects. More or less any language compiled by LLVM, GCC or similar optimizing compilers can't do so. (And even wrt. assembly the cpus could do optimizations which also brake this but it's less likely).

Same is true for any language using a JIT with optimizations like e.g. Java, JavaScript, C#, etc.

Practically the degree of how much compilers could mess up const execution time guarantes is language dependent but for many languages especially including such using compilers created for C/C++ basically you have no useful guarantees.

At best you can create some code which just with the current version happen to not be optimized in a bad way. But this means you will have to review the produced assemply code and do it again and again every time the compiler (or just surrounding code) changes... To make it worse the high level code is force to use all kind of tricks to prevent the compiler from doing certain optimizations, which often obfuscate the intent of the code.

So all in all having a small very well reviewed set of assembly snippets for the core primitives the the easier to review and more reliable way in many cases. (Note that "a small set" means that just a few primitives are assembly only, not all the crypto, which normally is good enough).

Through best would be to have some language which focuses on both making it reasonable to implement such primitives in a more high level language and has tooling for proving code correctness and also code-to-assembly transformation correctness (e.g. see what SeL4 did wrt. proving assembly).

You might be interested in https://www.microsoft.com/en-us/research/publication/coq-wor... if you've not seen it already.

MiTLS is written in FStar, not F#. FStar has some tools (like Kremlin [1]) for extracting C code from the verified FStar, so in theory it wouldn’t be too hard to also extract Rust code too.

[1] https://github.com/FStarLang/kremlin

Straight from https://mitls.org/

> The stable version of miTLS including the new 0.9 release are written in F#

With F# being a link to https://fsharp.org/

The original code was written in F#, then re-written later in F7 then F* (FStar).

Doesn't .net have pretty good C FFI support, what would make it harder than a Rust lib?

Happy to answer any questions that come up!

Hey @dochtmann :)

Isn't rustls [1] also built on very unsafe groundwork? Namely ring [2], which, according to github, contains 47.3% Assembly and some C as well.

I'm not trolling here - we were discussing this a lot in my peer group lately.

[1] https://github.com/ctz/rustls [2] https://github.com/briansmith/ring

One thing to keep in mind is that the low level building blocks of crypto algorithms can be relatively easy to test, compared to higher level protocol and application code. For example, a block cipher takes simple inputs, usually a couple of fixed-length arrays and maybe some integer flags. There might be a ton of assembly under the covers, but that assembly isn't responsible for reasoning about pointer lifetimes or parsing data formats or any of the usual things that tend to trip up unsafe code. (Like a TLS implementation!) Instead, the block cipher is a pure mathematical function of those inputs, and that makes it relatively easy to come up with a set of test vectors that cover the function. This also means that the C code and Rust code for the same block cipher tend to look very similar.

Now there definitely are some tricky requirements in crypto code that application code doesn't need to deal with, like constant-time requirements. But auditing for those isn't really any harder in assembly or C than it is in Rust. In the end, porting these sorts of core crypto algorithms from C to Rust tends to be more interesting from a build systems and tooling perspective than from a correctness perspective.

The goal of the ring project is to be much safer than OpenSSL without any notable decrease in performance. That is, my goal is to give you memory safety "for free" if you switch from OpenSSL/BoringSSL to ring. In some cases it is better than free because we end up being faster.

The assembly code in ring is some of the most heavily-tested code in the world. It's fuzzed pretty much continuously in various projects that use it, and a bunch of testing has been done on it. It is from BoringSSL, and much of it is shared with OpenSSL and/or Linux kernel. As we are able to replace the assembly code with safer code, we'll continue to do so, just like we've replaced most of the C code with which we started.

thank you for your work in this. we've been using rust-tls/ ring for some time now in our product.

It uses ring for cryptographic operations, yes. However, note that all the ASM in ring is meticulously kept up to date with BoringSSL upstream, which ring was derived from. Plus, there was pretty successful third-party security audit last year.

Also, the goal is definitely to bring all of that code into Rust, unfortunately Rust lacked the features to do that safely (things like const generics).

Could I ask why const generics would be a blocker? I see how it would help facilitate more elegant APIs as well as better stack allocated structures. However is it not possible to live without this, possibly with a slightly more clumsy design?

>Plus, there was pretty successful third-party security audit last year.

The security audit referenced by the post suggests offering EverCrypt as an optional alternative to ring. Are you going to act on the audit's recommendation, or continue to only offer ring?

> Isn't rustls [1] also built on very unsafe groundwork?

Depending on what you mean by "groundwork" literally everything is. Hardware doesn't obey Rust's rules, and you need to interface with hardware to get input, and do output, so literally every program will have unsafe code at the base.

The key difference is that Rust gives you the tools to explicitly demarcate what is safe, and what is not, and build safe abstractions on top of (hopefully validated) unsafe foundations.

> Hardware doesn't obey Rust's rules

Neither does the OS where rustls is running.

I think Rust will have more adoption and more libraries like Rustls will be developed. I also think that when this happens, also more exploits targeting Rust code will exist too. I guess the excuse (sorry for using this word) will be: "In fact, the Rust code is still safe. What happened is that a pointer returned by (or used in) an underlying C library got messed up with a very clever timing attack, and somehow the pointer emerged into Rust code... etc.".

Also, ring enforces everyone to use the latest version by pruning older versions from crates.io, which means your builds will fail every time they update.

Main reason I stopped using it.

Of course, if our claim is that we want to avoid security bugs, and we accept the principle that (without some more specific definition) all bugs are security bugs, then any time ring fixes a bug we want to avoid using the old version...

Now for all I know, ring has never fixed any bugs and it just loves adding new API features so that this pruning has no desirable security properties at all, but in principle I can see that this is the equivalent of the standard boilerplate Linux release text which tells you that you should update to the latest kernel because they fixed bugs.

If you have a complete threat model and if you are capable of the insight needed to examine all changes and determine how they impact that model, you could successfully choose whether to upgrade based on whether a new version fixes a bug you care about. But chances are you don't have such a model and even if you did you aren't capable of the inhuman levels of insight needed, even in a language like Rust (and forgetting that we're talking about this because large parts of ring aren't even in Rust).

crates.io and the Rust community adheres to semantic versioning.

If ring wants to notify me that I should update, they should send an email to a security mailing list, open a CVE, register the cve in any of the rust services to notify users with those dependencies (there are some, like crev), etc.

Pruning your releases from crates.io just means that I am going to be annoyed the first time it happens, will start looking for a solution the second time it happens, and it won't happen a third time (and it didn't). If you want to wake me up a Saturday at 4 am, the world better be on fire.

This is probably the only dependency I can remember as being... more than annoying, toxic. I still prevent any of my dependencies from ending up with ring as a dependency. If that shows up in our dependency tree, CI fails, and that change cannot be committed. Unfortunately, this pruning of old releases was only one of the issues with ring (there were others, like cross-compiling it wasn't easy, etc.). All in all it was a no brainer to drop it as a dependency.

I don't think I've ever met a rust dev with something nice to say about `ring`. In a meetup a couple of years ago another rustacean said: "`ring` is so secure that it protects you from using it in your projects". Sums it pretty well.

The library has couple of thousands of daily downloads so for the latest version, and like 25k daily downloads for other versions, so maybe things changed now.

When I merged security fixes from BoringSSL/OpenSSL, I yanked the old versions of ring that didn't have the security fixes. I thought that was a pretty reasonable policy, however people who like to comment in these forums disagreed very loudly, so I stopped doing that. Not sure that's better, but there's less complaining.

In general, my initial thinking was based too much on the assumption that people would help maintain the things that depend on ring to update them to the latest release. It turns out there's less cooperative maintenance like that than I expected.

I loved your policy and appreciated how principled it was. People here are too harsh and most of them don't write code in Rust where regular updates are much more the norm than other communities.

Yanking releases which have bugs / vulnerabilities in them is very much not the norm in the Rust community.

This is why projects like https://github.com/RustSec exist.

I don't know about that, crates that RustSec has advisories for are often yanked, in my experience.

Bugs? No. Security bugs? Yes.

Sure, it's quite possible that not every single one ever is. One single version of one single library not being yanked doesn't mean that nobody ever does it.

> In general, my initial thinking was based too much on the assumption that people would help maintain the things that depend on ring to update them to the latest release. It turns out there's less cooperative maintenance like that than I expected.

I think that's a reasonable assumption.

What isn't reasonable is to expect people to "upgrade right now". Not everybody lives in your time zone, so when you yank a dependency, you might be breaking a workflow in the other part of the world at 3 am, and if some webserver doesn't deploy or whatever, somebody will get a call.

I'm not suggesting this is an easy problem to solve, but there is a wide range of options before "never upgrading" and "force an upgrade right now". Some of these are supported by Cargo via Cargo.lock, etc. so the responsibility for how this is handled doesn't fall on one library or person.

Building a secure system is also not the exclusive responsibility of `ring`. If I'm building a secure system, I have to assume that `ring` will have a bug that's exploitable at some point, and that someone will use it in a zero-day, and my system needs to be secure even if that happens.

So "updating right now" doesn't buy me much. Its something that can wait until after a meeting, or after my vacation, or until monday. Its not a "the world is on fire" situation, even though it would still be pretty severe.

I'd still like to get "notified" ASAP and asynchronously somehow. While updating ring is low effort, the update still needs to "internal QA,..." etc. at companies, and that takes people's time that must be planned on.

In a properly-designed CI/CD system, a dependency getting yanked isn't an emergency unless you choose to treat it as such. In particular, if you don't want your build to fail because some dependency got yanked then you need to use a Cargo.lock, and you need to ensure that you're not overzealous in your use of cargo-deny and similar tools.

I'm not sure if you were affected by this, but Cargo introduced a (regression) bug a couple years ago that caused it to fail when a crate got yanked when it shouldn't have. This bug was eventually fixed, but lots of people blamed ring for this bug. If this Cargo bug hadn't been introduced then most people who were using Cargo correctly wouldn't have been negatively affected by ring's old policy.

With software which needs security considerations it's also common to want to test the vulnerable versions, write tests internally and verify the broken/fixed behaviour. Yanking old versions makes this annoying. (Yes you can rebuild from the repo tag, but that's annoying, especially in indirect dependencies)

> If you want to wake me up a Saturday at 4 am, the world better be on fire.

In a previous role I actually have had things set so that I might be woken at 4am on a Saturday, IIRC specifically under certain conditions it'd play "Straight out of Compton" at full volume on my Hi Fi to ensure my attention, which gave me about 10 seconds before it gets real loud.

I was leak hunting, specifically looking for a huge leak in a production system that we couldn't reproduce on smaller test systems - so I needed to wake up, attempt to diagnose the leak and then (regardless of whether successful or not) mitigate it (kill the bloated process, one transaction fails but everything else will auto-repair) and go back to bed.

But what I don't understand here is, why are you constantly rebuilding and alerting on failure? A CI flag can wait until Monday stand-up, are you auto-deploying any change of state even when there aren't any humans around to cause that? Why? That strikes me as up there with Apache's "But your Good OCSP response expires in 18 hours, so I stapled this newer Bad one instead" in terms of terrible mistakes.

If instead your new builds fail after pruning, the human who is causing a new build can decide what to do about that when it happens, no need for anybody to be woken at 4am.

Your Saturday 4am may be someone else's Friday 3pm. Humans may be around and doing their normal work.

Which is why you don't[1] push to production on Friday at 3pm.

[1] As always there are exceptions to this rule

That's not a great rule in this case. Unless you also don't push on Monday 1pm because it could be someone's Monday 1am?

I looks like we are talking about different things.

So you have a team (i suppose could be just 1) or teams across few time zones.

Each team should deploy in a way that they have people ready to fix thing if they go bad.

You can do that by having large enough team in similar timezomes (+/- 2-3 hr), or by paying extra for having people on standby at 1am.

My personal opinion is that having large enough team (again could just be 1 guy) in similar time zone is preferable.

Bottom line is, deploying new stuff is always risky. That's why people spend so much time trying to reduce this risk (various test, CI, staged rollout ...). And sometimes all of that still fails, and you need people to either rollback or fix it on the spot.

I thought crates.io doesn't allow removal?

You can yank crate versions, but that doesn't affect builds which lock a version via cargo.lock so failing builds shouldn't be an issue.

They meant yanking.

That hasn't been the case for a long time, a year or more.

Your builds will only fail if you don't check in your lockfile.

That issue should be fixed in the next release. It is definitely a nuisance, but fixing it is not easy due to the assembly code involved.

Had the same thought the other day: https://cryptologie.net/article/520/cryptography-and-assembl...

Not a fan of assembly for cryptographic code.

This looks great. As I understand it, the major reason openssl is still in use (at all, really) -is a very unhealthy dependency on it's demented and complex Api... Has anything changed that makes it easier to switch from broken ssl lib to a new "perfect" one?

Are we likely to see stdlib changes in python, ruby etc?

The site mentions:

> Enforce a no-panic policy to eliminate the potential for undefined behavior when Rustls is used across the C language boundary.

Is this a thing? Does panic open up for undefined behavior when using a rust library via C?

I can't recall seeing it mentioned before/in other rust threads/projects?

I whish you the best of luck- libopenssl is terrifying :)

Panicking (unwinding) across an FFI boundary is very much undefined behaviour. You'd get analogous cross language issues with C++ exceptions. Or mixing GCC ABI unwinding with MSVC ABI unwinding.

Nonetheless, there is a working group, proposing to make it defined under certain circumstances. https://blog.rust-lang.org/inside-rust/2020/02/27/ffi-unwind...

But is this undefined behavior in the typical c sense (lol, I see you call function possibly_undefined() I'll just make you a sandwich and clobber all registers instead) - or undefined behavior in the sense that it should crash - but might return?

I’m not sure I'd draw a distinction. People used to say you should initialise your variables in C89 to NULL, so you'd get a crash if you forget to properly set the variable. But enough security bugs have been caused by null pointer dereferences.

In the Rust case, IIRC, Rust annotates functions with the LLVM attribute no_unwind. I'd expect bad things, thanks to optimisation passes, if an unwind were to start. Doubly so if mixing GCC with MSVC ABI.

> is this undefined behavior in the typical c sense (lol, I see you call function possibly_undefined() I'll just make you a sandwich and clobber all registers instead)


Thank you for that link, I thought (production build) rust panic was simply a core dump. I didn't realize there was stack unwinding going on.

There are two options at the moment [1]:

  - panic=abort (uncatchable)
  - panic=unwind.
With the latter, you would need a catch_unwind at the FFI boundary.

[1] https://doc.rust-lang.org/cargo/reference/profiles.html#pani...

I see. I would think that for a library, in production, all you'd want for panic was a crash. (I guess this is the abort case).

AAre the panics being discussed for removal in rust tls "soft" errors that should not crash?

There's a categorization here (I don't have access to the triage spreadsheet though):


It seems like there's been the most effort on category 4, the ones that can't be reached but can often be proven away with refactoring and typestate. Removing these ensures that the APIs don't expose Result types to callers unnecessarily.

Category 1-3 have different severity, but they are all planned to be folded into Result error values.

On the Python angle, I recently spent some time building Python bindings for another Rust project with PyO3 and was quite happy with the result. Since then, I’ve been toying with the idea about building a somewhat compatible replacement for Python’s ssl module based on rustls. I’d be happy to work on that, but I’d need to find some way to fund it (Patreon or GitHub Sponsors might work, but so far I haven’t had a lot of luck gaining sponsors there).

What does this mean?

> Make it possible to configure server-side connections based on client input.

The other three bullet points I can immediately understand what they're achieving (the "No-panic policy" is trickiest but since I'm learning Rust I knew what that means, and it also links a ticket describing more)

But for this one I haven't a clue, maybe it should be obvious, and somebody else will explain.

Using (defined) properties of the TLS ClientHello to determine how the server will respond.

For example, changing the certificate used based on the ALPN identity, the SNI server host, and the advertised client ciphersuites.

I've been doing this by implementing ResolvesServerCert, which has access to the ClientHello. It covers the acme alpn use case.

Though you can't use it to pick other properties, like making an ALPN protocol conditional on SNI.

Rustls depends upon Ring. Ring itself has a lot of code from BoringSSL, an OpenSSL fork.

Long term, is this OK? How do we ensure the assembly and C code in Ring is as safe as Rustls itself?

See my other answer.

RFC 7250 support (Raw Public Keys) is not in the list¹ of “Current features”, but neither is it listed under “Possible future features” or even “Non-features”. GnuTLS supports this feature, so GnuTLS is what I use. What is the status of this feature in Rustls?

EDIT: I found issue #423², but the reporter is not making a compelling case, since the use case presented is for a prospective new application, yet to be written.

1. https://github.com/ctz/rustls#current-features

2. https://github.com/ctz/rustls/issues/423

I don't think the list of possible future features has been kept especially up-to-date -- we should probably fix that (or get rid of the list). If you want to weigh in on that issue to discuss why you think it's a valuable feature to add, that would be great!

I don’t have a Github account, so I won’t, sorry.

If you reply here, I'm happy to relay your comments to that issue.

My comment is basically “I have a project with components written in C which might benefit from Rust, and would probably also benefit from Rustls. But the project requires RFC 7250 support, which only GnuTLS currently has.”

Congrats for securing the contract! Looking forward to the IP address stuff especially.

Rustls is really cool. I came to it because one of my projects used a library that in turn (by default) used OpenSSL. After much tinkering and not getting OpenSSL play well on that particular system I switched to Rustls and it worked out of the box and like a charm.

That being said and from what I understand Rustls is not a drop in replacement and it is not that easy for all Rust libraries which use TLS.

Which brings me to the point that I think a big leap forward for wider Rustls adoption in the Rust world itself would be to make it easily usable with all popular and widely used Rust libraries that depend on a TLS implementation.

What I would like to see is sort of a global (not per dependency)

    features = ["rustls-tls"]
so that all libraries and their dependencies use Rustls automatically and OpenSSL is completely out of the picture. I know that this is not on Rustls alone but on the library writers too, but still it would be really cool to switch the TLS implementation like that.

Then we could even dream to make Rustls the default and use OpenSSL (or one of its relatives) only if need be.

isn't what you're arguing for essentially to make Rustls (or at least its API) part of a Rust standard library, so that everyone's forced (or very highly encouraged) to use it instead of alternatives?

I think that if tcp[1] is in the stdlib, tls probably should be too. Maybe even ssh. I get there was a time when "listen to the network" meant plain text networking - but today, for a large subset of applications and libraries, not supporting tls is a bug.

[1] https://doc.rust-lang.org/std/net/struct.TcpListener.html

Implementing crypto is much much harder than simply providing abstractions over the OS' TCP/UDP APIs.

Far better to let the experts do it.

That's not really an argument against having it in stdlib, though?

It's much simpler to develop complex libraries outside of the stdlib. And given Rust's strong stability guarantees, it can't stabilize any APIs until they're finalized anyway. So development of new APIs often happens outside the stdlib and then if a third party library proves itself and can commit to a completely stable API then it might be uplifted into the stdlib, so long as there's a very strong case to do so.

I should also add that most applications won't even be using the TCP/UDP primitives directly. They'll likely be using a third party library. Rust's stdlib is intentionally minimal and lacking a lot of higher level features that are practically necessary for many applications.

Tls is part of Golang’s stdlib

Not suggesting it should be done, but note that Go does r̵i̵g̵h̵t̵ ̵t̵h̵i̵s̵ exactly this. They also have world class crypto people on the language team.

Go has to do this because they include https in the standard library. Rust defers that task to the ecosystem, so it doesn't have the same requirements.

Seeing that HTTP/S is now just as fundamental as tcp and udp, perhaps Rust should also consider doing the same? Sure, leave the more complex HTTP apis to the ecosystem, but Rust should at least have some sort of basic HTTP support.

I don't think you can get away with not treating HTTP as a first class citizen anymore. IMO, Go got this right.

If you spend all day writing web servers and clients it might seem fundamental. Most of my code doesn't touch the network so I don't mind if I need the ecosystem to do it - I do that in C and C++ too.

I think people are fairly apprehensive towards a bigger std because firstly, dependency management is almost trivial (C++ needs the kitchen sink in std because dependencies don't exist to the language, for example) so it's not that big of a deal, and secondly because a bigger std means a bigger commitment to APIs and maintenance by a relatively small group of maintainers for all eternity.

At the level of the stack where it makes sense to use rust, it doesn't make sense to put HTTP in std. At least not like Go.

You may have been correct 20 years ago, but I don't believe this argument holds water nowadays. HTTP APIs are everywhere from being required to control small embedded systems to interacting with large complex daemons. These usecases are exactly the space where Rust/stdlib is meant to be used.

Rust has included TCP and UDP in the standard library* because it was rightly recognized that these protocols, like stdio and filesystem io, have become fundamental to how most modern software interacts with its environment (system, IPC). HTTP today has also become fundamental in a similar manner. I argue that it is time that we treated it as such.

* as apposed to leaving them as an external dependency, like mio.

Rust has HTTP and HTTPS support. You just have to enable it in your Cargo.toml.

The ecosystem is part of rust.

To you, perhaps, but I wholeheartedly disagree.

Rust will have support only when it is included in the standard library.

In what sense does Go do this right?

It does right this not, this right. It's accessible to every go program: https://golang.org/pkg/crypto/tls/

I've never heard someone use right this, is that similar to exactly this or just this?

Yeah, I'd expect that to be phrased as "does this" - the right seems utterly superfluous at best, and very misleading.

"Right this minute" is a common phrase, but "does right this" isn't (at least in UK English).

Hmm seems you are right. I guess it's a german-ism that slipped into my english. The english analog to "macht genau das" is either "does exactly that" or "does just that". The literal translation I used "does right this" seems to have no use. So my bad, I'm sorry.

No problem :)

Http and https is accessible to every Rust program as well. It just has to be opt-in in Cargo.toml

Many crates do this type of thing. I don't know how hard it would be to isolate this type of thing into its own crate.

I'm really not a fan of having a global TLS knob to turn in all Rust crates... that strikes me as heavy handed and inelegant. Why give a global configuration variable for something used by a tiny portion of crates? And what if a crate is incompatible with rustls? How do you express that with such a global knob in a sane way?

It's the wrong abstraction too - this is handled better by dynamic linkage. Rust just doesn't have a great dynamic library story (a C API/ABI is not exactly what I'm talking about here).

What you'd really want is a

   trait TLS {
      // ... 
in some base tls crate and leave the implementation up to other crates. And any upstream dependency would require something like:

   pub fn initialize <T: TLS> (tls:T) { ... }

https://crates.io/crates/native-tls/0.2.7 provides this for native TLS implementations (which is what hyper-tls uses).

It's pretty bare bones though, because to expose things through that, all of the underlying implementations need to support it.

Back when rustls was initially announced, people criticized how few ciphersuites it supported, like only TLS1.3 and parts of TLS 1.2. Not sure, I might have been among them. And it still doesn't support TLS 1.1 or TLS 1.0. But since then, browsers have dropped support for many of those ciphersuites, so it's far less interesting to implement support for them at this point.

It can be useful if you need to work with legacy system that only support old cipher and protocol. But then I guess there is always openssl and co who still do the job.

I had to struggle with this recently. I have to deal with something that uses a root cert generated in 2005, valid to 2025, and signed with 1024-bit RSA. Rustls silently ignores this root cert. I can add it to the root cert store with no complaints, but it will not be used.

One problem with Rustls is that the error messages are not very informative. The situation described above just generated an "Invalid certificate" message. More use of anyhow::Context would be helpful. I don't disagree with Rustls disallowing decade-obsolete crypto. It's the "silently ignores" part that's a problem.

> he situation described above just generated an "Invalid certificate" message. More use of anyhow::Context would be helpful. I don't disagree with Rustls disallowing decade-obsolete crypto. It's the "silently ignores" part that's a problem.

Because of how X.509 certificate validation works, in general it's not possible to tell you why an issuer couldn't be found, because there are many possible reasons.

Regardless https://github.com/briansmith/webpki/issues/206 tracks improving the situation.

Brian surely knows this, but for other people's context: Certificates in the Web PKI [and likely in any non-trivial PKI] form a Directed Graph with cycles in it. So "is this end entity certificate trustworthy?" is already a pretty tricky graph problem that you're going to have to solve unless you either out-source the problem entirely to the application (lots of older or toy TLS implementations) or just rely on your peer showing you exactly the right set of certificates matching your requirements (the "chain" you may have seen in tools like Certbot) without you ever saying what those actually are.

In a sense it's going to have a big undigestible list of reasons the certificate wasn't found trustworthy, like if you asked grep to tell you why all the non-matching lines in a file don't match a regular expression. "The first letter on this line wasn't a match, and then the second letter wasn't a match, and then the third..."

However, as that ticket says, one relatively easy thing the code could do is notice if there was only one consistent reason and if so tell you what that was.

Also I agree with several commenters that webpki's current behaviour, in which it says "Unknown issuer" even when that's not the problem at all is undesirable and an even vaguer error might actually be better for these cases. See also, languages in which the parser too easily gets confused and reports "Syntax error" when your syntax was correct but something else is wrong, "Parse failed" is vaguer but at least doesn't gaslight me.

Are you assuming that the set of certificates that is given as input is the complete search space? It isn't; the server might have failed to send some certificate that, if present, would have fixed the "unknown issuer" problem. Also, we fail fast on errors; as soon as we find a problem, we stop the search on that path. Because of these reasons, identifying a single error would be, potentially, misleading.

In some cases, e.g. the end-entity certificate is signed with an algorithm that isn't supported, or an RSA key that is too small, we could add special logic to diagnose that problem. However, all this special diagnostic logic would probably approach the size of the rest of the path building logic itself. It doesn't seem appropriate for something in the core of the TLB of the system. Perhaps at some point in the not-too-distant future we can provide some mode with more diagnostic logic, and find a way to clearly separate this diagnostic logic from the actual validation logic to ensure that the diagnostic logic doesn't influence the result.

No, I'm pretty sure I wasn't assuming that, and I'm not sure what gave you that impression from what I wrote.

There are a variety of interesting strategies to decide that an end entity certificate you were shown is trustworthy, and you're presumably aware that Mozilla (in Firefox) eventually chose to go with a strategy of

* Requiring all root CAs to disclose intermediate CAs as part of their root programme

* Bundling this set with Firefox

Whereupon Firefox gets to decide whether the end entity certificate it was shown was issued by one of these trustworthy intermediates and short cut to a "Yes" answer regardless of which certificates, if any, the server included in the supplied "chain".

In the general case this isn't very applicable, but I note that webpki is named "webpki" and not "General purpose certificate validator" so actually it could go the same route (with the caveat that this needs frequent updates to avoid surprises with very new CAs)

Mostly though if you're quite determined not to introduce more complicated logic into webpki (which is understandable) I specifically don't like the gaslighting of saying "unknown issuer" as I said, when the reality is that you don't know why you don't trust the certificate, so say that.

If std::fs::File::open() gives me Result with an io:Error that claims "File not found" but the underlying OS file open actually failed due to a permission error, you can see why that's a problem right? Even if this hypothetical OS doesn't expose any specific errors, "File not found" is misleading.

> Bundling this set with Firefox

I love that they did that; it was actually my idea (https://bugzilla.mozilla.org/show_bug.cgi?id=657228). I believe the list is pretty large and changes frequently and so they download it dynamically.

> short cut to a "Yes"

Do they really do that? That's awesome if so. Then they don't even need to ship the roots.

> I specifically don't like [...] saying "unknown issuer"


> If std::fs::File::open() gives me Result with an io:Error that claims "File not found" but the underlying OS file open actually failed due to a permission error, you can see why that's a problem right? Even if this hypothetical OS doesn't expose any specific errors, "File not found" is misleading.

A more accurate analogy: You ask to open "example.txt" without supplying the path, and there is no "example.txt" in the current working directory. You will get "file not found."

Regardless, I agree we could have a better name than UnknownIssuer for this error.

> it was actually my idea

I didn't know that. Congratulations, I see this survived an early WONTFIX that tried to downplay the privacy implications of AIA chasing, which is even more impressive in today's "Who cares about privacy" world.

> Do they really do that? That's awesome if so. Then they don't even need to ship the roots.

I don't in fact know if they do that. They will always conclude that a typical Let's Encrypt certificate from R3 is trustworthy via ISRG Root X1, even when the "chain" provided by the server leads to the (still trusted) DST Root CA X3 but they could actually be choosing that path on the fly rather than just short cutting.

They do need to ship the roots still because

1. The UX does actually show roots, I still have Certainly Something installed here, but the built-in viewer also shows them.

2. Users can manually distrust a root. It would be weird if either: we expected users to go in and manually distrust dozens of weirdly named intermediates that chain back to that root, or, disabling the root was possible but just silently didn't work in most cases.

3. Some trustworthy intermediate CAs can exist that aren't captured. Imagine if Let's Encrypt spins up R5 tomorrow because of some disaster that makes both R3 and R4 unusable, they can sign it with the ISRG root, and it'll work for lots of people - it's not great, but it's workable, however even if they tell Mozilla immediately, there's just no way everybody's Firefox learns about this instantly. So it's good that if your server shows leaf -> R5 -> ISRG X1 (or indeed leaf -> R5 -> DST Root CA X3) the Firefox browser can still conclude that's trustworthy even though it didn't know about R5.

I look forward to seeing issue 221 resolved, thanks.

Rustls is still not formally verified. It may not have memory errors, but there are a lot of other security errors besides just that (and there may be more of them since it hasn't been used as much as open|boringssl has been used). It's a slight improvement at best.

Ideally you want a formally verified SPARK or CompCert C implementation of TLS, but those are not open sores compilers so they won't fly.

> It's a slight improvement at best.

That's a hefty judgment. If we look at major attacks on TLS endpoints I'd summarize them as, in order,:

1. Memory safety

2. Weak / old configurations

3. Invalid state machine transitions

Rustls addresses all 3 of those, or at least it attempts to. (1) is obvious - it's rust. (2) rustls only supports the subset of TLS versions that are considered safe. (3) rustls avoids issues like gotofail and smacktls by encoding state machines as types, turning invalid state transitions into type failures.

Plus, the actual crypto primitives are extremely well tested and built off of other existing libraries.

So yeah, maybe some aspect of the crypto is incorrect, but, while interesting from an academic perspective, the real world ranks those other 3 things as way more important.


Half of these are caused by C problems not present in Rust like: null pointer errors, buffer overflows, integer overflows.

The rest are logic errors (e.g., parsing errors) or crypto vulnerabilities (e.g., side channel attacks). There's nothing about Rust that magically prevents these errors. These vulnerabilities are discovered through testing and more importantly real world usage. Rustls has not been used nearly as much as or as long as openssl, so critical bugs could be in Rustls. You could encode some logic in the type system, but the types are still code that have to be tested. It's not a formal proof. It could still be wrong.

It depends on your threat model and your risk tolerance whether you want to depend on relatively unvetted code.

I'm just going by publicized, real world attacks, which I think fall under the 3 categories I listed.

> These vulnerabilities are discovered through testing and more importantly real world usage.

FWIW I disagree that real world usage is necessarily more important. It is quite important, but at the same time the real world usage isn't going to hit all sorts of weird quirky paths. By contrast, Rustls has 97% line coverage, OpenSSL has ~65%. Of course, coverage isn't everything, but that's notable.

> It's not a formal proof. It could still be wrong.

So long as the type system is sound, I'm not sure this is true. If you embed a state machine into your type system your type system guarantees that the state machine executes as defined. Granted this is not checking against an external model.

I will say that rustls still will benefit from wider adoption and more evaluation but imo it addresses the largest concerns today.

That's if you look at major PUBLICIZED attacks on TLS endpoints. It's quite plausible that the people who've found (i.e. are looking for) attacks based on incorrect crypto aren't publicizing them.

Sure, but there's no evidence of that.

No evidence? We know for a fact that US, Russian, Chinese, British, Israeli etc. intelligence agencies are looking for crypto vulnerabilities, and we know for a fact that they do not publicize the vulnerabilities they find.

Yes, I'm aware of many, many people looking for crypto vulnerabilities. I'm not aware of many exploits in the wild.

CompCert is a verified compiler – as in the compiler itself won't silently miscompile your code. It has nothing to do with verifying your C code. For that, other tools can be used (e.g. Frama-C Wp).

One verified TLS implementation is miTLS, written in F*.

Indeed — unless it's built from source by a compiler I can trust (and also build from source), what security guarantees may it offer?

Having a SPARK-verified version would of course be great anyway.

Does it allow building as a completely shared library? If not, everything using it will have to be rebuilt from scratch on each update, which is highly annoying and a security issue on its own IMO. Apparently it has a Rust API, so it's probably unlikely.

I prefer to use openssl for this exact reason. Having to wait for everything to finish rebuilding every time a vulnerability is found is risky when you have public-facing services.

It’s rust

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact