I think GnuTLS is probably the second most popular TLS library, after openssl.
Though actually, maybe, firefox's NSS is now that I think about it a second more. Firefox is at least some fraction of C-library-based ssl traffic out there.
> LibreSSL is the drop-in replacement for openssl that everybody should have embraced back when heartbleed happened.
100% and 1000x times this. LibreSSL does have some minor incompatibilities, but those were mostly about removing obscure and very rarely useful APIs that were badly designed in the first place. It was quite likely that if your program used these APIs, it had security issues that OpenSSL couldn't solve with an update.
(Usability is security. If it's hard to use correctly, then it's easy to create a security hole.)
Remember folks, LibreSSL comes from the same people who made OpenSSH, and that "other" OS that had 2 remote holes in almost 30 years. You already trust them, you maybe just don't know it.
> but a year or 2 ago the went back to OpenSSL for some reason which I did not fully understand.
That would be Void Linux [1]. One of the reasons iirc, is PEP 644 [2], in which CPython drops support for LibreSSL due to it not being fully compatible with OpenSSL 1.1.1 APIs.
That was either Void Linux (see sibling comment) or Alpine Linux.
You might also have read Gentoo Linux announcement (https://www.gentoo.org/support/news-items/2021-01-05-libress... ); technically doesn't fit your description since Gentoo never "switched" or "went back", but rather supported them simultaneously, and just pulled support for LibreSSL at some point.
OpenSSL is very much part of the problem set here with a long history of dubious API choices, a performance sapping rewrite in 3.0, and many things retained for far too long like VMS support.
The problem is that killing it requires a lot of buy-in and dealing with FIPS 140-2 and no one was willing to do that last time.
The problem is that if you want Red Hat or Canonical to support integrating your new library across all the applications that need it, it has to support FIPS. If it doesn't people who need FIPS and have the money to support maintenance will avoid integrating it.
Crypto is hard, and crypto in C is fucking awful. Hence the pricing.