Hacker News new | past | comments | ask | show | jobs | submit login
Prepare for the OpenSSL 3.x Secvuln (xeiaso.net)
70 points by todsacerdoti on Oct 29, 2022 | hide | past | favorite | 18 comments



Isn't it just lovely how we've moved so many things to static linking and containers? Now what should be a simple `apt upgrade` and maybe a reboot to ensure everything loads the new version is going to instead involve checking every binary (and its dependencies) for statically linked OpenSSL, check if the maintainer has committed to releasing a new version after the fix and if not, getting a toolchain ready to recompile when it drops. Containers will also need to be rebuilt from source or hot-updated.


"Simple", then "apt upgrade" and "reboot" inside the same statement makes me believe that you're thinking small, like "my own server with my own self-written PHP application". Which is not wrong, but that's not how tech scales.

How would you apply that procedure to 1,000s of servers, in the right order, to prevent downtime? How much would it take?

Won't it be far better to create a new container image, run all the testing on it, and deliver it everywhere with the already-existing upgrade procedure for your cluster?


This sounds like container kool-aid to me. One can implement exactly the same processes using regular OS management tools as with container tools. They both still need the same information about order, testing, etc.

If you’ve only ever put the effort towards doing that in a container instead of at the OS level, then obviously your container procedure will be better because you already built it. Someone who already built it at the OS level will also have similar ease. (And those containers still need to run somewhere, and that somewhere invariably has an OS involved, so you always need the OS update process no matter what your container strategy looks like).

But you’re also thinking “small” in the sense that you’re only focused on your own app. I find it striking how so many developers these days have simply forgotten that they’re not the only ones who actually do tech stuff, and how limited their views are. Running IT at a company involves hundreds of applications that are not built by you. Just because they’re not the main thing you sell doesn’t mean you can ignore that they exist.


"Entering" a server and running "apt dist-upgrade" probably means you've got a stateful fleet, because some package upgrade can fail, it can be performed at different time from one server to another, et cetera. Your servers should be cattle, not pet. So, you should instead create a sort of manifest/snapshot/checkpoint with all your deps pinned, and distribute that to your servers.

You can do the same without containerization - you could use a fleet of machines and keep their versions pinned with a configuration management tool, but you'd still not want to manually perform an apt upgrade on your server. And manually "check that everything is fine and maybe reboot" to verify your new shared lib was loaded - you should already have a precise, reproducible, automated testing and upgrade procedure and just apply that.

So, containerization is not the point here. Automation of deployment and testing procedures is. But the OP was specifically attacking containerization (as if it were the problem), and I was answering him.

But, containerization is generally useful here, because the "container image" is, generally speaking, a good, well-defined artifact. So the OP's example was especially bad - telling "machine X is vulnerable to SSL bug Y" is hard, requires specific machine instance inspection: what happens if the process/server wasn't restarted and, even though package is updated, it's still using an old ssl version?

On the contrary, "this is using container image version X with openssl version Y which is not vulnerable" is a very precise True/False statement, easy to inspect and verify at scale.


I'm not thinking 1000s of servers, because that is extremely rare, but this definitely doesn't apply to only non-critical private stuff. Many orgs have up to a dozen servers, each running multiple services. If you have multiple replicas of certain servers, you can reboot them (or just restart the impacted services) one at a time with no disruption.

The thing is, you usually aren't building everything from scratch. A pretty typical containerized web app deployment might consist of postgres, redis, elasticsearch, nginx, task queue and app server containers. Each of these needs to be updated and you're probably only building two of them from Dockerfiles. Instead of just your host distro, you're now waiting on the maintainers of all of these images to push updates that you can pull. Big distros get updates early so they can push them out the instant the embargo lifts, but I doubt all of these Docker packagers and statically-built software maintainers do. You're either stuck waiting or you have to build all of them from scratch.


If you’re a small org with just a dozen servers and you’re overcomplicating your life with hundreds of containers, then I agree with you. But you can’t attack containerization if it’s used by proper orgs with the right scale.


OP here, most of my advice later in the article is aimed at people running smaller services on smaller scales. I'm (probably poorly) assuming that people at larger scales know what they are doing.


For OP I meant franga2000 btw. Your article is good IMHO.



That's a holiday in Germany. So "prepare" means we can take a day off our day off to patch OpenSSL on all systems? Obviously the OpenSSL team can't take all holidays in all countries in account but I still find it strange that the security problem is hinted at since like a week until there's finally the great day where the veil is lifted.


It's a holiday only in five of 16 states in Germany.

It's been hinted at for a week so that even people who are on holiday, and would not ordinarily read their news that day, have an opportunity to check how this - in their eyes critical - vulnerability affects them.


All Saints Day is quite a widespread holiday, at least in Europe. I found the date pretty curious as well.


Can we not just dump OpenSSL for once and move over to LibreSSL already!?


Does anyone know what whether the default apt repositories for Ubuntu 22.04 will be updated right away?


This is a pretty coordinated release, I think the ubuntu security team probably has packages ready to be released. The security repository should have the update right away.


Yep. Normally the security embargo for these things covers a large range of distributions and packaging organizations; Ubuntu should be one of them.


Thank you both!


Had quite a few packages updated on my Ubuntu servers today and believed it was this issue. But seeing this is still embargoed it looks like these weren't updates for this problem? Or is Ubuntu releasing binaries but keeping the sources embargoed?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: