The most frustrating part of Linux for me is all the damn mental CPU cycle I have to spend keeping up with the naming and positioning of the various distros. Often it borders on drama.
The other day I commented that I took my Pinebook Pro off the toy shelf and loaded up Manjaro. I commented on HN about how much I liked the user experience. The first reply I got was something about the drama with some dude named Phil. Alternatively I’ll put my head down for a while and forget about the lineage of the various options and then something will crop up about this or that distro is going away, being repositioned or whatnot.
Why does this matter? Because I want to use it for real work and I don’t want to spend time planning a long term OS strategy. There are other things to worry about. I’m far from alone in this. That’s why folks reach for Mac OS and Windows.
Downvote away, but I really do want to love Linux. I just don’t want it to be so hard to love.
I've been using Debian or a Debian derivative for over 15 years. At first, I wondered if it was a good choice because RedHat and SuSE were the "professional" distos. Debian was different and when Ubuntu came out, it was like Debian, but with some useful closed-source stuff. There's been drama, but it's not nearly so bad as some of the evolution that MacOS and Windows users have had to deal with. The security improvements on MacOS have really made doing simple things like a Zoom conference painful while Windows userspace is devolving to something that reminds me of Linux in 2005 or so where every window had it's own UI style and the userspace was a mess of competing ideas and ecosystems.
Meanwhile over on Debian/Ubuntu/PopOS/Plasma, I can finally run the latest games (thank you Steam & Proton!), edit video with pro grade tools (thank you DaVinci) and I still have a great coding environment... Oh, and thanks to cloud based office software, that's no longer an issue.
Ubuntu is seriously an amazing achievement and while I've used plenty of distros, nothing compares in terms of getting work done. I'm not an OS engineer and have no desire to get granular with my OS, I just want to be able to load it up and do stuff. The linux community, however, is loaded with OS-enthusiasts who don't understand the difference between using a tool and fetishizing it.
Recently I’ve been using Linux more, as I used to. Have to say Ubuntu remains the most polished and useful of the distro’s overall. Just works most of the time.
Maybe, but Ubuntu does have drama like the Amazon search link thing and now uncontrollable auto-updating snap packages, which arguably seem like an absolutely terrible idea for any sort of server.
For almost everything, Debian works just as well as Ubuntu, but it's more boring and has less drama.
This sounds refreshing. I've been trying to convert from Mac to Linux recently and after having tried a couple of distros like Pop and Neon settled on fedora, mostly because of all the modern stuff in there (GNOME on Wayland with zoom supporting it OOTB, btrfs). After having used it for a week intensively, I have experience way too many issues (zoom freezing other windows, firefox crashing when resizing the window and DevTools open where I was thinking is it the React website that I'm just building?, Wifi dropping in the middle of a zoom session and not auto-reconnecting) let alone the different keyboard mapping which I have found too limiting (as everything is done with Ctrl when there are so many other meta keys and Ctrl is just the worst from a positioning perspective as I have to use my little finger instead of thumb like on macOS. Also, fedora asks for too many OS reboots and I don't like to restart the full dev environment all the time.
So far, I have converted back to macOS, which is equally bad from UX perspective, especially the hardened security - I just don't want to opt into every program getting the permission to run, send notifications, access this and that. It's just too much.
I think I will give one of the more non-corporate Linux distros a try. Or, maybe it's Pop, but ideally it should be Debian based, just not with the "outdated" experience.
The problem was not Fedora per se, but GNOME and especially Wayland. Wayland is slightly above vaporware and infested with bugs.
Had you chosen Fedora with KDE you'd probably have 0 problems. I'm using it without any drama for 11 years and I keep suggesting it to co-workers and friends.
Wayland has done so much harm in the Linux ecosystem and that gives me a lot of grief :(
It's a pity that it really is that bad. It didn't have any glitches during normal usage but it quickly fell apart when challenged, obviously. So it's probably still too early and we need more distros picking Wayland up and help stabilise.
What didn't help was probably the desire to force myself into liking GNOME, when normally, I fall naturally into the KDE camp, because it is beautiful without having to make things too simple, like GNOME does. Even though I agree with many of the things GNOME stands for, there is lots of stuff you wouldn't NEED to customise, or use.
The GNOME Xorg session is still available if you need to use it.
It would be nice if it were possible to avoid the compatibility issues with Wayland, but the design of X (and the reliance of applications on X11-specific APIs) is what makes this not really feasible.
Those some things are things, you shouldn't do in the first place, like snooping on windows and events that do not belong to your app or injecting events that end up being processed by other apps.
I suspect they're referring to the fact that after the Zoom app is updated, you may find yourself in a meeting but unable to share a window because the updated app no longer has the "Screen Recording" privacy entitlement. Turning that on again requires quitting the app and therefore leaving the meeting.
This is exactly the issue. It's especially problematic when you are hosting a private meeting. When you exit, it will hang up on everyone, unless you hand control of the conference to a participant.
That does sound annoying. Though to be fair, Apple has plenty of reason to distrust Zoom updates. I'm a bit surprised there's no way for Zoom to re-try screen recording without re-launching the app. Is that a fundamental limitation of Apple's security model, or a failure of Zoom?
It's also all-app: they can change their homemade updater to a normal non-hidden updater and have normal signed distribution that doesn't require any of this re-auth.
Not a strange claim. Having to exit the conference to give the Zoom app permission to share the screen is painful, and if the Zoom app updates, you have to do it again. That means you have to leave the conference to permission the app, and if you are the host, hand off control to a participant so the conference is not ended early.
> and I don’t want to spend time planning a long term OS strategy. [...]
> That’s why folks reach for Mac OS ...
A conservative Linux distro choice is way more stable than Mac OS is in regards to "sudden" changes you need to deal with. And on the time scales we worry about CentOS or Debian, Windows has its pitfalls too.
In all cases, if you need a long-term strategy, it's always going to involve some level of preparedness to handle a change.
If you want to keep up with all the distros and their individual going-ons, that's your choice. You can use Linux just fine without doing so.
EDIT-that's-too-late-for-an-edit: To make the Windows example specific, there's a lot of companies that built their products on Windows 7 Embedded that would love to be on CentOS this week instead.
You seem to be massively overcomplicating things. If you don't want to worry about the "distro drama", then don't. Just use one of the major distros (e.g. Debian or Ubuntu) and move on.
CentOS has been around for a decade and a half. It hardly qualifies as drama when your OS fails to remain completely stable and dependable for more than a decade and a half.
With the number of changes that have happened with Apple's, Microsoft's and Google's various OSes over that period it's hard to say that GNU/Linux isn't being held to a higher standard here. CentOS being the trademark-stripped knockoff of a corporate OS is the reason for the instability - when you deal in corporate OSes, you're subject to the business strategies of their sponsors. Some would say (weasel words) that the strategies of the sponsor of CentOS have been the source of instability and conflict across Linuxen for a long time now.
You're not wrong. I've maintained a mission critical CentOS server for the better part of a decade and it has been quite stable and dependable, despite their embrace of systemd which broke stuff the system required. I think I only had to do a manual reboot three times during that point, which is good, because it is on a remote mountaintop with access to the range controlled by armed guards.
Still, you seem to be missing the larger point. Suddenly, CentOS is dead. Maybe I'm an outlier, but that seems like a disruption in its stability and dependability.
> If you don't want to worry about the "distro drama", then don't. Just use one of the major distros (e.g. Debian or Ubuntu) and move on.
+1 on Debian. It's as drama-free as it gets. I mean, the distro is criticized for being too boring and too stable and too rock-solid and too unsurprising. Those descriptions are also used to describe power lines and sewer lines, i.e., reliable infrastructure.
Systemd in general makes me sad. It feels like the complete rewrite that happens only for the people doing to finally learn all the project needs at the end, and immediately need another rewrite, but nobody has the stomach for it now.
Don't get me wrong, I think the authors are and we're probably the best authorities on what is and was needed in an init system, but the problem space was so poorly documented and understood because of all the crazy custom she'll init scripts people were using that they just kept having to tack on weird directives in piecemeal ways, until were left with what we have now.
What the world needs is for the systemd authors to take all that knowledge an apply it to a new init system in a sane way, but nobody had the will to go through that again, most likely especially them.
Who is asking for a rewrite of systemd? From what I have seen, there are two camps: Either you believe systemd is fine, or you believe sysvinit never should have been replaced or should have been replaced with something totally different than systemd with a much smaller scope.
I don't think anyone really believes that. They just believe the benefits it provides over sysvinit are real and generally worthwhile in most cases. Systemd has many warts. I'm not sure anyone refuses to believe that.
> or you believe sysvinit never should have been replaced
Yep, there's plenty of people that believe that. I've had discussions with them here.
> or should have been replaced with something totally different than systemd with a much smaller scope.
I'm sure there are people that believe this. Personally I think it's mostly due to being unaware of some facets of the situation. Systemd itself was much smaller in scope originally. The scope increased as they ran into things that it really made more sense to combine than leave separate. A different project may have chosen to leave those things out when encountered, but then it would have all the problems systemd tried to avoid.
That actually aligns quite a bit with my argument. With time and foresight (which wasn't necessarily possible the first time, since this is the first time an init system has tried to accomplish these goals that wasn't already intended for a tightly coupled system like Windows or MacOS), they could plan:
1) A more sane unit file syntax that wasn't ambiguous and confusing because of accreted features (After vs Wants, etc).
2) A sane API for subsumed services so if someone doesn't want to use the reference version provided by systemd, it's straightforward to drop in something else as long as you an make it talk systemd (or provide an interop layer). E.g. DHCP client, login managers, etc. This may or may not be done to some acceptable level currently, I'm not sure (but I suspect it depends on the service).
3) Take advantage of even more new kernel features for additional benefits (it's eBPF all the way down).
But here's the thing, while I think everyone would agree that something like systemd that's more modular in a well defined way and more sanely configured and documented would be preferable to systemd (I mean why not, it's basically "systemd but better" which is preferably even if you don't like systemd), I don't think anyone really wants to go through the process of either developing, integrating, or learning a new system at this point. We're all too exhausted.
And that's a shame, because what we all have now is the mad scientist's awesome flying car. Sure, it does amazing things, and you can do things that would be near impossible in the past, but you control part of it with your thighs and elbows and there are two steering wheels, and the one on the ceiling is the one that steers it like a car.
> hich wasn't necessarily possible the first time, since this is the first time an init system has tried to accomplish these goals that wasn't already intended for a tightly coupled system like Windows or MacOS
If we ignore similar systems on big iron UNIX and mainframes.
Are you making a case that big iron UNIX systems and mainframes had something with as many features and that tried to accomplish something similar in score to systemd (that is, far beyond what sysvinit does)?
I'm happy to hear of them, because I'm ignorant of them if they exist.
I was specifically comparing it not just to sysvinit replacements, of which there are many, but trying to bring a lot more features.
Launchd doesn't count, I already mentioned MacOS specifically to account for it. :)
SMF has some features beyong sysvinit that systemd also provides, but I'm not familiar enough with it's advanced usage to know how much beyond the obvious ones (service restart, etc) it does, so I'll try to take a look.
I don't know anything about the IBM one, so that's worth taking a look at. Thanks!
Personally I do think it's fine and I don't think that a technology having warts (which is essentially a given for any reasonably complex software) indicates a need for a rewrite. The rewrite will have warts too, just like sysvinit had warts.
It's not that it has warts, it's that it's overly complex for what it does, and beyond the initial feature set wasn't so much planned as organically grown.
I think the best and most thorough explanation of this is gotten through reading the systemd retrospective[1] that someone posted earlier this year. What becomes abundantly clear when reading it is what I think I was trying to express above, that there is so much that could be better in systemd but that it's hampered by it's own legacy baggage, which accumulated very quickly and very soon after the first implementation was released.
That link is a long read, but if you have any time or effort invested into systemd and systems that use it, and find yourself having to create unit files or diagnose weirdness with services, it's very worthwhile to read if you haven't already.
The systemd drama is redhat drama. The pulseaudio drama was redhat drama. The glibc drama was redhat drama. This is redhat drama. It's not all redhat drama, sometimes it's Microsoft drama or Gnome drama, but it's funny: all of the sources have really good relationships with each other and commingle their drama until it's nearly indistinguishable. Get involved with one of them and you now have dependencies on all of them.
The specific systemd drama the parent is referring to was really Debian drama and the requirements put on maintainers to support alternative init systems. If upstream drops support for non-systemd setups do the maintainers have patch in support? If upstream now has a hard dependency on systemd do maintainers have to patch it out?
Red Hat wasn't involved in any of the drama outside of being the ones writing and funding OSS software for themselves.
Redhat was involved because they heavily maintain some of the upstream packages that put in a hard systemd/logind requirement (GNOME), thus forcing Debian into the uncomfortable position of taking on a large compatibility project, when upstream could have instead taken small patches to continue to maintain compatibility with non-systemd systems.
Which packages had a hard systemd/logind requirement upstream? As far as I understand it, GNOME still gets upstream patches from BSD developers and is still supported there. (Some Debian packages do have hard dependencies on systemd, but this is a downstream packaging issue)
Interesting, I checked and the only upstream component of GNOME I could find that requires systemd and doesn't have a compile flag for it is gdm. This isn't required to start GNOME (you can use any other display manager) and FreeBSD/OpenBSD seem to be maintaining consolekit patches for it downstream. So I see why there is little incentive to fix this upstream. (On non-systemd Linux distros the focus has shifted to staying compatible with elogind, but that doesn't work on BSD)
More like "oh hey encrypted LVM doesn't work and the package maintainer is refusing to allow fixes because he hates systemd and is indulging in maintainer terrorism to try and block it."
Yeah true, the distro i had the longest time was arch and debian, then i changed to FreeBSD (because i can and i want), but on my laptop i need additionally a Linux...but installing arch when my main interests is *BSD..no, so i installed openSUSE tumble and that's it. The whole distro bs is just plain stupid, use what works and has a stable past...that's it.
> The whole distro bs is just plain stupid, use what works and has a stable past...that's it.
That's absolutely a reasonable point of view -- for your use case.
From the perspective of a variety of folks, including corporate IT, makers/vendors of drop-in appliances and other folks, the changes to the CentOS product will be disruptive.
This is because production servers and appliances generally need to run, for extended periods (measured in years) the application set specifically designed and implemented for the particular set of software versions on a particular OS/distro.
For many of the above folks, CentOS fit that bill nicely, as it reliably tracked (that is, was slightly behind) a commercial (i.e., you need to purchase a support contract to access updates and support from the vendor) Linux distro (Red Hat Enterprise Linux -- RHEL), but did not require any support contracts or fees above those for hardware, installation and administration.
The problems for the above implementation/management/administration strategy comes out of changes to the CentOS long term support policy.
CentOS (with versions 6 and 7) provide(d) a ten year support window for those versions and promised the same for CentOS 8 (released in 2019).
Many of those who had chosen CentOS for it's free (as in gratis[0]) access to the commercial RHEL platform took CentOS (whose developers/maintainers were hired by RedHat[1] to continue that work in 2014) at its word that support for CentOS 8 would also be supported for that same ten year span, updated their environments and applications to the latest version.
That promise of support has now been rescinded, as CentOS will now be the gamma/release candidate version of RHEL and will lead RHEL slightly rather than trail it.
AIUI, there are many reasons for this change to CentOS development, the biggest being that Fedora (the community project that RHEL used as its Beta platform) is now so far ahead (via its rolling release schedule -- for example the RHEL/CentOS 8 kernel is v4.18 vs. Fedora 33 kernel v. 5.9 -- more than two years) that it's no longer providing that utility.
I absolutely understand why some folks are upset, but claims that "CentOS is dead" and the like are rather hyperbolic in my estimation.
Disclosure: I have no financial interest in IBM or Red Hat, nor am I a user of CentOS
If you are running corporate IT on CentOS to save a few bucks over RHEL, any issues are 100 % on the maker of that decision. Because it's a dumb decision.
Licensing itself is a pain in the ass. We pay low teens for Oracle Java licenses. Every year we have to audit hundreds of thousands of machines to see if it is present. When it is, figure out if it is a public version or something that requires an extended support contract. There are things like Adopt OpenJDK that don't require any licenses. Having an OS where you did not need to worry about the licensing was huge - nobody cares what you did in a docker image, on a VM, or some experiment.
I think we were shelling out around $300 a RHEL license, which was a significant percentage of the docker boxes we started putting together with some low end hardware.
The point being that there is a trade-off to be made between paying for an OS instance and having certainty of support, versus community distros with no upfront sticker price but might cost more in man-hours when things fall apart.
Anything run-the-business should have support. Anything used by developers for R&D / messing about can be unsupported.
If companies want something to be reliable and accountable then paying for RHEL seems like the obvious thing to do. They are able to pay for Windows but why not for RHEL. The current change is not unexpected given their acquisition.
It's great; I've used it for over a decade.
These days it's only on my servers, though, due to lack of time. I'd very much like something along the lines of Sabayon with OpenRC for my laptop (currently using Fedora/KDE).
I don't think Mac OS and Windows are freer from long term maintenence, whether it's Apple doing stuff like phasing out APIs for OpenGL, migrating to M1, Gatekeeper requirements and noose-tightening, or whatever they'll expect you to keep up with next year.
Likewise on the Windows side, the SKUs vary from release to release, group policies come and go, and generally they've increased their push into seeing Windows as a way to promote their other services and don't really care what you want from it.
That's an interesting perspective, I've found the opposite. I distro shopped a lot around ~2006, settled on Ubuntu then moved to Debian for project philosophy reasons. It's great because I haven't had to think about "how do I install this again?". Same web page, same link to netinstall image, same prompts to install.
It's fun to try other things like NixOS and GuixSD, but also have the warm blanket of "just put Debian on it" to fall back to.
Your complaints would make a great infomercial setup.
"Are you tired of your Linux distro being bought out and crushed by IBM exactly one time in history?"
Cut to a big dopey confused person shrugging.
"Well then try Apple!"
Dopey person smiles.
Star wipe.
Microsoft turned down an offer to bid on Red Hat, whose enterprise focus is a much better fit for them, before the sale to IBM so there's no reason to believe they would bid for Canonical.
Wow. That sounds surprisingly probable, now that you bring it up. Ubuntu is wildly popular, but requires a lot of money. Microsoft has a lot of money and seems to be trending toward open source.
> The first reply I got was something about the drama with some dude named Phil.
At least for Windows, can't you just quickly compare the ethical universes? I'm assuming that Phil didn't embrace-extend-extinguish other distros, Phil didn't secretly change Skype to break the encryption for LE, and Phil didn't upgrade the distro to do constant, inescapable telemetry and update arbitrarily downloading Gigs of data such that it interrupted a user's live broadcast.
In other words, if your decision is between Phil's distro and Windows, Phil's distro wins so it's the end of the story. And if the HN trolls you're responding to aren't claiming a 1:1 feature correspondence with "ethical" distro vs. Manjaro, who cares?
Is there a story of someone actually getting sucked into doing, say, Debian maintenance within the past five years?
I mean, they have a command line program to report a bug. That and their monstrously complicated packaging apparatus would make me think someone at Debian considered the sucking-in of people to be unethical and thus designed their maintenance polices to prevent it.
.debs are gzip'd tar file with predictable top-level folders and some manifest files. An APT server can be as simple as an FTP site, again with a predictable hierarchy and some manifest files.
Keysigning gets a bit messy but it is still relatively simple.
You can set up your own private package repository in minutes using nothing more than ftpd, gpg, mkdir, and vim.
The documentation is clear, comprehensive, and relatively easy to read.
Try to build deb package according to packaging guidelines.
If you are new to debian packaging, make a reservation in your calendar for several evenings, just to figure out what exactly you have to read through.
Did that. Ran an internal APT server for years. Was a fantastic system for doing remote installs on field installations.
Like, you create a folder structure that mimics the final installation inside one of the two toplevel deb folders. The other folder had a manifest or two. Worked easy, and did not take long to figure out
In that src file are things like "usr/whatver", "opt/whatever", "etc/whatever" and "DEBIAN"
In DEBIAN is a really simply control file and sometimes some scripts to do things on install.
Getting a package approved by the debian maintainers is an issue if you don't follow the guidelines (and lintian will help) - just like getting a package approved by the redhat maintainers, but to make your own deb is barely harder than creating a tarball.
That's why I qualified it "according to packaging guidelines".
And I don't want to comply with packaging guidelines because I want to get it into the upstream repo; no, I want it compliant so it is not broken at upgrade time and I don't want to have to solve several problems at once at the most inconvenient time.
With rpm it is both easy and piecemeal - you don't have to have the entire knowledge in your head upright, you can improve it continuosly. You start with spec file and rpmbuild, then find out about rpmlint, then you will find out about mock (so you can build package for Fedora-any version and CentOS-anyversion from any distribution, including Ubuntu or Debian, inside chroot or container) to koji (full blown distro-scale build system).
There's a difference between software evolving and drama. I'm trying to remember an upgrade to Ubuntu or Debian that led to as much manual intervention as the latest update to MacOS or Windows 10.
The only real time I can think of where they caused any impact to the end users would probably be when they changed up a bunch of software because of licensing issues -around 2001 or so. That also led to them writing or re-writing a lot of software and gave us pf, too. So I'd call it a net gain, personally speaking.
On the linux front -libc6 and the elf change-over caused a lot of breakage; like you said -I'm not sure if I would call that "drama" or not.
Perhaps the difficulty here is that different people are using the same words to mean different things? I would have said that drama and controversy in this case were nearly synonyms, but it sounds like you're using drama to talk about instability in the product instead?
Controversy is a less emotionally loaded term for the same thing (drama/dramatic, toxicity/toxic, harmful/harming). Because of that, it receives more credit in a logic argument.
You are on about direct user impact, where a project discontinuing is perhaps an ultimate one (EOL being moved 8 years in advance is almost that).
Regarding OpenBSD, that isn't the only occurrence. You can grep mailing list archives for some keywords. Look at how people like Daniel Hartmeier, Niels Provos, and Wim Vandeputte got treated. Sure, it did not cause OpenBSD to collapse but I would say it was controversy.
You must have missed the FreeBSD code of conduct change 2 years ago and all of the port maintainers who left as a result. I’m not even a FreeBSD user but heard about it.
> Downvote away, but I really do want to love Linux. I just don’t want it to be so hard to love.
Folks don't go for proprietary systems like Windows and MacOS because of theatrics around distributions, they just choose one of those stolid long-time favourites like Debian, Ubuntu or Mint and ignore the squabbling. Those who want to get a more hands-on experience might go for Arch, Slackware or even LFS.
I think the point was that CentOS was one of those distros maintained by people who had a life... Maybe Cannonical won't be around in the future either.
Minor distros fit specific use-cases better but are more likely to disappear, major distros are here to stay. That's the tradeoff.
But in the the Microsoft world, you at some point had Windows Mobile and Windows phone, and developing apps for those is the true meaning of being alone.
My snap folder only has chromium in it. Like probably most users who don't (and shouldn't) care about snaps.
The snap drama is a ideological purity contest by people who should know better (aka use something else than Ubuntu if that pisses them off that much).
Snap drama is just writing comments about it without action.
It's not like OSS community would have rushed to provide alternative packages for Flatpak, AppImage, deb etc.
Snap automatic updates are still the fastest way to update servers with latest security etc updates, keeping servers secure. That's the reason usage of Snap packages is increasing.
With Docker, recent drama was Docker Hub rate limits, so I had to move Docker base images from Docker Hub to Quay.io, so that Docker images could be built successfully.
Are you like this with cars too? “Gosh there’s just so many brands and models to choose from, I wish there was just one choice so I didn’t have to think or make a decision”
With cars, they all do one thing good: drive. The other features are just that: features. With Windows vs macOS vs Linux, you’re dealing with software incompatibility and having to find replacements.
You don't have to evaluate every possible option before picking one any more than you have to read about every make and model of car or laptop but you probably ought to read about the one that you pick and manjaro is run by skeevy unprofessional people.
> Downvote away, but I really do want to love Linux.
This would be the problem; you treat Linux as a platform that one is supposed to love or not love as a hole.
Linux is not a platform; it s a component shared between a great many different platforms that are often quite far apart from one another.
These platforms often also share other things, or not, and don't Windows and FreeBSD also share several components?
I've noticed a common belief that completely unrelated, independently started and operated platforms have an obligation to coöperate because they share a single component, which most of them also heavily patch and modify.
> Recently I moved to Apple Macbook Pro 2019 (FYI before that I was using Linux Mint for years without any drama). But after using this very expensive MBP, I am not happy at all. Hardware is the best in class while keyboard is the worst in the class. But my biggest complain is keyboard layout. It is not ergonomic at all. In windows/linux, WINDOW key manages window manager while CTRL, ALT can be used to manage applications. On internet, many places it is mentinoed that, use apple COMMAND key as control key and OPTION key as alt key. Ok thats fine. But this behaviour is not consistent at all across apps. Many apps use CTRL key to control application additional to COMMAND key(e.g. in chrome/vscode, toggling tabs can be done using CTRL+TAB, not COMMAND+TAB). And this CTRL key is only one side of spacebar, very very unergonomic. Additionaly if you are a touch typer powered with mechanical keyboard, you can press CTRL key on windows layout using wrist. This is not possible on Mac as its first left key is FN.
After using MBP for couple of weeks, my hand literally hurt like mentioned in video. I dont think I can operate MBP for hours without looking at keyboard. In Windows you can do everything with mouse only. In Linux you can do everything with keyboard only. In mac, you need both and your hand hurts.
Disclaimer: I work for Red Hat but I'm here entirely with my own opinion
My God, it's been a long time (if ever) since I've seen so much misinformation out there. I mostly blame Red Hat for the very poor delivery, but tech people are not immune to the tendency for dramatizing, misrepresenting, and polarizing. Please stop spreading this. CentOS is NOT dead. CentOS is changing, but not all that much (yes, really).
I'll do my best to answer questions here, but I'm also going to rush a blog post to try and get it out ASAP that corrects a lot of this.
Edit: Here's the blog post. Excuse typos and other things, I rushed this ;-)
I appreciate the blog. I was pretty alarmed by the announcement, and I think as you mention a lot of it boiled down to that devilish-sounding line:
> If you are using CentOS Linux 8 in a production environment, and are concerned that CentOS Stream will not meet your needs, we encourage you to contact Red Hat about options
But, after reading your message I can partially see your point of view, in particular your gist seems to be that in the chain of development, CentOS Stream will sit where RHEL used to be, thus if you believed in RHEL as a boring, stable, reliable OS in the old world then so too should CentOS Stream be a boring, stable, reliable OS in the new one.
This logic makes sense, with one exception: this assumes that the standard for release quality remains unhindered pre/post swapping the position of RHEL / CentOS.
Before, Red Hat was incentivized to make sure RHEL releases were solid gold because paying customers were on the receiving end. CentOS releases were downstream of solid gold, making CentOS solid gold as well.
Now that the positions are reversed, what incentive does Red Hat have to make sure things are stable and low-risk before shipping to CentOS Stream? This is ultimately a matter of trust: if CentOS Stream gets a bad release, no paid enterprise server explodes. Quality no longer aligns with profit motive, a terrifying insight.
> This logic makes sense, with one exception: this assumes that the standard for release quality remains unhindered pre/post swapping the position of RHEL / CentOS.
There's another issue, which I haven't seen talked about so far. With CentOS downstream from RHEL, there were the Release Notes: before updating from CentOS 8.x to CentOS 8.y, you could read the CentOS release notes (and the voluminous RHEL release notes corresponding to it) to see if there was any breakage to expect for your use case. For the security updates within a release, you could follow https://lists.centos.org/pipermail/centos-announce/ (sadly non-working for CentOS 8) and/or the voluminous RHEL errata pages corresponding to it.
With Fedora, before every update I glance at https://admin.fedoraproject.org/updates/ to see if there is any known issue with the update I'm about to do; this has saved me more than once (for instance, there was a broken selinux-policy update which would relabel the whole /home not long ago). I recall Debian had something similar (which shows you any release-critical bug recently filed for the packages you're about to install). Will CentOS Stream have something like that? If I run "dnf update" and see it's about to update bash, is there anywhere I can look to see what changed, and whether someone complained about the update?
I got a response. It looks like this is not currently implemented, but some details are still being worked out so it's possible we could get something like that. A digest has been requested before and thought is going into what it would look like and what any technical hurdles would be.
I also learned a new command that I'm adding to my toolbox:
Thanks for the response, and thanks for reading. You make a terrific point:
> This logic makes sense, with one exception: this assumes that the standard for release quality remains unhindered pre/post swapping the position of RHEL / CentOS.
I don't really have an answer for that. It does seem like it would come down to "trust" and as a skeptical person I don't like that very much.
As I think about it more, while we have realigned some economic incentives for the better, it does seem (in addition to your points) there might also be an "incentive" to break CentOS from time to time, just to remind enterprises who are using it that "RHEL" is the gold standard and CentOS is "unsupported" D-: Knowing Red Hat and many of the volunteers that work on these projects, I would be shocked (and indeed would quit my job immediately) if such a thing were to happen, although it would be hard to prove.
That said such a thing would no doubt backfire. There would be outrage of course, and it would hurt the CentOS brand (which also hurts RHEL since these days the two are inextricably linked). We all fully expect another distro to "take the place" of old CentOS as a bug-for-bug rebuild (I'm excited about Rocky Linux personally) so not will there still be an option for that, but competition is great for quality of projects :-D
That ship has sailed. Even if what you say is true, the public reaction this is change has tarnished all the Brands, CentOS, RHEL, RedHat, and IBM...
It will be a very long time to rebuild the trust, and honestly if RedHat stays the course and refuses to support CentOS 8 for the full life cycle I think CentOS will usage (and RHEL usage) will drop off a cliff
I have already seen several software vendors with RHEL only official support announcing they are working on debian and ubuntu certification, and since it seems CentOS Stream at the most will only be supports for a 5 year life cycle down from 10, I am not sure why I would not move to debian at this point. debian is just as stable, the one thing CentOS had over it was the 10 year support, that is gone now
This still seems a little sensational. Fedora would be the explosion here. If Stream is a release candidate they still have every reason to keep it stable so RHEL updates more quickly.
That probably depends, how strongly Stream and RHEL actually are going to be linked. If this relation is indeed similar to current situation, but centos and RHEL changing places, then there isn't much point holding QA until RHEL release.
Actually, there is going to be good initiative: when you upstream is unusable, because too buggy and untested, then what? Whole point of Stream seems to be usable base to RHEL.
Cutting 8 years(I think? have something like 2029 in the back of my head) of support from the thing a bunch of people just moved to, with no publicized plan what they'll have to move to instead in the next months(?), doesn't sound like "not all that much" for the kind of environments I've seen CentOS in. What's the plan for them?
This is actually the big area where I think the criticisms are right and fair. Dropping 8 years was uncool.
Moving to Stream is an option immediately (see post[1] for more information about how CentOS is NOT unstable or "the new Fedora" or whatever else everybody is saying), but there are also forthcoming announcements about new free RHEL subs for individuals and edu institutions and the like. I don't know anything about those. I personally think it was terrible to announce the CentOS changes without those to accompany, but hey if I ran things everything would be perfect :-D
> there are also forthcoming announcements about new free RHEL subs for individuals and edu institutions and the like.
This is the surprising part. It seems like customer service 101 to have those announcements ready when this announcement was made. By not having those details ready in a timely fashion, they inherently devalue those programs.
Thanks. Luckily not my personal problem (or if it is nobody has told me yet...), but knowing how much hassle goes into any major upgrade (even if it's smooth sailing technically) in places with overly conservative processes, I hope those paths and changes get clearly communicated.
Totally agree. This has the potential to be a nightmare of epic proportions. I strongly wish Red Hat hadn't done that.
At the same time if you take any piece of free (as in beer and speech) software and don't pay a vendor promising support, you are kind of taking a risk that whoever is doing work for you for free might decide to stop. In no way does that take Red Hat off the hook here, but it is something I consider.
I've got a non trivial migration myself to handle before next December. Depending on how well Stream goes in my testing (so far it's been flawless but I'm not totally done with testing yet) and depending on what RH announces next month (or two), I may be cursing their names next fall ;-)
For me, with a single production server, migrating to Ubuntu LTS on a new server seems a much less risky proposition to me than an in-place upgrade to either RHEL or Stream.
Forgive me, but if you're just building out a new server, wouldn't it be easier and less risky to stick with RHEL or Stream since you already know your workload runs fine?
You also already know all the packages you need and probably have config files all set up and everything. At a minimum I would think they would pretty equal as far risk and ease if you're rebuilding.
My workloads are mostly containerized at this point so the host OS is pretty insignificant overall. Perhaps you're in the same boat, although I'd still think it's at worst equal, at best easier to stick with RHEL flavored.
Canonical has not, to my knowledge, ever dropped 8 years of support off a release - even the non-paid ones! Redhat has. In some people's minds, that changed the risk assessment significantly; not "well it worked on this RHEL-family OS, so it should work on this other RHEL-family OS", but "I need to replace my OS - should I use one from the same vendor that just screwed me over, or should I switch to literally anything else?" Honestly, you've got people seriously suggesting Oracle Linux over anything from Redhat; do you have any idea how much trust you have to burn to get people to treat Oracle as a good option!?
The majority of my service is a compiled C++ server with html/js front-end, so it runs on practically anything. I do need to run ffmpeg/firefox/pulseaudio to record sessions, which has been problematic on Ubuntu in the past, but it seems to work reliably on Ubuntu 20. I also have many customers running my service on their Ubuntu servers, so it shouldn't be an issue.
From what I've seen, Ubuntu 20 is probably going to be simpler to set up than Centos 8, as (I think) it has many of the required packages such as ffmpeg without needing to add epel and various other third party repos. It also doesn't have selinux, which I generally need to turn off on Centos (after spending a lot of time trying unsuccessfully to get everything working properly with it).
My only issue with Ubuntu LTS is the shorter 5-year lifecycle.
Perhaps by this time next year there will be some alternative repo that we can simply switch to in Centos 8 so that it just continues as-is, without us needing to take the risk of upgrading everything to a completely new distro.
Yep, if this had been for 7, it would’ve been just “meh” and move on to Arch BTW or something. But I just reinstalled new servers to move to 8 and now that was a waste of time.
I'm sure other commenters will put it much better than I could, but for those people that use CentOS for the purpose of a stable long term release, surely it is dead. The distribution now being called "CentOS" is almost at the opposite ends of the scale in terms of stability. The fact they've called this other thing the same name is purely marketing.
If you destroy something then create something entirely different with the same name then the fact remains that you still destroyed the original thing. That's not dramatising or hyperbole.
Rather than "stable long term release," I would phrase it as a "dot release frozen in time as much as possible modulo serious bugs and security fixes." Which is not the general direction that software is headed in. Changes going into CentOS Stream will have gone through QE tests. So, yes, it is different but, increasingly, most of the software you use will be using this type of model. Certainly SaaSs don't let you choose a dot release for the most part.
> Rather than "stable long term release," I would phrase it as a "dot release frozen in time as much as possible given serious bugs and security fixes."
Which is exactly what we want.
> So, yes, it is different but, increasingly, most of the software you use will be using this type of model. Certainly SaaSs don't let you choose a dot release for the most part.
Not all of us are happy with having to follow the update treadmill. Software is a tool, like an electric drill; one doesn't update their electric drill every day, otherwise we would be wasting all our time updating and dealing with the fallout of updates, instead of drilling holes in wood. If you develop (or do anything) on top of software, you want a stable base, instead of one shifting all the time; one builds castles in rock, not in quicksand.
Absolutely fair that you desire that, but it's not where the software world in general is headed and it's at least understandable that software companies are skating to where the puck is headed. (ADDED: Of course, with open source, you have no requirement to upgrade. It's just that folks want to make certain upgrades--like security fixes--and for serious bugs but no others.)
The nice thing about open source--rather than cloud SaaSs--is that there are more backward-looking options from a variety of sources. I'd also encourage users who need or feel they need that sort of option to contribute to such efforts so that they're sustainable.
> Which is not the general direction that software is headed in.
You could be right. But "we got rid of this thing (and gave something totally different the same name) because that's not the direction the software is heading" would be a different comment from the one I replied to, which was "this thing still exists, honest!"
> The distribution now being called "CentOS" is almost at the opposite ends of the scale in terms of stability.
No, this is totally wrong (which is why I'm trying to correct it). CentOS will be where RHEL used to be. It is trading places with RHEL. It's not "the opposite end of the scale in terms of stability." CentOS will be roughly a single minor point release ahead of RHEL.
It’s swapping places with RHEL beta, not RHEL. Not calling it a beta doesn’t not make it a beta. It’s directly taking the role that RHEL beta releases used to have (ie. be a point release ahead, and be for the public to test and report issues before going to stable RHEL).
Regardless of what your blog post says, RHEL was not a beta for old CentOS. Old CentOS was a rebuild/copy of RHEL, bugs and all included. RHEL will not be a copy/rebuild CentOS Stream, so it’s not a reversal of roles.
Using outdated packages doesn’t make a distribution stable. It’s stable when its combination of packages and configuration has already been tried by a bunch of people, and confirmed to work, or has had its issues thoroughly documented with effective workarounds.
I agree! Saying it's false repeatedly likewise doesn't make it false.
Where am I wrong? Can you point out my error and make a case for your position? What makes you think RHEL is still "stable" but suddenly a distro a tiny bit ahead is now wildly unstable?
Go be a child somewhere else. Your responses show a clear lack of understanding of the point of using CentOS and becoming petulant when someone calls you on it just makes it worse.
CentOS was binary compatible because the point was expecting it to be the same environment for application layer and testing. The change being described throws all that in the trash and its intellectually dishonest at best or just obtuse ignorance at worst to "misunderstand" the difference here, and how big of a change switching to a "Beta" effect release schedule is when there was an existing 10 year commitment to release schedule that was also thrown in the trash.
Don't try to insult the userbase by then gaslighting after the fact.
1) I obviously can't speak for anyone but myself here, but in my use case, which I think is pretty common, is to use CentOS for production deployments of various web apps and services where a person prefers the rock solid stability of RHEL without incurring the cost, and does not need support to go along with it.
2) I explain this in depth in this blog post which is up now ( https://freedomben.medium.com/centos-is-not-dead-please-stop... ), but the short version is that CentOS Stream is basically going where RHEL used to be. If RHEL was too bleeding edge for you, then CentOS Stream will be too, but otherwise it will be approximately the same as RHEL used to be. I consider RHEL and CentOS to be pretty equal, with CentOS lagging a bit behind. Now they will be pretty equal, with RHEL lagging a bit behind. Packages going to CentOS are packages that would have gone directly to RHEL previously.
> I consider RHEL and CentOS to be pretty equal, with CentOS lagging a bit behind. Now they will be pretty equal, with RHEL lagging a bit behind. Packages going to CentOS are packages that would have gone directly to RHEL previously.
There will be another small difference: the number of updates. Just for fun, I started a centos:8 image in podman, installed the package which switches it to centos streams, and ran an update (which by the way partially failed with an error message, IIRC it tried to change /proc in a way podman doesn't like). Then I looked at the version of one of the updated packages before and after the update (IIRC it was bash), and looked at its changelog. There were several releases between these two versions, and I would expect each one of them to appear separately as an update within CentOS Stream.
That is, packages that would have gone directly to RHEL previously will be going to CentOS, but not all packages going to CentOS will be going to RHEL; we will also be getting all the intermediate steps.
It's clear and literally self-described as a "development branch". To have anyone try to spin that as more stable and suitable for production is disingenuous.
> Before stuff goes to CentOS Stream it has passed RHEL QA and CI.
Do we have an official word on that? Because it feels like QA could just as easily be priority tasked on the next point release RHEL and starve stream of QA while it carries on ahead. I guess this boils down to the workflow which there doesnt seem to be a simple guide of with this news.
And no, I wouldn't agree that they just swapped places. CentOS stream is a rolling release. Unless I am mistaken that's not how RHEL is built right now.
I can see a reason for swapping them in that adding RHEL goodies/licencing into the free version is easier than removing, but the release cycle change, dropping CentOS 8 and as you admit, absolutely shambolic official press release, with lines that reek of a salesperson trying to squeeze out a bonus, do not inspire faith.
I see that you note in your profile that you work at Red Hat. May I ask in which department of Red Hat? The reason I ask that is, are you part of the decision making team which made this decision? If not, then no matter how good you think the intentions of Red Hat management here are, I can assure you that they haven't been made for the betterment of the CentOS community. If this was anything to do with the betterment of the community then this wouldn't have been a sudden announcement but instead would have been a long drawn discussion involving the community. I don't say this just for the sake of it, but I have closely followed how they killed the JBoss application server community and the project, as I note in my other comment[1] and this whole CentOS announcement is exactly on similar lines. I have known people in the application server team, who initially tried to justify the whole process, just like you are doing it now for this CentOS "migration", but even they have now realized there wasn't any truth to it.
It's not a co-incidence or an accident that the messaging around the community side of this CentOS announcement is confusing and unclear (they are letting the community members try and decipher what this all means for the CentOS project) instead it is intentional.
The bottomline, just like in that other case, is that the sales team at Red Hat cannot justify the hundreds of thousands of dollars they ask for supporting RHEL, when customers (or prospective customers) note that they can get the same quality from CentOS (stable release, regular bug fixes, community help).
The community is dead, the Enterprise part of the Operating system is dead
Sure the CentOS name will be still around, but is clear that CentOS is now a Dev focused operating system not designed or targeted for Stable Enterprise operations with a community of Enterprise SysAdmins around it
If you believe the changes are "not all the much" then you are clearly not in touch with the user base of CentOS, outside of Devs that is
The changes are GREAT for devs, and Appliance Manufacturers. Maybe even for Cloud customers
But for us Old School, OnPrem sysadmins running CentOS for Stability, and Long Term Support, CentOS is most certainly dead
> is clear that CentOS is now a Dev focused operating system not designed or targeted for Stable Enterprise operations with a community of Enterprise SysAdmins around it
This is absolutely not true. Would you have described RHEL this way? If not, then you shouldn't describe CentOS this way either. CentOS took RHEL's place in the chain. Instead of Cent being downstream of RHEL, the two were swapped around.
> If you believe the changes are "not all the much" then you are clearly not in touch with the user base of CentOS, outside of Devs that is
Say what you will. I've been using Cent since 2012 and currently run numerous workloads in production on it. I've shipped SaaS products on top of it and have been responsible for widespread maintenance. I've been a package maintainer for numerous packages as well.
> But for us Old School, OnPrem sysadmins running CentOS for Stability, and Long Term Support, CentOS is most certainly dead
I know plenty of old school on prem sysadmins who run CentOS for stability and LTS (I'm one of them. I have 3 bare metal machines in prod at the moment running CentOS). I may go with Rocky for one of those machines (it's a router/load balancer), I haven't fully decided yet.
That said I agree with you somewhat, that is the one area where this change could be somewhat negative. However, with CentOS being week and months behind RHEL right now, do you see that as a good thing? Do you like having an unpatched system for days or weeks at a time when an embargoed security fix hits RHEL and has to wait for CentOS to build and distribute? I don't. Security patches are top priority for me.
>>However, with CentOS being week and months behind RHEL right now, do you see that as a good thing? Do you like having an unpatched system for days or weeks at a time when an embargoed security fix hits RHEL and has to wait for CentOS to build and distribute? I don't. Security patches are top priority for me.
This is absolutely a False dilemma fallacy, there is absolutely nothing technically or legally prohibiting RedHat from releasing patches for both at the same time. Nothing. It is a sales/marketing or a internal process choice to do it that way.
Changes to do this process, even moving CentOS to be ahead of RHEL is not the problem. The "rolling release" or non-release, or what every model they are calling it now with CentOS Stream is the problem. The fact that it will no longer be Binary Complete with RHEL is the problem, the fact that CentOS will be used as a "beta" (and spare me the marketing BS claiming it is not that) is the problem
> Would you have described RHEL this way?
RHEL is a targeted, Released Enterprise Operating system with a 10 year support cycle
CentOS was a targeted, Released Enterprise Operating system with a 10 year support cycle
CentOS Stream is not a targeted, released Enterprise operating system with 10 year support cycle. CentOS Stream is pseudo-rolling release with no point releases, used to build a targeted, released enterprise operating system.
> RHEL is a targeted, Released Enterprise Operating system with a 10 year support cycle
> CentOS was a targeted, Released Enterprise Operating system with a 10 year support cycle
And you don't see the problem here? Why do you expect Red Hat to maintain a free OS which targets exactly the same use case as their paid OS? They are not a charity.
It would be great to have a free alternative to RHEL, but you shouldn't feel entitled to Red Hat providing that for you.
> Why do you expect Red Hat to maintain a free OS which targets exactly the same use case as their paid OS?
Because they said they would.
They could have made the change effective prior to releasing CentOS 8, or they could have made it effective for all versions after CentOS 8. Either of those paths would be understandable. Making the change after many have already migrated to 8 sends the message that Red Hat commitments are not to be trusted.
Well first and foremost because the software is open source released under the GPL so yes to any code that is released under the GPL I am entitled to it for free, that is the terms of the license and the fundamental promise of FOSS software
Secondly I do not believe I stated anywhere that I was entitled to it, this seems to be a strawman you created in some attempted to project your own beliefs of my motivations
In several comments I have stated my biggest issue with RetHat/CentOS announcement was not they were shifting focus, but rather that they did so while reducing their previously stated committed of supporting CentOS 8 until 2029, to 2021.
This is a betrayal of trust, and I do believe people are entitled to hold companies to account, and demand they honor their public commits of which RedHat previously committed to supporting CentOS 8 until 2029, and I feel they should honor that commitment. If they want to discontinue the support or shift the focus of the project for future releases then more power to them, the community will take the GPL code as we are entitled and create a replacement CentOS.
> CentOS will no longer be downstream of RHEL as it was previously. CentOS will now be upstream of the next RHEL minor release. This is NOT a “RHEL Beta”
That's... what beta means.
EDIT: Let me reframe it this way: If CentOS Stream is just becoming RHEL, then why does RHEL exist? If it's truly just as good as the old RHEL was, then explicitly merge them - either give RHEL away 100% freely like CentOS and charge for support, or discontinue the RHEL brand and sell support for CentOS. The fact that you have a downstream RHEL says that that is more stable, because that is what you're willing to sell for money.
EDIT2: Actually, thinking it over more and trying to be charitable, I can see a way for that to make sense: If you believe that CentOS is moving to RHEL's spot and RHEL is getting even more stable/slow/tested, then you can say that CentOS isn't getting any worse and ... sort of object to calling it RHEL beta. Except that it still is RHEL-beta, just moved up the scale of stablity.
Similar to forcing Microsoft or Google to prompt the user for their desired browser or search provider, it would be interesting if rhel was forced to prompt for a repo provider.
It would have been wiser to splash rhel support upgrades on CentOS gdm or cli cockpit than to threaten to kill a product.
Really, what were you thinking? Nazgul indeed! Fork you!
Oracle offered freedom from the perspective of rhel, and I used it.
I actually was granted IPO shares because of the things I wrote two decades ago.
I think there was a fundamental misjudgment in what you have done. Oracle is more expensive on most tiers, but offers a $100 support level (forum-only, with updates) that undercuts all rhel agreements.
Had you offered this support level, and advocated it (as I said) with gdm and CLI advertisements, you might have kept a great many in the fold.
You did not do this, and now Oracle updated their repo conversion for CentOS 8.
I don't work for Red Hat, but I saw a post that was saying that there is something in the works to fill the niche that CentOS offered. It just wasn't ready yet so the announcement hasn't been made yet. However, they needed to send a statement out regarding CentOS so companies were aware of what would be happening with CentOS 8 and perhaps avoid migration right now.
For a community that depends on LTS, the sysadmins seem to make some rash decisions/drama. I think the logical decision here would surely be to wait and see what happens and make a decision in a year when support ending would actually become relevant.
Many many many companies, right or wrong, wait until the very list minute to upgrade their legacy systems.
Many of us are either still in the process or worse just finish our migration from CentOS 6 to CentOS 8, only to have the rug pulled out from under us.
The sheer impact this move will have on many organizations is what is driving this emotional reaction
If they would have honored the LifeCycle for CentOS 8, there would not have been the reaction you have seen
100% of the emotion / drama is coming from the choice RedHat has made to cut the supported life of CentOS 8 by 7 years. that is a DRASTIC action, and their belief that 12 months is "enough time" shows a clear ignorance of how the product is used in enterprise or they fully understand and are banking on the desperation of enterprise to sell RHEL licenses.
Well, we mostly agree! I agree with everything until:
> their belief that 12 months is "enough time" shows a clear ignorance of how the product is used in enterprise or they fully understand and are banking on the desperation of enterprise to sell RHEL licenses.
There's another explanation, and I think it makes a little more sense than yours which would require Red Hat, literally the company whose core job it is (which they are damn good at, enough so that you use it) to provide an enterprise distribution to some of the most conservative companies on the planet, to not understand how the product is used in enterprise.
The suggestion that RH is "banking on the desperation of enterprise to sell RHEL licenses" at least makes sense given the facts, and that's kind of what I thought too when I first saw the news (the way they worded it sure made it seem like this was a money grab).
It is possible however (and indeed this is my belief) that Red Hat doesn't consider Stream to be that big of a change, and they've given a full year to migrate. Indeed if you read my blog post[1] I don't think it's that big of a deal.
I have zero inside info on this but I would bet highly that sales of RHEL were a factor in this decision for sure. I don't see how they couldn't be. But given how long Red Hat has been providing all their sources publicly (which they DON'T have to do) and acquiring and open sourcing companies, you don't give them even a tiny benefit of the doubt here that maybe their not just one-sided evil capitalists trying to squeeze nickels out of the CentOS community for short term gains?
I have not read your blog post yet, I will however I do want to comment on this
If you claim RedHat understands the CentOS Client base so well, they how did they not predict and address all of the concerns that we OnPrem Enterprise Admins have? How did they fail at the messaging soo bad, and why has there been no formal statements addressing the issues we have had?
Why has the CentOS Site not been updated to clearly address that "CentOS Stream is not that big of a change you guys are all over reacting"
I suspect because it is a pretty big change, and they know it
Ok so I have read your blog post, I would like to address some of the things you raise
First you ask "Did you think of RHEL as a beta for CentOS, if not you should not think of CentOs Stream as beta for RHEL"
This is flawed in a big way, CentOS was complied from RHEL sources, only having RedHat trademarks removed. It was binary for binary compatible. RHEL 8.3 is the EXACT SAME as CentOS 8.3, so not it was not a beta.
This can not be said for CentOS Stream, first off there are no point releases, no releases at all for CentOS Stream, it is a Continuous Stream of updates (thus the name) for the entire 5 year support window (down from 10 year support for CentOS). So where one could plan and update from point release to point release there is no such way to do that with CentOS Stream, that type of release cadence is not acceptable for a production work load
Further Red Hat has explicitly said CentOS Stream is to be used "for the development of RHEL" so how else can one take that, the updates applied to CentOS Stream will then be used to roll up into a point relase of RHEL, sure seems like beta in all but name to me... Sure maybe it will be more stable than other types of Beta Testing, and maybe the baggage that comes with the word beta is a bit to strong for this case. Still CentOS Stream is not binary compatible with a Point release of RHEL that much is clear, to many of use that would classify it as a Beta
Secondly you also claim that RedHat or CentOS has acknowledged that is botched the messaging and it is water under the bridge. I have been following this issue pretty closely, and I do not see any such acknowledgement. I suppose it could have been on social media, i do not follow social media, but it certainly was not added to the Blog Post, nor has a new blog post been issued acknowledging it, nor has the original blog post been updated to clarify things
So no I dont think it has been acknowledged, nor it is water under the bridge we should move on from
Third you talk about "CentOS being old crusty etc", for many of us this is the EXACT reason we use CentOS, we support long lived legacy system that we do not want to change very often, some of use run software are a DECADES old, some dating back to long before I was born (and I am a middle aged man). We need, want a "crusty old" STABLE system. I am not runing the latest NodeJS project, or Ruby, or Rust, or Go, or what ever the hot new startup lang of the week is today...
I am not running docker, k8s, or any of the "hot new" technologies.
I am running an old EPR, an old database, an old Java app, or some other legacy system. I want stability, not hot new feature. CentOS is my choice for stability, if I want to play in the wild world of new hotness I have Arch for that.
as to licensing sales, RHEL shots them selves in the foot here. For example tagging their no support license as "not for production use" is a bad move IMO, why is not for production use, if I do not need RHEL support, why should not use a RHEL licensed system in production?
There are many many other problems with RHEL sales but that is for another day.
Though I do agree shoving CentOS in the face of a RHEL sales person is a shit thing to do. I hope it does not happen all that often, that said if a RedHat sales person does not have ready answer for that well that should be day 1 training for RedHat sales staff.
One final thing, the publication of sources. Some of the code they absolutely are required to release the sources. any Code that GPL. Now I understand not all of RHEL is GPL, and some if it would get in the grey area of what is "linking" code, but that is for lawyers to debate over, and like you I hope the day never comes where the lawyer need to debate over that issue
RedHat has done alot of good in the community, but FOSS is a three leg stool. Trust is one of those legs, and RedHat is taking a hacksaw to that leg right now....
Completely agree. I am critical of Red Hat for not announcing this all at the same time, but I likewise very much agree with you. We still have a full year and they've said to expect an announcement within a month or two (at least that's what I read elsewhere, nothing I've heard internally).
“We aren’t going to fulfill our LTS support promises of CentOS 8, but just wait a few months until we announce the replacement. Trust us, it will be different this time.”
Thanks for the blog post. Ep383 of the Linux Unplugged podcast [1] also went into detail with CentOS members.
What CentOS gave you until Stream was (with some delay) the "same" set of packages that RedHat released in RHEL, which was (hopefully exhaustively) tested by RedHat QA, including making sure that packages interact in a sane way. That's what earned RHEL (and by extension CentOS) that designation of stability. By virtue of being upstream, CentOS Stream could have a lot more updates (the kind that would previously be internal to RedHat). I doubt same level of QA can/will be invested for each update state (if it could be, RHEL would probably be a rolling distro). If you don't have a QA safety net of your own, the risk is higher with Stream vs 8.
You could hack point releases into Stream: wait for RHEL release, then dnf versionlock Stream packages to the same release versions as RHEL (assuming you can map versions of modular packages).
Good luck with that position that your customers are wrong and you didn't just fuck them all over (even the paying ones). You might want to update your linkedin.
We call that "shooting the messenger." I didn't do anything to anyone lol. I was as surprised as everybody else when I saw the news (and I know you didn't read my blog post now btw because I discuss that in there).
If shitting on some person on the internet makes you feel better, by all means take it out on me.
A: Oracle Linux support costs money. If you just want the software, it's 100% free. [...] Yes, we know that this is Oracle, but it's actually free. Seriously.
[...]
Q: Why are you doing this?
A: This is not some gimmick to get you running Oracle Linux so that you buy support from us. [...]
"""
Wow, I’ve never thought of the people that have to go to work every day and try to convince people to use Websphere. What a soul crushing job that must be..
It's almost as soul-crushing to have to support a Websphere deployment. I've been trying to get our DevOps team to migrate off of RHEL 5 for 6 years, but they have to do upgrades to Websphere first, as well as the 3rd party modules that aren't supported under RHEL 7 (wth). Sometimes I think of becoming a lumberjack.
Redhat is still managed as geeky project, where you have tech wars.
Sadly, I think that in few years IBM will make sure that only wars will be internal turf wars between different departments.
Decision to drop something like btrfs will be based on which VP has better political game skills, and people working on it won’t have much saying in it. I’d love to be wrong tho, as redhat is one of the most important Linux project in the history, and it’ll be sad if it ends like that.
They do (I know them, and I have talked to them since Wednesday). That's not to say this was not a screwup, but I would say that rumors around the death of free RHEL rebuilds have been greatly exaggerated.
Many years ago, I fondly used and documented the free and open Red Hat distribution, which ended with release 9 in 2003. I still have a hard drive with the original Red Hat 6 based on System V init, not the later v6 based on Upstart.
There was a great feeling of abandonment then that is nostalgic in the death of CentOS now.
In the years that have passed, I saw a few licenses purchased in my workplace, then support suddenly stopped by corporate sources who instructed all license holders to convert our installs to Oracle Linux support.
I remembered my feeling of abandonment by Red Hat, ran the script without complaint, and all was well.
In later years, focus returned to Red Hat licensing, and I was strongly encouraged to reinstall my Oracle Linux systems (which had grown greatly, as they were free). I resisted vehemently, objecting to an inferior kernel (compared to the UEK), reduced hardware support, and the pointless inconvenience of license keys, activation, and renewals for a product of generally lower quality.
Fortunately, I have avoided this inconvenience.
In light of the decades of Red Hat's behavior, I will say one thing: you reap what you sow.
> In light of the decades of Red Hat's behavior, I will say one thing: you reap what you sow.
I'm surprised to hear you say that, because this is exactly why I personally won't touch anything from Oracle with a ten-foot pole; I'm not fond of RH yanking support, but we're talking about the company that is known throughout the industry for license audits and calls from legal as a sales strategy. Is there some reason why you trust this particular product more?
Article kind of hints at it but I suspect RHEL is going to get a new licensing model to capture the CentOS crowd under the RHEL umbrella. Once that's done it will be easier to upsell support contracts since they'll have the users in their license DB.
If this is their plan they have screwed up pretty bad and shows a clear mis understanding of enterprise customers
We like stability, if they were going to change RHEL licensing to capture CentOS enterprise customers they should have announced those changes ahead or in tandem with the CentOS Announcement
The CentOS Announcment as is read was "Hello Enterprise Users of CentOS, we know we promised to support CentOS 8 until 2029, April Fools, now we are going to cut support at the end of 2021, and maybe we will have some options for you early 2021 so call us"
This uncertainty and betrayal of trust is going to be hard to come back from
I’m honestly surprised they don’t just make RHEL free and take the CentOS market for themselves. Gate the current RHEL specific features behind support contracts and call it a day. Why make your customers have to migrate to pay you?
Yeah I really wonder if this is coming. It's wishful thinking on my end so maybe isn't reasonable, but man it'd be neat. I get annoyed when people point out "you can already get a dev license for free." Sure, but there are hoops to jump through and you can't run it "in production" (which I think is mostly a meaningless claim, but still not something I like).
Nevertheless I'm optimistic and hope they announce it sooner rather than later. Hopefully the current blowback will move things along ;-)
This is what Oracle Linux already does, so Red Hat is only hurting themselves by not doing so, since anyone who wants this can just go to Oracle. (Notwithstanding the fact that Oracle is evil.)
But here's the kicker: the worst thing Oracle can possibly do is to say "yeah, that whole Linux distro that we used to give you for free? We're not doing that anymore." So even if you do move to Oracle Linux and they go full evil, you're no worse off than you are on CentOS right now.
Look I just did an Oracle to OpenJDK transition for the same reason but I don't really think Java SE licensing is an example of Oracle being particularly Oracle. It's just that Oracle Java isn't free and they're doing what every company is doing switching from one-time license fees to subscriptions.
Au contraire, there are and they're quite valuable their enterprise customers. The most important is the detailed security errata published along side their RPM packages which are necessary to do things like "only install security updates" or "check if any packages have upatched vulns."
RHEL but not CentOS also come with security/regulatory profiles like STIG which are necessary to run in certain environments like the DoD.
Red Hat also has a license to distribute Oracle Java that CentOS lacks and so it's not available in their repos. They also ship with the Cisco network drivers that CentOS again doesn't have a license to distribute.
There's also the extended support for minor releases. With RHEL, it's possible to stay in a minor release for longer while in CentOS you always have to migrate to the next minor release. For instance, according to the graphic at the middle of https://access.redhat.com/support/policy/updates/errata with Extended Update Support one can stay at RHEL 8.2 until the middle of 2022 (so RHEL 8.2 has a full 2 years of support), while with CentOS, you had to migrate to CentOS 8.3 this month if you want to keep receiving updates (so CentOS 8.2 had only 6 months of support before being replaced with CentOS 8.3).
The IBM bean counters have spoken and Red Hat is to be squeezed of every last drop of revenue. I bet they are demanding Red Hat employees apply for patents for everything they are working on too.
CentOS has had a stability advantage by being downstream of RHEL. It makes sense to switch the free version to be upstream of the paid version.
Unexpectedly changing CentOS 8 after many had already migrated to it makes no sense and thoroughly damages Red Hat’s reputation in my view.
Perhaps Red Hat will consider continuing CentOS 8 under the same terms they had promised, while also releasing CentOS Stream. Many would value earlier updates over additional stability and make the switch willingly. The rest would avoid having the rug yanked out from under them without warning. Obviously, this dual path would require more effort, but it would engender more trust in Red Hat going forward even if we knew there would be no CentOS 9. It would also keep many from migrating elsewhere.
While CentOS users might not represent that many IBM/RH customers, I suspect CentOS's availability has contributed to RHEL being so widely supported.
If fewer people, even people who will never pay, use CentOS, then (potentially) fewer people will explicitly target RHEL/RPMs, and there's a weaker case for enterprises adoption RHEL over its competitors.
CentOS led to people learning "how to use RHEL". While RH is going to try to bring developers and other non-commercial uses into free licenses, I'm not sure this will be enough.
Since Centos Stream is upstream of RHEL, is it possible for an organization downstream to just manage a set of repositories (or whitelists) so that it only releases the Centos Stream versions that reached RHEL?
AFAICT, Amazon Linux 2 is largely based on RHEL 7, not 8. So not at all nearly the same system. If you're "simply switching" from CentOS 8 to Amazon Linux 2, prepare to put some serious work into it.
We have an internal RHEL vs AMZ Linux 2 battle, because of the RHEL support (would be a no brainer for AMZ otherwise). And it's tilting, because we run almost everything in Docker now, and we're not entering the OpenShift/podman ecosystem.
Let's say you run RHEL on your production servers. You pay for licensing those servers, which gives you Red Hat support, and also because you have proprietary software which is only supported on Red Hat.
You pay a pretty penny for those production servers. But you also need development servers, and staging servers. For those servers, RHEL would be a gigantic waste of money. It's the production servers you need enterprise support for, none of the rest.
CentOS was a downstream release of Red Hat, with the same packages from Red Hat compiled. It was, for the most part, identical to Red Hat without the support. This meant you could develop on CentOS Dev Servers and test deployment on CentOS Staging servers, and expect it to work exactly the same when you sent it to production. Lots of less licensing costs, same results.
Now, CentOS Stream will be upstream of Red Hat. Instead of getting the same packages as Red Hat, now you're getting the testing packages of Red Hat. You can no longer expect to get the same results from CentOS stream as from RHEL. The end of life for updates for existing CentOS releases has also been moved up drastically. By all appearances, Red Hat and IBM are essentially forcing companies to license and use RHEL on their dev and staging servers, by making CentOS Stream unusable for it.
Lucky for many, Canonical went and got Ubuntu certified for many of the same proprietary hardware and software systems that were previously only supported for proprietary linux distros. This means your dev, staging, and production can all be the exact same distro, and the only difference is which servers you're paying for Canonical support for. This is a recentish change though, so many companies locked into Red Hat / CentOS because of the clear path that was established between Development and Production. And certainly not every proprietary software/hardware is supported on Ubuntu, yet. Lots of companies won't switch (the structure of Red Hat to Debian is pretty drastically different), and IBM is counting on that.
I mostly agree with you (minus the cynicism), although CentOS Stream for dev and staging is still a pretty reasonable approach, in fact a good idea as if the next version of RHEL will break your app, you'll find out in development.
If you're already a CentOS/Red Hat shop there will be much easier options than Ubuntu (nothing against Ubuntu, I use it for some things and it's great, but it's pretty different so all your automation will have to be updated for the most part).
If you don't want Stream then I expect Rocky Linux will be available long before end of 2021, and RHEL might be free for your use depending on what they announce in the next month or two.
The easiest path forward is to convert to oracle linux. You don't even have to reinstall centos for that. Being rhel clone it's already supported by many commercial apps and unsurprisingly by oracle apps.
Given that's an option, this recent redhat announcement is really shooting themselves in the foot.
2. Vendor compatibility, many enterprises non-free software packages only offically support RHEL / CentOS
3. Familiarity with tooling, while it is all Linux Debian and CentOS do have ALOT of difference when it comes to administration of the system. Due to 1 and 2 outside of web development, in the US most enterprise users of Linux (ERP System, databases, enterprise Java Apps, etc) are a huge part of CentOS's user base
Number one for me is familiarity since I have a long history with Fedora and CentOS.
Secondly though selinux (once you learn to use it) is so powerful. I have seen it stop attacks in their tracks or render them inert (unable to call their C&C server for example because selinux blocks the socket attempt).
10 year support is also icing on the cake, although I typically upgrade a year or so after a new major release.
This is the primary reason I use CentOS and RHEL as well. If you are a Fedora user especially I wouldn't write off CentOS stream so quickly as it might match your needs just fine. If it doesn't Rocky Linux seems quite promising as well (though very early days still). More info elsewhere on this large thread or in this blog post: https://freedomben.medium.com/centos-is-not-dead-please-stop...
Fedora won't be changing at all. It's always been the upstream of the next major version of RHEL. Now it will be the upstream of the next major version of CentOS and CentOS will be the upstream of the next minor version of RHEL. By the time stuff makes it to CentOS it will have had lots of time to bake, will have passed RHEL QA, CI, etc.
Yeah it's surprisingly nuanced and took me time to grok it as well. Part of the challenge was there wasn't really an organized collection of info so people were left to speculate and make assumptions.
Happy to answer any questions you may have afterward, though if you post them here I may miss them but my email is in my profile.
So, what I don’t understand is that I thought the whole point of CentOS was that RedHat didn’t control it? I guess I missed the acquisition announcement, but a RedHat-owned CentOS makes a lot less sense to me.
The point as I saw it was that CentOS didn't require a licensing fee. RedHat decided to make them into an official free version whose users they could then upsell onto licenses. IBM seems to have decided that they'd make more money in this segment by squeezing licenses out of existing users and killing the product.
CentOS was never officially an "official free version". After Red Hat acquired CentOS, they positioned as a base for open source projects to develop on RHEL (which was what Red Hat needed and why they hired the developers in the first place).
Of course, users that just needed a free RHEL kept using it that way with the added bonus of corporate backing, so much that Scientific Linux said "screw it we'll just use CentOS".
IBM had nothing to do with it, unless you think every Red Hat employee would lie about it. I would think if anything RH would like to blame IBM :-)
They legitimately believe this is better. Maybe not for everybody, but for most. And for those who it's not better for they are announcing options in the next month or two for free RHEL access.
The whole point was that it was free. I’m not aware of anyone choosing CentOS over RHEL because of the governance structure. The feature set of CentOS directly depended on decisions by Red Hat anyway.
I would be suspicious of running essential business operations on a free version of a product the company also sells licenses for: I’d always be wondering if they’re going to change strategy on me.
>I wonder why the CentOS people agreed to this and did not walk away.
Cynical answer of money aside, someone else mentioned the project was running into resource issues so they may have chosen RedHat over a slow inevitable death.
Is there any mention about it on Linux Mint website?
Mint only has desktop .iso images. No server .iso images without desktop. Mint also mixes packages from Debian and Ubuntu that would probably cause conflicts on server usage.
CentOS is usually used on servers, with server .iso images without desktop.
Kind of. When you hear people say they use a "Debian based" distro they'll be using apt as their package manager.
You have 3 main "base" distros/package mamagers:
Debian/apt
RHEL/yum/dnf
Arch/pacman/pkgbuild
There are of course many more but these are the overwhelming majority.
This means if I wanted to make my own distro I could do as little as take Debian stable as my upstream, rebuild a few packages with a s/Debian/MyDisro/g and change a few default packages and call it that.
Or I could track a mix of packages from Debian testing and stable.
Or I could decide I prefer to build everything my self but still use apt for my package manager.
This is of course turtles all the way down eg: Mint is based of Ubuntu which is based of Debian.
Just nitpicking, but the package manager for the Debian family is dpkg, not apt; apt is a layer above dpkg. The same way, the package manager for the RedHat family is rpm; yum and it successor dnf is a layer above rpm.
If you do "sudo apt update && sudo apt -y dist-upgrade" then it breaks Mint desktop menu. To fix it, you can install all updates with mintupdate, and then "sudo apt -y install mint-menu --reinstall"
Package managers are one of the main things that define a distro. The big split is between packaging formats (DEB and RPM [0]) and then there are package managers for that for that (apt* for DEBs and yum, dnf, zypper etc. for RPMs [1]). You could run a foreign package manager on any distro, there even exist ones for that purpose [2], but it's ostensibly considered a defining feature for a distro.
[0] There are more of course, but those are the big two.
[1] I'm really only familiar with the Debian/Ubuntu space (Deb)
[2] See flatpak, snap, and nix/guix. Also some distros like Bedrock Linux let you have completely foreign environments mixing together.
You can install and use different package managers on different distros, but with the exception of things like cargo it's highly recommended to not, as you are likely going to break your system.
For all intents and purposes, yes. Besides making software easy to install, package managers figure out all the dependencies that a given package requires and install those as well. Can you install yum/dnf on Debian? I'm sure there's a way to do it. But good luck handling all the dependencies for a given package. Plus, there will be file conflicts when dnf installs a base dependency that apt already installed.
In other words, nothing good comes from trying to install two different package managers on one system.
There's no particular reason. The OS layer is just another variable which needs to be accounted for in the planning stage, so pulling the rug out like this is a nuisance.
That was to be expected for a long time. Embrace (financing most of Linux development, with SuSE historically a close 2nd), Extend (introduce Linuxisms such as systemd, containers), Extinguish (make putting together all the F/OSS amounting to a complete O/S an art, make changes for the sake of it, sell "support").
What's not answered is why now, in the middle of an active CentOS release cycle? Smells like IBM/RH doesn't see growth in new enterprise customers/startups coming to Linux (going to clouds/k8s instead and focusing on creating Docker images for workloads). RedHat surely must've been fully aware that discontinuing CentOS would create a giant backlash. I guess, like me, few people haven't used CentOS as base for new builds anyway; OTOH corporate clouds (OpenShift, k8s) for big customers seems like an attractive market for IBM/RH.
Edit: a good outcome of this catastrophe might be to shift focus towards POSIX, LSB, and other means for portability which historically had strong following in the Linux and BSD communities
And the only thing they're extinguishing is a free (as in beer) distro that they controlled anyway (and choose not to continue putting resources into).
When they do it by screwing over their users, yes.
> Introducing Linuxisms in ... Linux?
Introducing non-portable ways of doing things that were previously portable to non-Linux OSs, yes. It's certainly a minor point since if anything some of those changes increased portability within the Linux ecosystem, but I can sorta see "introducing Linuxisms in areas that previously weren't Linux-specific" (ex. `service foo start` did the same thing on RHEL and FreeBSD; `systemctl start foo` isn't going to be a thing on any BSD).
> When they do it by screwing over their users, yes.
I agree that screwing over your users is a bad thing, but I do think RH contribute a lot of time, money and effort in a way that benefits a larger Linux user base. Just by looking at Linux kernel contributions, RH are one of the largest companies contributing developer time.
> Introducing non-portable ways of doing things that were previously portable to non-Linux OSs.
I think SysVinit was going to be replaced sooner than later. There's a lot to be desired in a way that some systemd developers evangelized it, but the other major alternative (upstart) was also a Linuxism.
Even BSD systems are moving (or considering moving) away from it. From all the "standard" Unixisims, I think that's one of the parts that's aged the worst, and it was a matter of time before various operating systems (BSDs, Linux distros,...) replaced it.
The other day I commented that I took my Pinebook Pro off the toy shelf and loaded up Manjaro. I commented on HN about how much I liked the user experience. The first reply I got was something about the drama with some dude named Phil. Alternatively I’ll put my head down for a while and forget about the lineage of the various options and then something will crop up about this or that distro is going away, being repositioned or whatnot.
Why does this matter? Because I want to use it for real work and I don’t want to spend time planning a long term OS strategy. There are other things to worry about. I’m far from alone in this. That’s why folks reach for Mac OS and Windows.
Downvote away, but I really do want to love Linux. I just don’t want it to be so hard to love.