
Red Hat is pretty good at being Red Hat - ridruejo
http://redmonk.com/jgovernor/2017/09/21/red-hat-is-pretty-good-at-being-red-hat/
======
StillBored
I've been hearing the "boring" comment a lot lately, and I totally agree. I
don't want the exciting filesystem (btrfs which just took me for a ride), or
the exciting programming language/library that will strand me in 6 months with
multiple man-years of work.

No, what I want is boring. That means everything works as I expect it too, and
it doesn't create drama every few months when the cool kids decide that the
old way of doing things isn't cool and rewrite a library i've based my
application on or change the syntax of the compiler so my code doesn't work.

A few years ago, I started challenging people at work when they were trying to
pull in the latest language/library/toolkit, with a simple question. "How does
that help our customers". Sure, I got plenty of the 2/3rd order arguments
about how making the dev team happy/more productive would result in better
feature development/etc. But I've seen these kinds of changes enough to know
that usually these 2nd order affects are crushed by the unseen problems that
_ALWAYS_ seem to crop up after your already committed to that new technology.
Its truly rare to have a new technology that is so much better than the last
that the cost of switching pays for itself. That doesn't mean it doesn't
happen, it just means that the immature crap everyone is yammering about is
likely not that technology. Give it a couple years and if it turns out to
still be around (and people are still actively switching to it) then its
probably good. This mindset has produced a long list of "successful" products
that pay the bills and make people happy without making a lot of noise and
crashing/burning regularly.

Bottom line, I will take the old "cruddy" technology with the long list of
known problems, to the new cool one with the long list of unknown problems. At
least part of that seems to be what RH provides for the opensource community.
The old guys that actually fix the bugs, rather than switching to some cool
new product.

~~~
cyphar
The thing that makes distributions like Red Hat and SUSE "boring" is not that
we only do maintenance work, it's that we've effectively solved the problem of
maintenance and release engineering -- though of course there's still
improvements to be made (and on the SUSE side, openSUSE Leap and the plans for
SLE15 are quite interesting developments).

We do a lot of work on exciting and interesting projects, some of which crash
and burn as well. But part of the benefit of working for an enterprise
distribution is that generally if some project is not ready to ship yet, we
don't ship it. Having an out-dated version of some package is fairly common,
and as long as we can do the maintenance work on it, it doesn't really matter.
So we have a lot of time to improve upstreams to be stable enough to ship.

[I work for SUSE.]

~~~
exikyut
> _The thing that makes distributions like Red Hat and SUSE "boring" is
> ...that we've effectively solved the problem of maintenance and release
> engineering_

Hmm.

There are a lot of developers that could probably really do with some exposure
to this kind of environment.

I could definitely see myself hugely appreciating, say, a 6 month stint
doing... _something_ relating to the nitty-gritty of keeping a distribution
going, learning what the pitfalls are, what the unintuitive stuff is, etc.
Basically being some key point-person's assistant (the one who deals with
major disasters), or something.

It would be even better if some of this work could involve direct enterprise
support: learning what customer requirements are, getting a sense of scale,
etc. I can see this kind of information (enterprise support) being the most
valuable - actual bespoke software would likely be a major focus of such
training, and this would tech would-be developers about the disasters that can
happen 5 years or 10 years on from "I have an awesome idea" "wow it's so
elegant" etc etc. (Continuing from the previous point, perhaps there could
also be an opportunity to be the assistant to the on-call person who sometimes
wakes up at 3AM.)

You could position this as something for people looking for DevOps jobs, and
_maybe_ make some kind of cert for it as well (this would likely help
with/create traction for/be needed for enterprise-level interaction).

I personally would have zero issues with signing a small _book_ of NDAs to get
this type of training myself. The NDAs would just cover _how_ I learned, not
_what_ I learned, and in this context the _how_ would be largely irrelevant
(but at the same time incredibly invaluable). FWIW, I'd personally greatly
benefit from a good debrief at the end of the course to help me find all the
little ways I might accidentally leak something (eg a few too many words fall
out before I stop myself etc) and learn to catch myself quickly.

------
organsnyder
My company (20,000+ employees) just started an engagement with Red Hat this
week. We went with them for exactly what the article describes: open source
software, packaged in an enterprise (i.e. expensive) fashion to make it
palatable. We could have implemented these tools ourselves, but the executives
appreciate the support contract and training, and I'm already seeing how we
can benefit from the consultants' knowledge and clout.

Red Hat is making a very strong play in the microservices/container space.
Openshift is a pretty dang good k8s distro, and they're also doing a lot in
the API space (especially with 3scale).

~~~
snaky
That's funny how things are done in big shops. It's pretty much okay to spend
a good six digit sum for some tool done by 3rd party, but just impossible to
pay 100x less to someone in-house who could add a couple of features into
existing in-house software on a weekend.

~~~
jacques_chester
> _but just impossible to pay 100x less to someone in-house who could add a
> couple of features into existing in-house software on a weekend._

They don't spend more money for fun. They do it because they've had
experiences of many on-a-weekend projects become critical-path nightmares.

Owning your own bespoke platform is a giant money hole. The arguments in
favour of doing so are plug interchangeable with "let's write our own
operating system kernel / database / programming language / HTTP server / CPU
architecture / web framework".

I also think it's a bit rude to assume that Red Hat has hundreds of brilliant
engineers just phoning it in. I work for Pivotal and we are flat. out. every.
day. writing this stuff, because it turns out that this stuff is a lot harder
than it looks from the outside.

And the reason it looks easier is because we did the hard work for you.

~~~
cat199
> They don't spend more money for fun. They do it because they've had
> experiences of many on-a-weekend projects become critical-path nightmares.

yet strangely these sorts of straw-man organizations adamantly refuse to open
source any such projects even if they have nothing to do with their business
because of the perceived loss of intellectual property value..

~~~
zzzeek
if you're a business with capital-M-Money, and you put "hello world" up on
github, if someone wipes out their machine running it they're going to sue
you. Doesn't matter what license you put on it, you're going to need the
lawyers and they cost more money than you would have theoretically gained by
putting your crappy "hello world" with the accidental "rm -fr" command in it
up on github.

~~~
cpitman
Has this ever happened? There are plenty of companies that are not software
companies releasing open source software.

------
breser
OpenShift is a fork of Kubernetes. Case in point. OpenShift has functionality
called a Route. That isn't in Kubernetes. Kubernetes of course went and added
something similar called ingress.

This means that anytime Kubernetes does a release that RedHat has to manage
merging all those changes in with their local changes that aren't part of
their project.

This is exactly the same sort of thing that RedHat does with their Linux
kernels. It's exactly why you RHEL has been such a terrible platform to run
Docker on. Because RedHat is only pulling in some of the upstream changes into
their kernels rather than upgrading everything.

This sort of merging is slow, error prone and costly. If you use anything that
they've added that isn't in the upstream it creates vendor lock-in. Sure
OpenShift's Route code is open source, but you can't take that to any other
vendor without building everything yourself. Want that new feature in the
latest Kubernetes release, you better be prepared to wait.

As time goes on it will become harder and harder for them to maintain this
fork. If OpenShift doesn't become the dominate Kubernetes distribution then
RedHat may lose interest in it and then you'll lose your maintenance of that
fork. For that matter, if RedHat loses the maintainers as employees they may
lose their ability to maintain this fork.

It's my opinion that anyone buying OpenShift is playing with fire.

The fact that RedHat has failed to get their modifications integrated upstream
and have to maintain themselves is a massive failure on their part. This sort
of thing was understandable in the past, but we really should expect more of
them now.

~~~
smarterclayton
> The fact that RedHat has failed to get their modifications integrated
> upstream and have to maintain themselves is a massive failure on their part.

Not really correct. We treated ingress as routes v2. Some of the design
choices for ingress (which is still beta, and may change again before reaching
stable) were improvements, but others created more problems.

RBAC, most of the authentication code, a huge amount of performance work,
podsecuritypolicy, egress network policy, and many others all originated in
OpenShift, and then we moved or helped move them into Kube. When we did that,
we worked in the community to improve those features. And then we do all the
work in OpenShift to make them transparently (mostly) available to the early
adopters. For instance, OpenShift RBAC APIs in 3.7 now sit on top of Kube
RBAC, and you can use either API. We'll continue supporting that for a long
time so that users can switch at their leisure.

It's just what we do.

Edit: templates are the only thing that didn't get upstream, and it was
because Helm was good enough at that point that we didn't need it in Kube. We
continue to support templates, and they are exposed under the new service
catalog work as a broker so users can be completely oblivious to their
consumption like we're hoping to do for Helm. Everyone wins.

Edit2: deployment configs also are an example of predating Kube deployments -
the fundamental design choice is actually different (DC can fail and inform
you something is wrong, deployments just try forever). We continue to add
capability to deployments to make them better than DCs, and then add the same
improvements to DCs. If we picked a new name it would be DeploymentJob - it
can run hooks, have custom logic, and fail. It's not upstream, but will be an
extension API soon.

~~~
shaklee3
This is easily confirmed. Just look at the companies upstreaming into k8s, and
you'll see redhat is dominating. They have people in almost all SIGs, and are
very active in the community. Thanks for all the contributions.

------
throw2016
Redhat might be doing well but many open source projects clearly aren't. There
are routine SOS posted here about projects in dire straits. So how much of
this revenue does downstream see?

Something as critical as Gnupg was recently in trouble and got funding from
Stripe and Facebook.

Companies like Redhat acquire projects or hire project leads for influence but
few really seem to 'support' open source. Docker is a VC funded company that
was entirely built on open source and has made many acquisitions so they had
the money but how much support did they give the LXC project, Overlayfs, Aufs
and all the other open source projects they use/used?

Github uses Redis but let alone support they never even bothered to tell the
author. Do they even support Git in any meaningful way?

If open source is used to provide free software to startups who them promptly
forget about it we will get more SOS and interesting projects will dry up.

We will then be left with companies like Redhat and others building open
source software requiring large teams which cannot be easily replicated by the
open source model.

~~~
StillBored
Redhat supports the community by providing tons of maintainers/maintenance of
projects. So, they may not support $PROJECT with dollars, but for a few
thousand projects they base their business on, they provide hundreds of
thousands of dollars in year in engineering time fixing bugs, or updating
things which are bothering their customers. Particularly on all the unsexy
portions of the OS that everyone else has moved on from.

~~~
throw2016
'Maintenance' sounds like a takeover without compensation. Surely with $3
billion in revenues Redhat should have a open source fund to sustain people
whose code they use? Shouldn't some money find its way back to all the open
source projects?

~~~
cyphar
Red Hat (and SUSE, Canonical, etc) also employ a lot of upstream maintainers
to continue doing their job. That's a much more sustainable model than having
companies donate to upstream.

------
strooper
Redhat is technically doing the enterprises a big favor by adding support on
top of tested open source technologies and making the applications enterprise
ready. Redhat (IBM, and so on) is also doing open source community a big favor
by making the open source technologies ready to compete with the closed source
enterprise application providers, such as- Oracle, Microsoft and so on.

If it is about packaging open source for profit, then Redhat has a lot to
learn from Google. There is nothing that comes close to Android OS and
Google's strategy to make the use of the term "open source" practically vague.

------
jacques_chester
I work for Pivotal as an engineer.

Red Hat was my first distro, back in '97\. I'm grateful.

I wasn't around for the discussion about why Cloud Foundry originally
standardised on Ubuntu, but if you're Red Hat, you're hardly going to bet on
that. Red Hat's lifeblood is RHEL, anything which threatens it is existential.

And PaaSes threaten RHEL, _unless_ Red Hat controls their own.

Red Hat deserve credit for making a big bet on Kubernetes so early on, when
the picture was by no means clear. It's definitely put them well in the hunt
against us as k8s picked up momentum. I think Openshift 3 plays to their
historical strength, which is in packaging upstream and supporting or feeding
back to the upstream.

The article mentions Kubo. We're adding PKS (based on Kubo), jointly developed
with Google (Kubernetes, Kubo) and VMWare (Harbor, NSX).

In Labs, from the balanced product team up to the C-suite, we can transform
the way you build software. A lot of companies come to us to learn how to
thrive in the world of disruption.

So it would've been downright silly not to spot one happening to us and ... uh
... pivot.

(Disclaimer: Nothing in my comments should be seen as official comment by
someone who decides anything of any consequence. Not forward-looking. Consult
your lawyer, doctor, dentist and gardner before taking this comment as
medication.)

------
monkchips
as author of the original post above, i am gratified by the quality of the
commentary. thanks all!

