
Kubernetes 1.16 - onlydole
https://kubernetes.io/blog/2019/09/18/kubernetes-1-16-release-announcement/
======
bt848
I wish their tech writers would give us more English and less jargon. I
pondered these release notes and the documentation earlier today and I still
can't puzzle out why "custom resource" is initialized as "CRD".

And the rest of this gibberish is just meaningless.

"""A resource is an endpoint in the Kubernetes API that stores a collection of
API objects of a certain kind."""

OK, almost completely free of meaning ... then later:

"""A declarative API allows you to declare or specify the desired state of
your resource and tries to keep the current state of Kubernetes objects in
sync with the desired state."""

But wait, I thought the resource _was_ an API endpoint? The API allows me to
declare the desired state of my API?

I know K8s is not Borg and I know a lot of xooglers hate Borg but at least the
Borg documentation was concise and made sense.

~~~
atombender
It's not gibberish, though.

A resource _has_ an endpoint, i.e. a URL that represents its configuration and
state. But a resource and its endpoint can be thought of as the same thing, in
that the endpoint is the resource's canonical URI. Think REST/HATEOAS.

A resource is any object that Kubernetes can track. They include objects like
services, pods, nodes, ingresses and so on. Together resources form an API.

A CRD is a Custom Resource Definition. It defines the schema and API of a
custom resource (as opposed to a built-in one like "service" or "pod").

A resource is pure declarative data -- configuration and state commingled in
one JSON document -- but can have behaviour associated with it.

(Confusingly, Kubernetes also uses the word "resource" to refer to compute
resources like CPU and RAM. For that reason, you'll often see "object" used
instead.)

~~~
jeffdavis
"But a resource and its endpoint can be thought of as the same thing, in that
the endpoint is the resource's canonical URI."

After reading your description, "a resource is an endpoint" still seems wrong.
Perhaps just lazy, but laziness in technical docs gets confusing fast.

~~~
rumanator
It's not laziness at all. It's the project's Ubiquitous Language, which helps
refer to concepts objectively and free from ambiguities. You're complaining
because you are not familiar with Kubernetes, and thus you are not the target
audience. Do keep in mind you're reading the release notes of a minor release.

~~~
bt848
With respect I disagree and my objection is that this verbiage is ambiguous
and imprecise, the opposite of a ubiquitous domain language. Why does the
“declarative API” allow one to “declare or specify”? Which is it? Is there
meaning in the clause “or specify”, or is it a superfluous verbal tic? If the
API is declarative why is it called a Definition? Declaration and definition
are well-used terms of art in our industry but they are not synonymous. And
what about “tries to keep them in sync”? Tried by what means? Are they
eventually synchronized or not?

------
sergiotapia
We're moving away from Kubernetes because managing even a simple
implementation is a full-time job. Obviously Kubernetes is meant for
enterprises, but I've seen seed round/series A startups with a small
engineering team using it.

We were at a crossroad. Either we hire an SRE that has extensive kubernetes
experience for $130K/year or we move to a managed platform until we actually
need those capabilities.

We're happy with the move and the pressure relief is tremendous. Now we can
focus on features and not whether or not deploying is using ENV vars from our
local machines and causing the system to crash.

~~~
manigandham
Why not just used managed Kubernetes? Every cloud vendor has it, even Tier 2
operators like DigitalOcean.

Running Kubernetes is vastly different for using it to run your apps. You
really shouldn’t do the former unless you have to be on-Prem or have some
other need.

~~~
013a
There's a ton at play here.

"Managed Kubernetes" really runs the spectrum between "one step above just
installing it yourself on a bunch of VMs" and "I spend 1% of my time managing
anything below the product." Each cloud provider exists somewhere different on
this spectrum, with none of them being in quite the same location, and some of
them have _multiple different products_ which exist at different points.

For example: AWS is among the most bare-bones. EKS is just a managed control
plane; coming from GKE, you might click "create an cluster" then be very
confused how there are no options for, say, instance size, or how many...
because you have to do that all yourself. There are tools like eksctl or
Rancher which can help with this, but ultimately, you're managing those
instances. You're doing capacity planning (you think kube would be a great
pick to integrate with spot fleets because of its ability to schedule and move
workloads to a new instance when one goes down? have fun setting it up, hope
you like ops work.). You're doing auto-scaling (and that ASG? its not going to
know about your pod resource requests, so you either need some very smart
manual coordination between the two, or you need to set up cluster-
autoscaler). You're setting up cluster metrics (definitely need metrics-
server. not heapster, that was last year, metrics-server is this year. but how
to visualize? do i host grafana in the cluster? then i need to worry about
authn. cloudwatch really isn't made for these kinds of things... maybe I'll
just give datadog a few thousand bucks.) Crap, 1.16 is out already? They only
support 9 months of releases with security updates?! I feel like I just
upgraded my nodes! Oh well, time to lose a day replicating this update across
all of my environments.

I'd go on, but you get the point. There is nothing "managed" about EKS.

DigitalOcean is pretty similar to this (it does provision instances, but the
tooling beyond that is barebones). Google Cloud/GKE is "more managed" in a few
sense; the cloud dashboard provides some great management capabilities out-of-
the-box, such that you may not need to reach for something like Datadog, and
the autoscaler works really well without a lot of tinkering. There are still
underlying instances, so you're worrying about ingress protection, OS
hardening, OS upgrades, etc... but its not as bad as AWS. Not by a long shot.

The holy grail (for some companies) is really something like Azure AKS + Azure
Container Instances. No instances to manage. Click a button for a kubernetes
cluster. Schedule workloads. Get functional metrics, logging, tracing,
dashboards out of the box. Don't worry about OS upgrades, hardening,
autoscaling, upgrading the cluster, etc; we'll do it all for you, or at least
make it one click to configure. That's the ideal situation. I haven't used
AKS/ACI so I can't comment on whether Azure gets us there, but the idea is
sound; even if its more expensive.

This sounds like an anti-Kube post, right? Wrong (its a Tide ad). The
beautiful thing about Kubernetes is that _it can span this spectrum_. The same
exact API surface can scale from a fully-managed abstract platform where you
just say "take this git repo and run it" (see: Gitlab Auto-DevOps), all the
way to powering millions of workloads across dozens of federated clusters at
Fortune 500 companies.

But, to the OPs point: We're close to solving that right end of the spectrum,
and a lot further away from the left end. We're getting there, but we're not
there yet. There isn't enough abstracted management of these compute
resources... yet. But there's enough money and desire for there _to be_ that I
know we'll get there.

~~~
deepsun
> The holy grail (for some companies) is really something like Azure AKS +
> Azure Container Instances. No instances to manage...

Google Cloud have been having that since before it's being called Google Cloud
-- I mean Google App Engine. It has all those features, and with its
"Standard" environment you don't even need to build docker image.

~~~
9nGQluzmnq3M
Cloud Run is even closer. App Engine Standard requires you to use the sandbox,
App Engine Flex is tied to VMs, but Cloud Run makes containers serverless:

[https://cloud.google.com/run/](https://cloud.google.com/run/)

...and since it's based on Knative, it's portable-ish.

~~~
gorbypark
I can attest that Cloud Run has been great for me. I literally "ported" over
some of my apps that were already running in containers by replacing
references to my existing env PORT_NUM to GCR's PORT and it just worked. It's
been a life changer. Now if it could run non-http workloads, I'd be able to
shut down every one of my VMs.

------
jcastro
Just a reminder to check out the API deprecations on 1.16:
[https://kubernetes.io/blog/2019/07/18/api-deprecations-
in-1-...](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/)

~~~
zeeZ
> DaemonSet, Deployment, StatefulSet, and ReplicaSet (in the
> extensions/v1beta1 and apps/v1beta2 API groups)

That seems quite significant. I still come across those regularly.

~~~
wereHamster
You gave me a heart attack. I thought they were deprecating Deployment,
StatefulSet… But it's only the old API versions they are deprecating, not the
features themselves…

~~~
zeeZ
Sorry about that. Yeah I meant the old API groups that I still come across.

------
whotheffknows
I enjoy Kubernetes because it weeds me out from all the people who aren't used
to actually having to read docs to understand the layered concepts of k8s as a
networking model, of which there can me multiple opensource Implementations.

The opsys bros I know who are used to hacking together docker-composes that
are more insecure than your grandmother's internet explorer browser in 30
seconds and calling themselves geniuses fail miserably when they are left to
design construct and optimise secure kubernetes clusters and elegantly port a
variety of full stack web applications in a coat efficient manner.

It requires planning and design. Just because you haven't invested hundreds of
hours into reading the docs.and testing, and building k8s clusters for which
the docs are incredibly useful...

Also "kind" represents the type of k8s object, like service or deployment.
It's not ambiguous at all, it just requires reading to the third paragraph of
"general Kubernetes concepts" in the Kubernetes docs to understand it.

------
winrid
So, do we expect Kubernetes to ever replace Borg?

~~~
dward
An ecosystem has evolved around Borg. Custom hardware, kernel, schedulers,
telemetry, atomic clocks, networking, security, management... have all evolved
around Borg proper to meet the "enterprise" needs of one of the largest
enterprises in the world. It has many niche (i.e. not generally useful)
cababilites built to service hardware melting XXX megawatt applications like
web indexing, gmail, colossus. Even if Kubernetes targeted this customer, Borg
has a significant head start. At this point the Borg ecosystem is almost old
enough to by cigarettes in Mountain View.

~~~
ngrilly
Custom hardware? What kind and how is it related to Borg?

------
_-___________-_
Great to see IPv4/IPv6 dual-stack support in this release, if only in alpha
state.

------
znpy
My biggest quarrel with kubernetes is that its API Reference often contains
broken link to non-existing documentation page, usually pages that have been
moved somewhere else (but who knows where?).

------
MichaelMoser123
A question on endpoint slices:why dont they send patch updates? A patch is the
delta that changes an object instance.

~~~
haimez
Because safe generalized PATCH operations require the client and server to
agree on the immediately prior state and the order of application of patches
to avoid creating “Frankenstates”. They’re not a good general purpose API
building block.

~~~
MichaelMoser123
The client could do a full list request when it starts to get the baseline and
then listen for patch updates.

That would be kind of an operator in reverse.

------
tomphoolery
that owl looks pissed

~~~
dankohn1
That's Captain Kube. More info at [https://phippy.io/](https://phippy.io/)

~~~
hinkley
What the actual fuck.

~~~
imtringued
Cartoon giraffes in general look very disturbing to me. Generally the length
of the neck and legs are about the same but somehow every cartoon giraffe has
amputated legs.

~~~
hinkley
The horse with dwarfism and hightops is making me uncomfortable. But the
gopher will haunt my dreams.

