
Migrating ZooKeeper into Kubernetes - bognition
https://product.hubspot.com/blog/zookeeper-to-kubernetes-migration
======
zek
I work at HubSpot (on Kafka) and so I was a "user" of this migration because
kafka uses Zookeeper for coordination. Its pretty amazing how convenient Kube
services made this whole transition and we actually learned a lot from this
that we will likely end up applying similar strategies for migrating other
services onto Kube. Allowing kube services to point to either external
resources or pods/internal ones is a probably the best feature I have found in
Kube so far (and there are a lot of great features)

------
klysm
it’s a little funny to think that if you’re running ZooKeeper in Kubernetes
that you’re using etcd to manage the state of the servers of your state
management servers

~~~
FBISurveillance
In context of Kafka, hopefully KIP-500 [1] will get implemented sometime soon.

If you're feeling lucky, you can also use zetcd [2] to connect ZK apps to
etcd. I've been able to actually run Kafka with it as a toy project a little
while ago.

[1]
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A...](https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-
Managed+Metadata+Quorum)

[2] [https://github.com/etcd-io/zetcd](https://github.com/etcd-io/zetcd)

~~~
takeda
What is the problem with ZK though? The only thing I can think of is that
requires JVM, but other than that it is quite solid as far as the locking
service is concerned, and since you are using Kafka you already are using JVM.

~~~
FridgeSeal
Because if I already have a K8’s cluster, and I want to run a application that
requires ZK, I now have to dedicate resources to running a ZK cluster, within
my K8’s cluster just to run the one application I cared about.

I also personally find configuring and running Java applications confusing AF.
Why are the configs seemingly split into different places and environment
variables? Does it get clearer after dealing with Java things for a while?

------
hinkley
So do none of consul, zookeeper and etcd have a tool for migrating from one of
their competitors?

I suppose you end up with Zookeeper running in Kubernetes because the only way
to migrate service discovery is to have all machines report to both clusters
and then start moving to reading from the new one.

~~~
hinkley
Looks like etcd has a module that implements the zookeeper API, but the
logistics of moving a bunch of services (without an outage) still seems
massive to me. Because old servers still want to discover in the old registry,
not the new one.

You can’t just bridge two Raft protocols. If the bridge goes down even once,
good luck getting consensus again. And based on the benchmarks I can find, it
seems the wire protocol is part of the secret sauce for at least etcd.

