
Walmart Will Manage Distribution Centers with OneOps, Jenkins, and Kubernetes - frostmatthew
http://www.techbetter.com/walmart-will-manage-200-distribution-centers-oneops-jenkins-nexus-kubernetes/
======
avitzurel
I had a very interesting discussion about this subject a couple of weeks ago
with a CTO of a big company.

I've commented before that Kube absolutely takes over the bigger more complex
cloud installations out there, you can see how many companies are betting
their infrastructure on it.

The only thing that I don't see is standardization of the cloud, just like
what Amazon did. You see too many companies doing too many of the same things
and reinventing the wheel.

Personally, I would love to see smaller installation as a standard of how to
take things into the cloud as a cluster. Imagine what Heroku did to
deployments. You can't beat this ease of use. Deis and Convox are both trying
but not really "hitting it".

As for Walmart, absolutely stoked seeing it from them. This move and what the
white-house did with the digital shows a lot of promise and hope. I wonder how
much of this is on top of "older" management and how much is just complete
restructuring.

~~~
hueving
Kubernetes != virtualization

Some virtualization use cases can be solved with kubernetes, but many things
an enterprise runs do not work at all in the kube paradigm. The majority of
applications are pets (i.e. stateful) and many applications run on Windows or
have some other kernel requirements that mismatch the baremetal kubelet.

If you think Kubernetes 'absolutely takes over the more complex cloud
installations', you're living in an echo chamber of 12 factor apps that
doesn't line up with the majority of what I've seen in big enterprise cloud
workloads.

Case in point: Walmart has one of the largest openstack deployments in the
world.

~~~
maloney
Have you checked out pet sets[1]? The last couple of kube releases have
introduced some really nice work to help with these cases

[1][http://kubernetes.io/docs/user-
guide/petset/](http://kubernetes.io/docs/user-guide/petset/)

~~~
tmzt
Does this fit in with hyperkube in some way? Can you see 'pods' that are
really complete VM's which may or may not include OCI containers on them.

For instance, would a complete Windows Server instance fit the Pet Set concept
in the scheduler?

~~~
sandGorgon
hyperkube is just a repackaging of all the different packages of k8s as a
single binary.

so rather than 'kubectl -v', you will run 'hyperkube kubectl -v'

~~~
tmzt
Closer to minikube then? That was not at all clear to me from reading the
site, where it seemed to be discussing a kubelet on an OS on bare metal or
hypervisor, but not managing containers.

------
empath75
I messed around with oneops a bit and it's basically a bunch of tooling built
around their own fork of chef.

It's almost impossible to just drop into an existing workflow without a whole
team of people managing it.

------
philip1209
NFS in a modern system seems like a huge red flag.

~~~
cosmie
What's wrong with NFS? If you need a shared, POSIX-compliant drive between
multiple hosts, there's not really an alternative. Even the fairly new AWS
Elastic File System service is just a NFS4.1-based managed service

~~~
falsedan
You can pay money for a SAN, which is $$$$ & enterprise but is a viable
alternative…

~~~
jo909
A SAN gives you block storage - virtual hard disks. You will need a cluster
filesystem to be able to use the same virtual disks on multiple machines
concurrently, which are not easy to operate and have their own bunch of
performance problems.

------
bluedino
Wal-Mart is a very odd to work with from the vendor side. They have a purchase
order tracking system named PULSE that is completely independent from their
EDI system and their supplier Retail Link program (which is the slowest, most
horribly designed website you could imagine).

You're required to SFTP up a CSV file of what you've shipped to them that day,
along with information about. They also have a creaky
acknowledgement/acceptance procedure and the technical folks (seems
outsourced) aren't very impressive. You have to pass a 'visual inspection'
with your test files and it's the most ridiculous process you can imagine with
box-checkers flagging you for the dumbest reasons.

So from this side of the transaction it always makes me wonder about articles
like this, all their good tech must be internal only. To be fair to Wal-Mart
the situation with other big store chains usually isn't any better.

~~~
user5994461
SFTP and CSV are robust and simple. That's solid choices.

------
jacques_chester
This is one of the better bespoke cloud efforts I've seen, but in their
position -- given the decision to bet on Kubernetes -- I might've chosen
OpenShift instead of rolling my own PaaS.

I've also seen the Jenkins-to-Nexus thing a few times, never particularly
happily. That said, I don't have particularly deep experience in Java shops,
so it's possible that it works really well in some places.

Disclosure: I work for Pivotal, we're the majority donors of engineering to
Cloud Foundry, a PaaS competing with OpenShift.

------
falsedan

      > As shown in the above diagram, Jenkins pipelines deliver application updates directly to the enterprise Nexus instance
    

Well, at least they're not using Jenkins to run their deploys.

