
Istio – An open platform to connect, manage, and secure microservices - kumaranvpl
https://github.com/istio/istio
======
olafmol
Istio adds to Kubernetes:

\- Automatic load balancing for HTTP, gRPC, WebSockets, and TCP traffic.

\- Fine-grained control of traffic behaviour with rich routing rules, retries,
fail-overs, and fault injection.

\- A pluggable policy layer and configuration API supporting access controls,
rate limits and quotas.

\- Automatic metrics, logs, and traces for all traffic within a cluster,
including cluster ingress and egress.

\- Secure service-to-service communication in a cluster with strong identity-
based authentication and authorization.

Istio on its own is powerful and flexible. It can also be hard to understand,
setup, and manage, plus can be brittle.

(disclaimer: I'm co-founder of Vamp.io, a canary test & release system for
DevOps teams that works with both HAProxy and Istio)

What Istio imho misses is a user-friendly way of setting up, managing and
automating Istio configurations for microservices topologies to quickly
achieve automated canary-releasing and A/B testing pipelines. Call Vamp a
canary-releasing and A/B testing “control plane” for Istio/K8s if you will.

Basically the potential of service-meshes like Istio lies in "driving" it from
higher-level metric/KPI-based automation systems.

We wrote several blog-posts on how to make use of Istio, starting here:
[https://medium.com/vamp-io/taming-
istio-76fab339f685](https://medium.com/vamp-io/taming-istio-76fab339f685)
(more blogs on Istio can be found here [https://medium.com/vamp-
io/tagged/istio](https://medium.com/vamp-io/tagged/istio))

Hopefully this gives some more insights in what Istio is, what it isn't, and
how you can leverage it in your pipelines and architectures.

~~~
chrisweekly
Thank you. Your comment is exemplary, just super-helpful.

------
Tobani
I like the concepts behind istio. I want to be able to use istio, but I had it
running in a test/dev environment, and it would just randomly start saying 503
for my virtual services. Removing and adding the virtual service would fix it
for a couple of days. I didn't have time to really spend debugging it, and I
didn't see any obvious errors in the logs. :-/

~~~
charles_f
Yeah I'm always worried about using a product in the middle of an architecture
that has 800+ open issues on GitHub.

~~~
briffle
To be fair, Kubernetes has 2137 issues open right now :)

~~~
peterwwillis
I don't think there's any defense to this. If you look through the open issues
you find stale issues, feature requests with no planned work, already resolved
issues left open, "we'll get to it in a couple of releases", etc. Nobody is
managing the queue.

I know it's a big corporate product, but this is a sign of bad project
management, imho. At the very least they could offer to let a community member
go through and clean it up if they won't do it themselves.

~~~
ecnahc515
They have a bot which cleans up inactive stuff, marking it as stale,
eventually closing issues stale with no activity. Anything worth keeping open
is re-opened or a maintainer will tell the bot to un-mark it as stale.

~~~
peterwwillis
It takes four months to close an issue with the bot. But before it closes,
someone comments or removes the tag.

There are 195 issues marked stale going back two years.

------
altmind
Can somebody explain what istio does in simple words?

I see that it hooks into micro/services communication, but why do we need
that? isnt policies enforced by cni plugins? and can microservices communicate
already with k8s and mesos default setups?

is istio an network abstraction layer for k8s and mesos? why would anyone want
that, is there anybody who changes scheduling runtimes often?

~~~
unscaled
Kubernetes already gives you basic service discovery network policy and a
network abstraction layer, but Istio is more about abstracting and enriching
communication patterns on (sometimes) Layer 4 and (more often) Layer 7.

It allows you to use kubernetes manifest to centrally control policies such
as: \- Canary load balancing (pushing a certain percentage of the traffic to a
canary release) \- TLS Mutual Authentication \- Retries with exponential
backoff \- Circuit breakers \- Instrumenting your traffic for distributed
tracing (e.g. with Jaeger) \- Adding your own custom HTTP headers

All of that is transparent to your microservice. This comes instead of having
to repeat the service mesh logic that does everything above in each
microservice. From the microservice 's perspective they're just sending simple
traffic without implementing retries, circuit breakers, HTTPS or
instrumentation. Istio takes care of all the rest using sidecars and and
ingress controllers.

~~~
avita1
Is the Kubernetes native (for lack of a better word) service discovery used in
many real world use cases? The sense I've gotten talking to people who use
kubernetes is that they fall into one of two camps:

\- they're moving "legacy" infrastructure off EC2 or similar onto Kubernetes
and their service discovery stack is already in place. They end up bypassing
cluster IPs and dealing directly with pod IPs. \- they're starting with Istio
or something similar from the get go because the native service discovery is
hard to grok and leaves out a lot of the nice things that Istio gets you for
free (once you've set it up)

There's lots of overlap, but I've not heard of anyone making a strong argument
for the native service discovery.

~~~
nprateem
Unless I've misunderstood you, if you're dealing with pod IPs instead of
service DNS addresses you're not fault tolerant and you might as well not even
bother with K8s.

~~~
avita1
Well you're not necessarily dealing with a single pod IP. Two versions of
skipping cluster IPs are either having smart client libraries that are capable
of multiplexing between different IPs, or using something like consul DNS to
resolve a URL into multiple pod IPs and rely on client DNS to do the right
thing.

------
ojhughes
The elephant in the room with Istio is that the more functionality you move
into sidecars, the harder issues become to to test and reproduce. This is
because Istio is transparent to applications and runs out of band so it is
much harder to write tests to verify your application works with the latest
Istio manifest that is deployed. In theory great for operators as they get
more control over network comms & security but bad for developer workflow.

~~~
otterley
It’s hard to imagine something easier and more reproducible for developers
than connecting to localhost and having your request routed to the correct
endpoint automatically, whether you’re on a disconnected laptop or in
production. That’s the power of the ambassador pattern, which Envoy enables.

~~~
geofft
When the request is routed correctly, yes, but the question is when it's
routed incorrectly (perhaps because information you're providing to Istio is
incorrect but you haven't realized it yet, perhaps because of bugs in Istio).

------
sszuecs
Running >100 Kuberenetes clusters with a team is already quite complex. Teams
already struggle with Kubernetes core concepts and we have to make sure the
time spent does not grow, because of some nice to have features that are not
business relevant. Adding more complexity adds a lot of costs and most of the
people looking into features like istio might see that it provides less value
as promised by the advertisements. We use instead of service mesh is simple
and flexible ingress controller, which can be used as api gateway
[https://opensource.zalando.com/skipper/](https://opensource.zalando.com/skipper/).
With simple annotations (core Kubernetes concept) we can do blue/green
deployments, A/B tests, shadow traffic, you can build complex http routing as
you wish and change everything in http in the request or response path. IMHO
there is no much value left that service meshes provide: mtls and network
policy maybe, everything else skipper provides. Blue/green with a simple
kubectl plugin: [https://github.com/szuecs/kubectl-
plugins](https://github.com/szuecs/kubectl-plugins) More advanced automated
blue green deployments via [https://github.com/zalando-incubator/stackset-
controller](https://github.com/zalando-incubator/stackset-controller). Skipper
also supports several openapi providers and is used by several production
users.

We add dns names for cluster internal communication through coredns templates
to this api gateway. [https://opensource.zalando.com/skipper/kubernetes/east-
west-...](https://opensource.zalando.com/skipper/kubernetes/east-west-usage/)
You can adapt it step by step and stop without a bid all in approach as
service mesh vendors promote.

If you have not enough you can get even more advanced and add hpa by requests
per second.

We use all of these in >50 production clusters with regular shop and order
traffic far beyond 10k requests a second. We are interested in features, but
even more in stability. We are interested in the community and will fix most
of the issues you will show us to make errors less likely happen for us, too.
We don’t sell software nor support, that’s why we have less advertisements and
don’t show up on every kubecon.

I hope to read you in our community channels:
[https://github.com/zalando/skipper/blob/master/readme.md#com...](https://github.com/zalando/skipper/blob/master/readme.md#community)

------
ETHisso2017
How does the K8s / Envoy / Istio stack compare to Nomad / Consul from Hashi?

~~~
alpb
Once you install Istio, all your traffic goes through Envoy sidecar proxies
automatically injected to your pods. Consul just provides service discovery
(like DNS, which Kubernetes offers by default). By proxying traffic, Istio
lets you program how your traffic works (traffic splitting, policies,
monitoring, ...).

~~~
manigandham
Consul 1.2 supports service mesh deployments with a new feature called Consul
Connect. It includes a sidecar proxy.

Good podcast episode about Hashicorp today:
[https://softwareengineeringdaily.com/2019/02/04/scaling-
hash...](https://softwareengineeringdaily.com/2019/02/04/scaling-hashicorp-
with-armon-dagdar-and-mitchell-hashimoto/)

~~~
alpb
Oh lordy, I guess everyone’s making service meshes now.

------
michaelangerman
The cool thing about Istio is that you do not have to modify your current K8s
application. You simply need to reroute who your microservices call. So
instead of calling another service or message queue; you configure all of your
calls to call the envoy proxy. Then the proxy takes over and gets configured
according to the control plane definitions.

One of the great things about Istio is that it is a super interesting and
informative way to better understand the Kubernetes, Docker, Helm stack.

When I first started learning Kubernetes it wasn't clear to me how a small
startup or company could leverage the power of Kubernetes because in reality
you need some scale to make it worthwhile.

However, once you can play with, observe and better understand some
application infrastructure which sits on top of Kubernetes like Istio the
light goes on in your head and things get much more interesting.

Here is a really nice short simple introduction to Istio

[https://www.youtube.com/channel/UC-
zVlo1F3mUbExQ96fABWcQ](https://www.youtube.com/channel/UC-
zVlo1F3mUbExQ96fABWcQ)

And here are the playlists...

[https://www.youtube.com/channel/UC-
zVlo1F3mUbExQ96fABWcQ/pla...](https://www.youtube.com/channel/UC-
zVlo1F3mUbExQ96fABWcQ/playlists)

------
cromantin
Thou idea is cool it product in itself. As you need a k8s person you would
need istio person. It has lots of interconnected components, it own
configuration primitives (lots of them) and if any of them misconfugured or
wonky (looking at you mixer) you will have to spend days debugging it (mixer
produce 10k of unreadable logs a second).

It’s great if you know it’s codebase. Not for general case.

~~~
michaelangerman
The complexity of our world today in the software space is truly astounding.
Agreed you would need an istio person, but that can be the same person who is
the k8s person. If you have a company with large enough scale to be
contemplating istio and service meshes in general; then if Istio makes your
life easier in terms of dev ops in general, it might actually be worth it.

------
fosco
> Citadel - A centralized component responsible for certificate issuance and
> rotation.

That interests me, while I know nothing about istio, I'd love to have
something like his for me personally, if I understand correctly this could
centrally manage certificates used as password and their management?

~~~
nprateem
Cert manager [1] also automatically provisions and rotates TLS certificates.

[1] [http://cert-manager.readthedocs.io/](http://cert-manager.readthedocs.io/)

~~~
cromantin
Citadel does this only for internal certa. It does not provision lets encrypt
(at least didn’t several months ago)

------
lookingfj
Does anyone else get intermittent 503 "upstream error or disconnect" when
running istio? We get this fairly regularly and have to patch services to get
it to run again. Would love to share ideas if anyone faces similar issues.

------
coleca
The promise of ISTIO is great, but the complexity it adds are not worth it
IMHO. If you want to run the BookInfo demo app, ISTIO will work great for you.
If you want to run anything else, buckle up. I'm sure there are people running
ISTIO successfully, but there is little information out there about how they
do it, at least not with gRPC.

We fell for the hype and tried running ISTIO for a production workload and
spent months trying to get it working with little success. Our use case was to
load balance gRPC traffic amongst a small number of services on K8S.
Regardless of which load balancing algorithm we used or the amount of traffic
that was pushed at it we could not get ISTIO to actually spread the traffic in
any sort of reasonable pattern. For example, if we had 8 containers running 4
would get a ton of traffic, 2 would get a small amount, and 2 would just have
heartbeats.

We tried to get help, but the RocketChat channel was a ghost town and got
little response on the Google group. Lots of people asking questions but no
one answering. Some friends at Google told us there's an invite-only Slack
channel, but we could never figure out what secret incantation we would need
to do to get access there. Evidently only people at Google, IBM and Lyft are
allowed into that one.

I'll contrast this to the LinkerD2 (linkerd.io) project. We ripped out ISTIO
and installed LinkerD2 and it just worked. We didn't have to spend days hand
writing YAML files to configure our VirtualServices, DestinationRules,
Gateways, and ServiceEntries. One command and the whole thing was installed
and running. We injected the sidecar with their cool CLI utility and boom we
had a service mesh that actually load balanced gRPC properly. The LinkerD2
dashboard is a very powerful tool to visualize how your traffic is being
routed, response times, success rates, etc. When we had questions, we could
pop into the LinkerD2 Slack and someone from the team got right back to us
with suggestions and sincere offers to help. The CEO of Buoyant even reached
out personally to us, a fledgling Miami startup.

We also had issues where ISTIO would route requests to the wrong micro-service
at random. We would run the same request multiple times and get different
results every time. After moving to LinkerD2, those issues completely
disappeared.

LinkerD2 is still a very new project, but as far as we have experienced, it is
far more production ready than ISTIO. I can only imagine how far along
LinkerD2 would be if it was given a tenth of the resources that are pushing
ISTIO.

Our experience was back in the Fall of last year, but we tried ISTIO again
last week and had the exact same results on a new installation. Once again, we
had to pull ISTIO out and put LinkerD in and were able to get the system
functioning immediately.

ISTIO did do a great job of load balancing HTTP/HTTPS traffic, it's just gRPC
that is a long way off from being ready for prime time. Maybe if you have an
army of consultants at your disposal it will work for you, but as a startup
I'd stay away.

------
tak2siva
Another alternate for Istio would be to use a custom xds-server (control
plane) and Envoy as sidecars.

With few caveats

\- Can become a configuration nightmare if not managed properly

\- Not as sophisticated as Istio

On the positive side

\- There won't be a lot of moving parts as in Istio.

\- Envoy has excellent documentation for debugging

\- Get things done with all the perks of Envoy

~~~
tybit
I hope someone builds this out at some point, a no fuss alternative control
plane for envoy would be great.

~~~
tak2siva
I wrote one [https://github.com/tak2siva/Envoy-
Pilot](https://github.com/tak2siva/Envoy-Pilot)

Although my team is evaluating Istio, currently we are using this to keep
things minimal.

------
jxub
So this is like Kubernetes is to Docker, just to Kubernetes ;)

~~~
lcalcote
One way of interpreting this simile is to say that as just as one technology
has subsumed (become the focal point) another technology, so does a service
mesh as a layer on top of a container orchestrator. A fifth layer as described
in The Enterprise Path to Service Mesh Architectures -->
[https://layer5.io/books/the-enterprise-path-to-service-
mesh](https://layer5.io/books/the-enterprise-path-to-service-mesh)

------
wendydean
This seems very interesting but I'm also getting HTTP response codes. Also,
how does this compare to something like Nodered?

~~~
reitanqild
TBH node red always came across as a toy / prototyping tool.

Istio seems to be used in production a number of places.

Edit: also node red seems to be more about coding and istio seems to be more
about configuration.

