1. We have a lot of code paths, there isn't an integration test for everything. And for runtime, just because something is rarely called doesn't mean it never called; a bank can have yearly processes
2. We'll do service mesh too but this was probably an easier first step. We have rolled our own mesh with envoy sidecars and moving it towards istio style behaviour isn't trivial
> We have a lot of code paths, there isn't an integration test for everything.
This is just saying that writing "tests is hard". I agree, it is.
I always wonder about the value of static analysis. I've used it myself and have found bugs with it, but they are mostly easy bugs to find, and there's a lot of noise. The hard bugs are often only found with serious integration tests. I'll feel much more confident with a codebase that's 90% covered by tests than one that's statically analyzed.
You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer.
Dynamic analysis, instrumentation and intelligent logging are invaluable in catching a good chunk of the remaining non-trivial errors.
(I have a soft spot in my heart for fuzzing, which really should be considered as just another form of exploratory/destructive testing.)
> You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer.
I actually agree. But the implication is that static analysis should really live in the IDE, not the compiler or test suite. It's best when catching the silly whoopsy bugs as you're typing them. But I've yet to see a static analyzer that can catch race conditions, network or I/O errors, or something that can verify what you expect to happen actually does happen.
Thinking about static analysis on a higher level, arguably they shouldn't be necessary at all. If a system can statically analyze your code and find faults, then it implies that there is a deficiency in the language itself.
> We have a lot of code paths, there isn't an integration test for everything.
but you will have the service names that you're contacting configured somewhere (or in the worst case hardcoded). I would think this is much easier to analyze than the traffic flows.
It's slightly more complicated than that because you can import protos to use a constant or to consume an event from another service. But that is the gist
They say that they do both, though. First static analysis, then also log whatever requests that analysis missed, so that they can add it later on.
And they also add the static analysis to their CI pipeline, so that it can be checked then and there, not by obscure "happens once every half year during runtime" integration with a very uncommon service that they don't build very often.
1. Why was static analysis of the code chosen over observing the system during runtime and integration testing?
2. What was the reason rhe CNI layer was chosen for the implementation of this over the service mesh layer?
Something that really interests me about bazel/buck/pants/please is it automates #1 entirely with dep queries.