
Product Integration Testing at the Speed of Netflix - hepha1979
http://techblog.netflix.com/2016/07/product-integration-testing-at-speed-of-Netflix.html
======
joneholland
Usually Netflix engineering impresses me. This does not.

This sounds like all of the typical separate test org antipatterns that
healthy companies eventually eliminate by eliminating the entire test team.

Integrated tests are a scam. The only way to succeed in a micro services
architecture is with a strong focus on quality inside each service and
consumer driven contract testing between.

~~~
existencebox
I have to disagree with your last statement (integrated tests are a scam) very
strongly. Think about it as follows:

We assert unit tests are useful, because although we may "trust" primitives
(1+1=2) we can introduce semantic bugs in how we compose them.

Does it not also follow that if our primitives are now "microservice
functions" and our composition is the "integrated service", we should test the
composition for validity?

I truly believe in E2E testing, I could go on at length about esoteric bugs we
found only in interactions between properly functioning microservices (Hell
just today I watched a coworker work out a fun timezone doozie). One could
make a feasible argument that with a more well defined set of constraints and
capabilities, the microservice could have sufficiently covered unit test cases
for all possible interactions from a stub perspective, but pragmatically,
_this does not happen in the real world_, and E2E testing is a nice way of
making sure your product is indeed continuing to not be on fire (mostly).

One could feasibly (from my view of the world) say "Integration tests are a
cheap fix", to which I'd say "yes, yes they are" and then continue to perform
them, even if they are only in the form of a click through smoke test by hand
after a prod deployment :)

~~~
joneholland
JB says it better than I can:

[http://blog.thecodewhisperer.com/permalink/integrated-
tests-...](http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-
scam)

[https://vimeo.com/80533536](https://vimeo.com/80533536)

~~~
existencebox
He had me agreeing at first, but the blog at least kinda devolved into a
highly numbered breakdown of the impossibility of total test coverage. As I
said in my post, and the blog seemed to echo, I accept that an integration
test is a sign that unit tests are not sufficient, but I didn't see a
compelling argument in his blog as to how perfect unit test coverage
(sufficient to not integration test the _happy path_, not all things ever) is
obtained.

I certainly can see the situation he's describing, but it feels like he goes
on to expand it into a pathological strawman (ALL the unit tests EVERYWHERE
testing EVERYTHING run at EVERY BUILD) more than a situation you manage a
little more smartly. (As the oft-hammered quote, these things are all tools,
the moment one stops not being useful/stops being worth the cost, we stop
using it)

------
was_boring
Really disappointed everytime one of these types of articles comes out without
any working code to go with it. It's great that high level information is
being shared, but even better if the code was shared as well. It's not like
Netflix is in the business of selling testing services.

~~~
nitinreddy88
Exactly my thoughts. Its been same in many startups and companies. Whats
something they are uniquely doing is not mentioned at all

------
bballer
> "Secure Connection Failed"

Link is broken.

~~~
joshuacc
Looks fine here. Are you running HTTPS everywhere?

~~~
bballer
No. Doesn't appear to be an add-on problem. Firefox reports "Secure Connection
Failed" and chrome reports "This site can’t be reached".

Weird, just tested on Firefox for android (still connected to my local wifi)
and it works fine. Debug time.

