
Generating unit tests in Go - jyz
http://magictests.com/demo
======
andrewguenther
This is a really poor demo.

I opened up the demo page and generated the tests. On first glance, I see a
bunch of tests that just verify that the given code does exactly what it was
written to do. That misses the _entire_ point of tests. Oh, and only a single
test case for each function.

Looking a bit more closely, the test cases wouldn't even pass. They're just
filled with placeholder values. Okay, fine, the boilerplate is generated, but
you have to fill in the expected returns.

What about mocks? Why can I only have one test case per function? Also, None
of these test cases are documented. You're really still writing the bulk of
the tests yourself anyway.

I'm not saying there is no value here, but if there is value, that demo shows
none of it.

~~~
iofj
Yes, but when have you ever encountered a large company where unit tests are
judged by what needs testing as opposed to "coverage" ?

There is almost no use for unit tests, as they lock down function
implementations without verifying functionality. They have some amount of use
for a programmer to verify if what they wrote really is what they wrote, and
maybe for data structure methods (not in Go of course), but that means maybe 1
in 50 methods justifies a unit test.

Everything else should be system tested, to see if the components fit together
and if the interactions between various parts of the application really are
what you think they are.

It's like javadoc back in the day. "Document all your function variables,
you've only got them for x%", ... you write a parser that parses the app and
simply adds javadoc everywhere, giving the obvious descriptions to obvious
names. 2 hours of work for the generator, but actually interesting work, half
an hour to read through it and change a few things, and boom, 30000 lines
added in a day. And the worst part is, you've just made everything harder to
read, but everybody's happy with you.

~~~
bluejekyll
I'm sorry, but what?!

Unit testing is not an exercise in the mundane by making your manager happy,
it is about validating that the thing you just wrote actually works, so that
you can move on to the next thing, eventually aggregating all of that work
together and constructing code that works!

As others have said, you test inputs and outputs (and maybe you need to test
some failure modes depending on the complexity of the situation). When you're
working on a parser, or lexical analyzer of any sort, wouldn't it be nice to
know that it is capable of parsing what you thought it should?

System/BlackBox/Integration tests are too high-level and the more you rely on
those as your sole testing, the more brittle and difficult you make it to
track down the cause of test failures. It also means that you have to spend
more time constructing test scenarios to testing the inner-workings of the
system, things that are easier to do in unit tests. Bloated tests at that
layer actually make it MORE difficult to refactor.

Decent unit tests help you define the inputs and outputs of your
functions/classes/design. If it's hard to write a unit test on it, then you
probably have side-effects in the code that you can't properly account for
(bad code design). When it comes to a point that you need to refactor and the
unit test is in your way, then guess what? refactor/delete the test...

~~~
iofj
> As others have said, you test inputs and outputs (and maybe you need to test
> some failure modes depending on the complexity of the situation). When
> you're working on a parser, or lexical analyzer of any sort, wouldn't it be
> nice to know that it is capable of parsing what you thought it should?

That would be a system test. Nobody would write a parser in one function, so
it's full functionality can't be tested in a unit test. This is also exactly
the sort of test I'm advocating. The thing I'll guarantee you though : it
won't provide 100% code coverage.

Unit tests are this kind of crap, testing trivial things 100 times. You
immediately see why they provide 100% code coverage :
[https://en.wikipedia.org/wiki/Unit_testing#Design](https://en.wikipedia.org/wiki/Unit_testing#Design)

> the more brittle and difficult you make it to track down the cause of test
> failures

Yes ! This of course happens because they find ever-more subtle bugs, not
because there's something wrong with the test.

~~~
bluejekyll
> it won't provide 100% code coverage.

Yeah. 100% coverage is BS, again I go back to proof of quality.

> because the find ever-more subtle bugs

Well, yes and no. Yes integration tests are important for finding subtle bugs
between different components of a larger system. No, bc an over-reliance on
these instead of proper unit-testing, you end up with code bloat in your
integration tests, and a maintenance nightmare because the tests are more
complex. On top of that because you have so many integration tests It becomes
unclear when the integration test is testing a valid API construct, vs
something internal, making it very difficult to refactor code because you
don't know what's expected API response vs some odd internal bug that you had
to test for bc you didn't have a good unit test for the internals.

------
Zikes
This would be great if it were implemented as a "go generate" compatible
script or executable. [1]

[1] [https://blog.golang.org/generate](https://blog.golang.org/generate)

------
twic
I assumed this was a joke about how bad some unit tests are. Then i thought it
might be a satire on unit tests, from someone who really thinks they're
pointless.

Then i realised that this is a demo of a company that someone actually wants
to make their fortune with:

[http://magictests.com/](http://magictests.com/)

[https://angel.co/magictests](https://angel.co/magictests)

------
mholt
500 Internal Server Errors out the wazoo; e.g.
[http://magictests.com/static/js/bootstrap.min.js](http://magictests.com/static/js/bootstrap.min.js)

------
donatj
For some reason pasting code into it in chrome on Mac, it gets completely
mussed up.

