
BDD testing for golang: Ginkgo and Gomega - tissarah
http://pivotallabs.com/announcing-ginkgo-and-gomega-bdd-style-testing-for-golang/
======
buro9
I wouldn't mind a BDD tool written in Go, but not for this purpose.

What I would like is a tool that allowed testers and QA people to define tests
using BDD for RESTful APIs.

Something along the lines of this syntax:

    
    
        Using https://example.com/myapi
            With access_token [accessToken]
            With content_type application/json
            When I create:
            {
              "somejson": true
              "foo": "bar"
            }
                Expect status 200
                Expect returnData.somejson = true
                Expect returnData.foo = "bar"
                Store returnData.id as myid
        And
        Using https://example.com/myapi/{{myid}}
            When I get
                Expect status 200
                Expect returnData.somejson = true
                Expect returnData.foo = "bar"
        And
            With access_token [accessToken]
            With content_type application/json
            When I update:
            {
              "somejson": true
              "foo": "foo"
            }
                Expect status 200
                Expect returnData.somejson = true
                Expect returnData.foo = "foo"
                Store returnData.id as myid
        And
            With access_token [accessToken]
            When I delete
                Expect status 200
        And
            When I get
                Expect status 404
    

I've seen a few things like this written in NodeJS, such as apieasy and vows.
But nothing with this level of succinctness.

And I'd like it to be in Go as I'd like to just give a single binary to the
testers (rather than an "install node, get project, fetch dependencies,
resolve any issues, run the project").

When I read the headline, this is roughly what I hoped for. If I can find the
time (unlikely), I'd build this. But then... maybe someone reading this knows
something very close to this.

~~~
pjmlp
At the boring enterprise projects we just use SoapUI for this type of testing.

------
redbad
I can't help but feel like the authors of this tool (and ones like it) are
fundamentally misunderstanding either the language, the purposes of testing,
or maybe even static type systems in general.

~~~
shoo
if you are willing to go into more detail, i'd be interested in hearing what
these fundamental misunderstandings are.

~~~
afandian
(Note I am not the GP)

This feels weird. I have a gut reaction to this, and it's conflicted. Perhaps
the same feeling `redbad` has, perhaps not.

On the one hand, I feel like this is a million miles away from the Go I know
and love. I've used Ruby, Python, C#, Java, C, Obj-C, JavaScript etc in the
past. In terms of mental processes, the way I write go is closest the way I
would write C or Java.

I write tests that do the same thing as this, that have pre and post
conditions and assertions and explanations in comments. The whole point of
static type systems, surely, is that you represent things (constraints, state,
possible values, etc) in code rather than comments and rely on the language to
enforce them rather than the next developer to read your comments, so the
explosions happen at compile time rather than production. So, by that token,
this kind of DSL seems like a step in the direction of static typing, rather
than the other way round.

I've not looked at the source, but the line
`Expect(scoreKeeper.Stats["Manning"]["Touchdowns"]).To(Equal(1))` might use
all kinds of clever closure stuff but ultimately wouldn't compile if the right
interfaces weren't satisfied. Unless there are `interface{}`s all over the
place, which would be regrettable.

On the surface, this seems like a step toward Ruby with its endless DSLs which
feels like the antithesis of static typing. That naturally feels a bit weird
and a bit of a culture lurch. But perhaps the DSLs are closer to those of
Scala than Ruby.

Both feelings I get are fuzzy and not particularly arguable, but I think
peoples' relationships with languages are very personal and that's such a
fundamental quality of the PL landscape that it's worth not discounting.

~~~
shoo
(context: i've been working mainly with python, and have almost no experience
with go)

My initial reaction to the example was that it looked like something that
could be easily unit-tested, without the added layer of BDD over the top. But
that's probably just a symptom of the example appearing as something that
would be simple to build and test.

E.g. in python i'd translate
`Expect(scoreKeeper.Stats["Manning"]["Touchdowns"]).To(Equal(1))` to `assert
scoreKeeper.Stats["Manning"]["Touchdowns"] == 1` which is essentially the
same, but only uses one bit of machinery (assert) rather than three.

That said, the more ways to test things the merrier. I think if you are
interested in testing larger chunks of functionality / user-facing bits of
behaviour rather than invariants of the building blocks of your program, then
perhaps BDD starts to look more appealing, so the two approaches seems
complementary.

Having an expressive static type system is again complementary, as that
prevents you from making many trivial errors, and you just have to test for
the remaining ones that the type system cannot express. (i miss that in
python)

after a little dig through the code, it looks like "EXPECT foo TO EQUAL barr"
translates as

[https://github.com/onsi/gomega/blob/master/gomega.go#L15](https://github.com/onsi/gomega/blob/master/gomega.go#L15)
[https://github.com/onsi/gomega/blob/master/actual.go#L27](https://github.com/onsi/gomega/blob/master/actual.go#L27)
[https://github.com/onsi/gomega/blob/master/actual.go#L48](https://github.com/onsi/gomega/blob/master/actual.go#L48)
[https://github.com/onsi/gomega/blob/master/matchers.go#L7](https://github.com/onsi/gomega/blob/master/matchers.go#L7)
[https://github.com/onsi/gomega/blob/master/matchers/equal_ma...](https://github.com/onsi/gomega/blob/master/matchers/equal_matcher.go)

so there's lots of `interface{}`s and reflection and so on

~~~
karma_fountain
Unless I'm missing something, assert
scoreKeeper.Stats["Manning"]["Touchdowns"] == 1 is not the same unless in
python you have some magic reflection. The output you get from that is just
pass/fail. The output you get from the matcher method is the expected and
actual values in a much nicer error message.

~~~
shoo
you (& afandian) make a good point, and i believe that's roughly how py.test
does it, by rewriting the code of test modules before running tests. which
would be fair to call magic.

[http://pytest.org/latest/assert.html#advanced-assertion-
intr...](http://pytest.org/latest/assert.html#advanced-assertion-
introspection)

------
Perceptes
Looks great. I'm a huge fan of RSpec and really miss it in other languages. It
didn't occur to me before reading this post, but the ability to implement a
testing framework like RSpec is a great way to gauge the expressiveness of the
language.

------
karma_fountain
Interesting choice of the Omega symbol for the actual symbol. Not sure if I
like that or not. Vim users can use CTRL-K W * in insert mode to get a nice
block character (at least on my build of vim) on their screen.

I prefer to use Gherkin syntax for by bdd testing, but the matchers are nice.
The bootstrapper and helpers seem to fail hopelessly on windows, and the
console writer outputs a lot of control characters which do not work in
windows command prompt.

------
pjvds
Here are 3 alternatives:

1\. Mao: [https://github.com/azer/mao](https://github.com/azer/mao)

2\. Zen:
[https://github.com/pranavraja/zen](https://github.com/pranavraja/zen)

3\. GoConvey:
[https://github.com/smartystreets/goconvey](https://github.com/smartystreets/goconvey)

------
SmileyKeith
I love that Go is getting the kind of attention and use that leads to the
creation of a framework like this from a relatively well known company.
Although I can agree with some other comments here that it doesn't
particularly feel native to Go.

------
programminggeek
This looks pretty cool, but for some reason I tend to like using the built in
tools. Maybe I hate external dependencies or I have trust issues, but I'm way
less likely to use this for that reason alone.

------
maurycy
Please no with this Ruby madness in the Go world.

------
okpatil
First of all, good job.

Are you planning to add selenium support?

------
mongrelion
This is totally awesome.

------
onsi
Hey all, this is the author of Ginkgo & Gomega. I'm excited to see all this
thoughtful conversation and figured I'd chime in with a response to keep the
conversation going. (Apologies for the long post!)

\- Godoc documentation is coming soon. Godoc is great for API-style
documentation but not particularly appropriate for
tutorial/structured/narrative-style documentation. Think golang.org/pkg vs
golang.org/doc. I wanted to write the /doc first as I believe it to be more
valuable for beginners and is often overlooked by the go community. /pkg is
coming soon (a few days) and is much easier to write.

\- @buro9: Golang is BDD-style in the same way that Cedar and Jasmine are BDD-
style. This is not Cucumber (way too much DSL for me!) With that said,
asynchronous testing support is baked right into Ginkgo and Gomega so the API-
testing that you'd like would be easily expressed in Ginkgo and Gomega.

\- Assertions that I have fundamentally misunderstood Golang or the
distinction between dynamic and static typed languages are interesting and I'd
like to address some of them.

BDD-style testing need not be limited to dynamically-typed language. Yes
dynamically-typed languages need far more comprehensive test coverage to make
up for the missing compiler. That's why I tend to prefer statically-typed
languages. To me BDD-style (vs the XUnit style) isn't primarily about
addressing these deficiencies in dynamic languages -- it's about
_expressiveness_.

To that end, I think that BDD and Golang go hand in hand. Let me explain. BDD
is exceptionally good at describing the behavior of branching code. Golang is
filled with branching code that needs to be described in test. Here's a
classic example in pseudo-go:

    
    
        func DoSomethingAwesome(dependencyA DepA, dependencyB DepB) awesome Awesome, err error {
            stuffA, err := dependencyA.Gimme()
            if err != nil {
                return Awesome{}, err
            }
            stuffB, err := dependencyB.Gimme()
            if err != nil {
                return Awesome{}, err
            }
            ....
            return awesome, nil
        }
    

With Ginkgo you can cover these branches expressively:

    
    
        Describe("Doing something Awesome", func() {
            BeforeEach(...) //common happy case setup
    
            Context("When all is well", func() {
                It("should be awesome", ...)
            })
    
            Context("When dependencyA fails", func() {
                BeforeEach(...) //set dependencA up for failure
            
                It("should return a zero Awesome", func() {
                    ...
                    Expect(awesome).To(BeZero())
                })
    
                It("should error", func() {
                    ...
                    Expect(err).To(HaveOccured())
                })
            })
    
            Context("When dependencyB fails", func() {
                //etc...
            })
        })
    

Compare this to the XUnit style:

    
    
        func TestDoingSomethingAwesomeWhenAllIsWell() {
            //setup
            //some sort of assertion
        }
    
        func TestDoingSomethingAwesomeWhenDependencyAFails() {
            //setup + tweak
            //some sort of assertion that awesome is zero
            //some sort of assertion about err
        }
    
        func TestDoingSomethingAwesomeWhenDependencyBFails() {
            //setup + tweak
            //some sort of assertion that awesome is zero
            //some sort of assertion about err
        }
    

I prefer the former (note: that's a subjective statement). IMO there's nothing
fundamentally more Golangish about the latter (if anything the fact that
func's that begin with Test are special is kinda weird), and since Gomega's
matchers are fluent in Go they know about things like zero values and errors
having occured. Moreover, both Ginkgo and Gomega have been built to make
testing asynchronous code (read: concurrent goroutines) easy and expressive.
This isn't a carbon copy of RSpec/Cedar/Jasmine, it's a synthesis of the best
ideas from those testing frameworks expressed in ways that cater specifically
to Golang.

\- Concerns about using interface{} and reflection seem odd to me. First off,
the Go authors provide both of these things to precisely solve the sort of
problem that Gomega is trying to solve. Nobody wants to write and maintain
matchers that look like:

    
    
        Expect(foo).To(EqualInt64(3))
        Expect(bar).To(EqualString("blarp"))
        Expect(baz).To(EqualMyCustomAwesomeType(awesome))
    

besides the reflect package's `DeepEqual` does a _great_ job comparing
`interface{}` to `interface{}` correctly.

Most importantly of all: we're all using interface{} _all_ the time:
`fmt.Sprintf("%d %f %s %v", ...)`!

And thanks for seeing the point @karma_fountain: Gomega's matchers have
excellent reporting about what precisely went wrong when a matcher fails. Bare
assertions lack this and a look through the Go tests shows a lot of
reinventing-the-wheel to provide what probably amounts to inconsistent error
output to the developer. Why not put that all in one place, call it a matcher
library, and make it _dead easy_ to write custom matchers?

\- And @shoo: yes, thanks for pointing it out: the example is somewhat
fabricated and isn't a compelling argument for BDD vs XUnit. It's hard to cook
up a good BDD example in very short space!

~~~
redbad

        > Nobody wants to write and maintain matchers that 
        > look like: Expect(foo).To(EqualInt64(3))
    

And nobody does do that. Idiomatic Go is to write

    
    
        if foo != 3 {
            t.Errorf("foo: expected 3, got %d", foo)
        }
    

I truly cannot explain why _so many people_ find this style of testing _so
objectionable_ that they invent entire DSLs to avoid it.

~~~
onsi
Doing this for every test:

t.Errorf("foo: expected 3, got %d", foo)

gets old quickly.

