
Greatest: C unit testing library in one file - luu
https://github.com/silentbicycle/greatest
======
awda
More sophisticated C unit testing libraries (libcheck, for example) use nice
compiler features beyond ANSI C, like linker sets, in order to avoid error-
prone duplicating like you see in these examples.

While that may not work in some environments that don't support these
extensions, I'm having trouble imagining ones that support C stdio but do not
support linker sets.

~~~
silentbicycle
I (the author) have been doing a lot of embedded development lately, and in
those cases you really can't assume very much beyond core ANSI C features.
That's also why greatest doesn't do any dynamic allocation, at all. There's a
relatively small struct for the current testing progress, and everything else
can easily run out of ROM.

My primary goal with greatest is to assume as little as possible, and avoid
imposing any additional constraints on projects just because people want to
test them. I also want the implementation to be very transparent -- it's just
a 600-ish line header file, carefully documented. It's a major pet peeve of
mine when I have to work around features that try to automatically be helpful,
but have edge cases that break down in opaque ways. (Build systems tend to be
particularly bad in that way.)

Also, I'm beyond shocked that the name wasn't already taken. ;)

One of my upcoming tasks is making stdio _optional_ in greatest, because
printf doesn't do you a whole lot of good if you're running it on an Arduino
or something, but in those contexts automated testing is especially valuable.

~~~
meekrosoft
Cool, I really like this! I wrote a popular C function faking framework in a
single header file for exactly the same reasons: all bets are off in embedded
targets.
[https://github.com/meekrosoft/fff](https://github.com/meekrosoft/fff)

The test C test framework in the project is 4 lines long, but does require the
standard library.
[https://github.com/meekrosoft/fff/blob/master/test/c_test_fr...](https://github.com/meekrosoft/fff/blob/master/test/c_test_framework.h)

~~~
silentbicycle
fff looks cool, too. Thanks! I'll check it out. :)

I've tended to use CMock
([https://github.com/ThrowTheSwitch/CMock](https://github.com/ThrowTheSwitch/CMock))
for mocking, though it depends on some Ruby-based tooling. The last time I
looked into replacing CMock I wound up hip-deep in the binutils. (It was fun
when I stopped getting google results from StackOverflow and started getting
them from Phrack, though. ;) )

~~~
justinmk
Did you try cmocka? [http://cmocka.org/](http://cmocka.org/)

It was the most featureful that I found when I was looking last year.

~~~
silentbicycle
I have not, but I'll check it out, thanks.

------
silentbicycle
Author of greatest, here.

I wrote a blog post about it a couple months ago:
[http://spin.atomicobject.com/2013/07/31/greatest-c-
testing-e...](http://spin.atomicobject.com/2013/07/31/greatest-c-testing-
embedded/)

~~~
exDM69
Nice unit test framework you have written, exactly what I have been looking
for my C projects. I, too, dabble in embedded environments and can't rely on
big libraries.

One question/suggestion, though. In the end of all the samples you need
GREATEST_MAIN_DEFS/main()/GREATEST_MAIN_BEGIN/RUN_SUITE(xxx)/GREATEST_MAIN_END.
Could this be a simple oneliner like GREATEST_MAIN(xxx)?

I do understand that this is probably required for embedded systems where you
might not have a regular main(), but I would just like to write
GREATEST_MAIN(xxx). Does this make sense to you? Are there use cases where it
makes sense to do something else in main()?

Anyway, good stuff!

~~~
silentbicycle
Thanks!

I use GREATEST_MAIN_BEGIN and GREATEST_MAIN_END so that people can put other
things in main, around them. Also, there are often several RUN_SUITE calls,
and I can't depend on vararg macros portably without also depending on C99.
While there are some ways to work around that, they don't deliver enough value
to justify the added complication.

~~~
exDM69
Yeah, I just realized that you can't really recurse with macros without doing
nasty repetition, so it is not possible to do GREATEST_MAIN(suite1, suite2,
suite3, etc), not even using C99 variadic macros.

Had it been possible, it would definitely make sense to add this oneliner
utility. Since it would end up being very limited, it probably doesn't.

~~~
silentbicycle
I think it _could_ work like this:

    
    
        SUITE_LIST(suites) {
            suite1,
            suite2,
            // ...
        };
    
        GREATEST_MAIN(suites);
    

but, again, I don't know if that's necessarily an improvement. RUN_SUITE is
encapsulating saving the suite's name, optionally only running suites with
names that match a certain pattern, and some other features that become tricky
to implement with just a statically allocated function pointer array. I'm also
trying to avoid doing anything "clever" with macros.

------
abtinf
C unit testing framework in 2 defs:

[http://www.jera.com/techinfo/jtns/jtn002.html](http://www.jera.com/techinfo/jtns/jtn002.html)

~~~
silentbicycle
I wrote greatest because minunit doesn't provide any of the infrastructure
that nearly every test runner grows eventually, in some ad hoc way. Halting
after the first error is just not good enough after projects grow beyond a
certain size. Also, being able to run just specific suite(s) or test(s) by
name patterns from the command line really speeds up one's workflow.

I still wanted something as straightforward to use as minunit, though. Not
knocking it. I'd rather use minunit than something that imposed its own
architectural opinions on my projects, or consumed resources already in short
supply. Also, I make a lot of small C utilities and libraries, and having to
define a "project" just to have tests adds a bit too much friction.

(I wrote a bit more about that here:
[http://spin.atomicobject.com/2013/07/31/greatest-c-
testing-e...](http://spin.atomicobject.com/2013/07/31/greatest-c-testing-
embedded/) )

~~~
abtinf
Agreed. I've skimmed the source and it looks like it would be much easier to
integrate into a build system as well.

I find Minunit helpful because its easy to memorize and dump into a new
project for quick testing. Its brevity also seems to help introduce c devs to
unit testing.

------
aslakhellesoy
Joe Walnes' tinytest is just a 113 line header file:
[https://github.com/joewalnes/tinytest](https://github.com/joewalnes/tinytest)

Greatest is 591 lines .h and 133 lines .c.

You can see tinytest in action here:
[https://github.com/cucumber/bool/blob/master/c/test/test.c](https://github.com/cucumber/bool/blob/master/c/test/test.c)

~~~
silentbicycle
They've got a lot in common. tinytest looks a bit simpler, but also doesn't do
any allocation and looks reasonably portable. Cool.

The single biggest difference is that greatest has more control over about
which test(s) it runs. tinytest just bails at the first failure, whereas
greatest can run everything and report, or individual test(s) or groups of
tests whose names match a substring. That adds some to the implementation, but
I've found it particularly valuable once a program has several modules and
hundreds of tests.

Also, greatest supports parametric testing, which means that adding
fuzzing/randomized testing is pretty straightforward (though usually project-
specific).

tinytest requires C99, greatest doesn't. That _shouldn 't_ matter, but
unfortunately sometimes it does. Embedded compilers for proprietary
architectures can be lacking, in particular.

greatest also doesn't assume ANSI escape sequences are okay. Sometimes dumping
out a bunch of [1;31m stuff is annoying. That's a matter of taste, though, and
it'd be easy to make it optional.

------
dasmithii
Macros are surprisingly useful in the unit testing context. Whether or not
it's considered good practice, I like to embed my tests within the header
files that they're associated with. With macros and gcc's -D flag in order to
generate production and testing builds separately.

I'm new to the world of C, is this typical?

~~~
simias
It's fairly typical. The standard "assert" macro behaves exactly that way, it
does not generate code if the code is compiled with NDEBUG defined.

Of course you shouldn't abuse it but in this instance it's perfectly fine
IMHO.

This reminds me that in addition to that the linux kernel has a pretty cool
feature for debugging called "dynamic debugging" where you can enable and
disable "printk" (the kernel's version of printf) messages dynamically from
the shell through the debugfs:

[http://lwn.net/Articles/434833/](http://lwn.net/Articles/434833/)

------
mratzloff
I'm using Seatest[0] for a hobby project I'm writing. I was looking for simple
xUnit-style assertions. Works pretty well out of the box, but I did modify it
to make it ANSI C compatible and change the output. (I want colored terminal
output, darn it! Green if OK, red if not!)

[0] [https://code.google.com/p/seatest/](https://code.google.com/p/seatest/)

------
thirdsight
I've never used a unit test framework for C ever.

I usually write a test exerciser for the library/command and a "known good
output" file then diff the output and the known good.

will take a look at this for certain.

------
habitue
I used to use FCTX for unit testing, which was just a header file. I don't
know if it used nonstandard compiler features under the hood. Unfortunately,
while it's mirrored, it doesn't look like the main development website is
around any more.

Github mirror: [https://github.com/imb/fctx](https://github.com/imb/fctx)

~~~
RhysU
+1 on FCTX, even in its dormant state. A sample:
[http://agentzlerich.blogspot.com/2011/01/c-header-only-
unit-...](http://agentzlerich.blogspot.com/2011/01/c-header-only-unit-testing-
with-fctx.html)

------
norswap
In the same flavour:
[https://github.com/orangeduck/ptest](https://github.com/orangeduck/ptest)

------
Mo1oKo
In our team we relay on Unity for our unit tests. We do embedded C and try to
practice TDD as much as possible. The author claims it depends on Ruby and
such to work. I wanted to point out this is not the case, and that actually we
are using it 'clean' with no trouble.

~~~
silentbicycle
Unity alone doesn't, but ceedling and CMock do. I was referring to them as a
group, because I usually see them used together, but you're right.

(I work with most of the maintainers of those tools, and have some minor
commits myself.)

------
puzz
Yet another very simple single-file in 65 lines of code here:
[https://github.com/tkrajina/yactest](https://github.com/tkrajina/yactest)

------
kabouseng
CppUTest isn't bad for C projects, even if the test framework uses C++.
[http://cpputest.github.io/](http://cpputest.github.io/)

edit - grammar

~~~
kabouseng
Sorry I don't want to hijack the authors effort, I am just mentioning it for
anybody considering additional testing frameworks.

------
NigelTufnel
Question to professional C programmers: Do you write unit tests?

At my current job I've never met C++, let alone C programmer who writes unit
tests. Is it common?

~~~
silentbicycle
In my experience, unit testing is more common in embedded C development,
because the code is held to much higher standard of accountability.

Sloppy embedded code can _kill_ people, or at the very least lead to
incredibly expensive hardware recalls. With the exception of poorly thought-
out privacy policies, it's unlikely that errors in a web app can cause that
level of problems.

It's also much easier to deploy an update to a web app than a Mars rover or a
car's brake controllers.

------
SvenDowideit
will you be adding it to CCAN (the C code archive, in a similar spirit to
Perl's CPAN)?

see [http://ccodearchive.net/](http://ccodearchive.net/)

------
gcb0
this is a bloated minunit :)

just the short of thing a QA dev would do to keep his job.

i kid. kid. It is nice, but overkill in my humble opinion. Failing on the
first error for unittest is a feature. because you are not supposed to even
commit it if it is failing, so no need to fancy graphics and reports. It
either passed, or not.

~~~
exDM69
> Failing on the first error for unittest is a feature. because you are not
> supposed to even commit it if it is failing, so no need to fancy graphics
> and reports.

A test suite may run for a long time and bailing out at first failure is a
very stupid thing to do. It may be nice to have it as an _option_ but not the
only way of doing it.

Even a humble test suite might take 5 minutes to run, going to grab some
coffee and coming back to see the first test fail would be a bad thing. A big
test suite might take hours on a CI server, having it fail on the first test
would be a major waste of time.

And with a modern distributed version control, there's no reason why you
shouldn't commit code (not to the master branch!) that doesn't pass tests. To
share it with your team mates or push the code to be tested on a CI server,
for example.

~~~
gcb0
if you unit tests take that long, break it up.

or you are probably doing functional testing in your units.

