

Introducing the FreeBSD Test Suite - profquail
http://julipedia.meroh.net/2013/12/introducing-freebsd-test-suite.html

======
justincormack
For background, this is essentially the NetBSD test infrastructure ported to
FreeBSD. But it needs a lot more tests to be fully useful. Getting decent test
coverage for an OS (userspace and kernel) is pretty hard...

~~~
jmmv
Not exactly. NetBSD is still using the deprecated ATF tools in its test suite
while in FreeBSD I went straight to their replacement, Kyua.

Also, because FreeBSD uses Kyua, its test infrastructure can be composed of
test programs of various kinds (including legacy test programs and TAP-
compliant test programs) and not only ATF-based test programs. This is
something that NetBSD cannot do yet.

Note that NetBSD does have Kyua in the base system (optionally), but moving
the continuous testing systems over from the ATF tools is a stuck project. The
reason is that Kyua needs some more work to make that a reasonable migration
for the people that run such machines. In particular, the generation of
reports needs much improvement as well as the collection of results from
various hosts. See
[http://wiki.netbsd.org/kyua/import/](http://wiki.netbsd.org/kyua/import/) for
some details.

However, because FreeBSD needn't be tied to the deprecated tools, and because
I am the one setting things up from scratch, I am using this project as a
playground for the new framework. Doing this deployment in a major project
quickly surfaces all kind of big issues that need addressing -- and is
certainly a boost in willpower to fix them :-)

(Yes: an eventual goal is to also move NetBSD over to Kyua but that will have
to wait until Kyua proves itself "good enough" on FreeBSD's land. Homogenizing
the two systems will have the nice result of simplifying the sharing of
tests!)

~~~
justincormack
Yes I saw your talk at Eurobsdcon, need to talk to you sometime about adding
tests written in Lua to Kyua...

------
bch
I didn't see it in article, but Kyua is supposed to be pronounced "queue-ay"
(QA). Now you know.

Edit: Ha! I see Julio adjusted his stance on this:

The name is Kyua, which is just a play on the pronounciation of the Q.A.
acronym. Originally, my intention was to pronounce Kyua as Q.A., but in
reality this never happened. Today, just read the name as your instinct would:
"Kyu-ah".[1]

Oh well. Now we all know the backstory, and we can all talk like learned
experts to others.

[1] [http://wiki.netbsd.org/kyua/](http://wiki.netbsd.org/kyua/)

~~~
jmmv
Yes I did :) I found myself and everybody else never reading it like Q.A...
so, well, just use the more natural pronunciation!

~~~
dded
Reminds me of the scene from the Tom Hanks movie _The Wonders_ where the band
is originally cute with the spelling, Oneders, and an emcee introduces them as
the Oh-Nee-Ders.

For the opposite effect, does anyone else find themselves pronouncing _gnu_
(the animal) with a hard G, a la GNU?

~~~
bch
guh-new vs. new (referring to wildebeest) is something I remember from days
before rms' GNU even existed.

------
zachrose
Could anyone give some backstory to Joe Web Developer over here? What does it
mean to test an OS? Are these unit tests, or is it more like installing the
FreeBSD on hardware or a VM and verifying behavior?

