
How SQLite Is Tested - deegles
http://www.sqlite.org/testing.html
======
voltagex_
Previous discussions:
[https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Fte...](https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Ftesting.html&sort=byPopularity&prefix&page=0&dateRange=all&type=story)

------
antonios
This post is the most reliable karma generator I've seen on HN.

------
a3_nm
It's interesting that, even though sqlite is free software (it is even public
domain), some of the tests are proprietary:
[https://www.sqlite.org/th3.html](https://www.sqlite.org/th3.html). I wonder
what's the goal of this...

~~~
acqq
How about the goal of allowing the author to actually earn something?

~~~
hueving
And how exactly would the author do that?

~~~
acqq
The author is selling the support to the companies that can request some
changes. Having the best tests, he can claim that the modifications he
introduces are "production ready" and charge for that too. Others can just say
"we changed something and it works maybe."

Just writing the code is significantly easier than having it verified and
tested.

See specifically:

[http://www.sqlite.org/th3.html](http://www.sqlite.org/th3.html)

"SQLite itself is in the public domain and can be used for any purpose. But
TH3 is proprietary and requires a license. Members of the SQLite Consortium
get free and unlimited access to TH3. Others can contact the SQLite developers
for information on how to obtain a license to access and use TH3."

------
michaelmcmillan
Does anyone know how long time it takes to run these tests?

~~~
SQLite
The minimum set of tests needed to provide 100% branch coverage takes about 3
minutes on a fast machine.

To run all of the tests under various compile-time options and on all
supported platforms usually takes two to three days, assuming everything
works. A full-up automated test on just Linux x64 takes about 18 hours.

~~~
delinka
Does this include the tests that run on other database servers?

~~~
SQLite
The sqllogictest suite
([https://www.sqlite.org/sqllogictest](https://www.sqlite.org/sqllogictest))
only has to be run on all the various servers when the tests themselves
change, in order to validate the tests. Otherwise, those tests only need to be
run on the system under test, since the tests themselves contain the correct
answer to check against.

The sqllogictest suite takes about 15 minutes to run three times against
SQLite (once with all optimizations turned on, and twice more with various
optimizations turned off) on a fast machine.

------
NelsonMinar
I wonder if this test suite will be the last known use of TCL in modern code?

~~~
kijin
AFAIK the test suite for Redis is also written in TCL.

~~~
bch
Indeed - both antirez and drh are big Tcl users. drh is a Tcl Core Team (TCT)
member emeritus, and antirez is responsible for this Tcl love letter:
[http://antirez.com/articoli/tclmisunderstood.html](http://antirez.com/articoli/tclmisunderstood.html)

