
How PostgreSQL is tested [video] - linuxhiker
http://blog.pgconf.us/2017/02/how-postgresql-is-tested.html
======
xwvvvvwx
Anyone feel like giving a summary?

~~~
stonemetal
Postgresql testing works the same way it has worked for the last 20-30 years.
Each test consists of a SQL script, an expected result file, and a result
file. A shell script runs the SQL script through the pg commandline, and dumps
the results to the results file. Diff then tells you if the test pass. Also
the shell scripts were rewritten in C for cross platform support. Not
available via SQL, ie vacum? No tests. Concurrent modifications? No tests.
Testing query plans that don't use order by have fun.

They have a new test suite built around TAP
([https://testanything.org/](https://testanything.org/)) and perl so now they
have automated tests for more things like certs, and replication.

~~~
ioltas
There are three basic suites of tests:

\- pg_regress, which is as you say a set of SQL scripts run on a running
server with generated output compared to the expected output. There could be
multiple expected outputs, see for example float tests.

\- pg_isolation_test, which is a part you are misunderstanding. This facility
has been introduced with SSI (serializable transactions) to test concurrency.

\- TAP tests (author of the replication/backup/recovery part here), which has
roughly three test suites:

\-- src/bin/ to test all the in-core binaries

\-- src/test/ssl to test SSL configurations. This has proved to be useful
lately when moving all the SSL parameters to be reloadable.

\-- src/test/recovery to test all the kind of replication scenarios.

There are as well modules in src/test/modules that use TAP tests to run
regressions on pg_dump, commit timestamps and extension interactions.

~~~
rattray
Thanks for the helpful detail!

Are there, in fact, no tests on VACUUM and others? Is that a problem? Are
there plans to add them? Would you be looking for contributors to do so?

~~~
ioltas
The query VACUUM is being tested via SQL. Now if you are referring to
autovacuum, there are no tests directly designed with this purpose in mind.
autovacuum is running in any instance running the regression tests as it is
always enabled by default.

