
Using “Git for data” to find and analyze test regressions - zachmu
https://www.dolthub.com/blog/2020-04-24-using-dolt-to-find-test-regressions/
======
zachmu
Blog author here. This post discusses how we use our git for data product,
Dolt, to store test results, and how this lets us use SQL queries to analyze
test regressions on our 6M+ test query dataset. Example:

    
    
      dolt_sql_performance> select nightly.query_string, nightly.error_message 
                         -> from nightly_dolt_results nightly join releases_dolt_results release 
                         -> on nightly.test_file = release.test_file and nightly.line_num = releases.line_num 
                         -> where nightly.result = 'not ok' and release.result='ok' limit 1;
      +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+
      | query_string                                                                                                                  | error_message                         |
      +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+
      | SELECT * FROM tab0 WHERE - + 63 IN ( col3, - col3 * col1 + + ( + + col4 ), 67 * col3 + - + CAST( - col1 AS DECIMAL ) + col0 ) | Unexpected error invalid type: string |
      +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+

------
detaro
hm. pushing from the test runners to a central place would make sense for
that, and could be quite interesting to keep track of some things, but it
doesn't seem like there's a viable way of doing that internally (=not relying
on some external service) yet?

------
timsehn
CEO of company that built this here. You need a versioned SQL database in your
toolbox.

