8 ms·
"Test what you fly and fly what you test" is what we live by at SQLite. Testing is done three times: (1) One test uses compiler and preprocessor options set up
by SQLite 7y ago
"Test what you fly and fly what you test" is what we live by at SQLite. Testing is done three times:
(1) One test uses compiler and preprocessor options set up to measure coverage. This is not so much a test of SQLite itself as it is a test of the test logic, to verify that the testing provides 100% MC/DC.
(2) The second test runs with assert(), ALWAYS(), and NEVER() enabled. This is like a unit test. It verifies internal assumptions and state, at the expense of running 3x slower.
(3) Finally, we build as for delivery and test once again. This is the string test, where we "test what we fly".
All three test runs must give the same answer (modulo performance) before a release.
During day-to-day development work, we usually run (2), but occasionally toss in a (1) or (3) just to confirm that we haven't introduced any gaps in test coverage, or bugs that are masked by the debugging logic.
So you are correct that you should not do all your testing using one configuration and then deliver a different build. But that does not mean you can't have extra logic in your code that helps do unit testing and debugging during development and which is excluded from release builds. You just need to make sure that you rerun all tests in the release configuration.