5 ms·

If you can prove that your testing process is perfect, then your entire development process can then be reduced to the following, after the test suite is writte

If you can prove that your testing process is perfect, then your entire development process can then be reduced to the following, after the test suite is written: cat /dev/random | ./build-inline.sh | ./test-inline.sh | tee ./src/blob.c && git commit -Am "I have no idea how this works, but I am certain that it works perfectly, see you all on Monday!" && git push production master --force When presented like this, relying on human intelligence and experience doesn't seem like such a bad thing after all. Just so we're clear, my username was not inspired by this scheme.

anamexis 9y ago
It seems to me that such a "perfect" testing process would basically amount to declarative programming.
- hgdsraj 9y ago
  Well...SQL is declarative, and we still see "Select * from Users"
- TeMPOraL 9y ago
  Indeed it is; a perfect test is equivalent to the code being tested.
ZeroGravitas 9y ago
Several other people are complaining about 100% coverage sometimes being misleading. One way to test your test is to randomly modify your code e.g. swapping a greater than for a less than. If your tests still pass, then they obviously missed this change in behaviour. This is known as mutation testing. https://en.wikipedia.org/wiki/Mutation_testing https://en.wikipedia.org/wiki/Mutation_testing I've tried it a few times, I generally found it too slow to be an everyday part of my routine, but interesting tool to have in your belt for e.g. evaluating test in a codebase you didn't write yourself.
- rjdavis3 9y ago
  Have you ever tried it with one of the automated tools? I use Pitest extensively to mutate my Java code bases and found it is not too much slower then regular line and branch code coverage tools when you use a history file so only diffs are needed. The trade off of a few more seconds of build time is worth the benefit of the coverage report and being able to fail the build if coverage drops too low for me. http://pitest.org/quickstart/maven/ http://pitest.org/quickstart/maven/

anamexis 9y ago

It seems to me that such a "perfect" testing process would basically amount to declarative programming.

hgdsraj 9y ago
Well...SQL is declarative, and we still see "Select * from Users"
TeMPOraL 9y ago
Indeed it is; a perfect test is equivalent to the code being tested.

ZeroGravitas 9y ago

Several other people are complaining about 100% coverage sometimes being misleading. One way to test your test is to randomly modify your code e.g. swapping a greater than for a less than. If your tests still pass, then they obviously missed this change in behaviour. This is known as mutation testing. https://en.wikipedia.org/wiki/Mutation_testing https://en.wikipedia.org/wiki/Mutation_testing I've tried it a few times, I generally found it too slow to be an everyday part of my routine, but interesting tool to have in your belt for e.g. evaluating test in a codebase you didn't write yourself.

rjdavis3 9y ago
Have you ever tried it with one of the automated tools? I use Pitest extensively to mutate my Java code bases and found it is not too much slower then regular line and branch code coverage tools when you use a history file so only diffs are needed. The trade off of a few more seconds of build time is worth the benefit of the coverage report and being able to fail the build if coverage drops too low for me. http://pitest.org/quickstart/maven/ http://pitest.org/quickstart/maven/