I've just spent some time modifying Open Babel's tests so that (a) they are all bundled in a single executable and (b) the parts of multi-part tests are called separately. The CTest docs are sparse on how to do this, so here is an attempt to describe the general gist of what I've done.
First of all, the relevant CMakeLists.txt is here. The following discussion changes some details for clarity.
This shows how to define the test names and consistuent parts:
set (cpptests automorphism builder ... ) set (automorphism_parts 1 2 3 4 5 6 7 8 9 10) set (builder_parts 1 2 3 4 5) ...For tests where a list of parts has not been defined we add a default of 1:
foreach(cpptest ${cpptests}) if(NOT DEFINED "${cpptest}_parts") set(${cpptest}_parts "1") endif() endforeach()Don't forget the .cpp files for each test:
foreach(cpptest ${cpptests}) set(cpptestsrc ${cpptestsrc} ${cpptest}test.cpp) endforeach()Each of these .cpp files should have a function with the same name as the file as follows:
int automorphismtest(int argc, char* argv[]) { int defaultchoice = 1; int choice = defaultchoice; if (argc > 1) { if(sscanf(argv[1], "%d", &choice) != 1) { printf("Couldn't parse that input as a number\n"); return -1; } } switch(choice) { case 1: testAutomorphisms(); break; // Add case statements to handle values of 2-->10 default: cout << "Test #" << choice << " does not exist!\n"; return -1; } return 0; }Now here's the magic. Using the filenames of the .cpp files, CMake can generate a test_runner executable:
create_test_sourcelist(srclist test_runner.cpp ${cpptestsrc} add_executable(test_runner ${srclist} obtest.cpp) target_link_libraries(test_runner ${libs})When it's compiled you can run the test_runner executable and specify a particular test and subtest:
./test_runner automorphismtest 1All that's left is to tell CMake to generate the test cases:
foreach(cpptest ${cpptests}) foreach(part ${${cpptest}_parts}) add_test(test_${cpptest}_${part} ${TEST_PATH}/test_runner ${cpptest}test ${part}) set_tests_properties(test_${cpptest}_${part} PROPERTIES FAIL_REGULAR_EXPRESSION "ERROR;FAIL;Test failed" endforeach() endforeach()Now, when you run "make test" or CTest directly, you will see the test output:
C:\Tools\openbabel\mySmilesValence\windows-vc2008\build>ctest -R automorphism Test project C:/Tools/openbabel/mySmilesValence/windows-vc2008/build Start 6: test_automorphism_1 1/10 Test #6: test_automorphism_1 .............. Passed 2.83 sec Start 7: test_automorphism_2 2/10 Test #7: test_automorphism_2 .............. Passed 0.31 sec Start 8: test_automorphism_3 3/10 Test #8: test_automorphism_3 .............. Passed 0.13 sec Start 9: test_automorphism_4 4/10 Test #9: test_automorphism_4 .............. Passed 0.10 sec Start 10: test_automorphism_5 5/10 Test #10: test_automorphism_5 .............. Passed 0.15 sec Start 11: test_automorphism_6 6/10 Test #11: test_automorphism_6 .............. Passed 0.12 sec Start 12: test_automorphism_7 7/10 Test #12: test_automorphism_7 .............. Passed 0.11 sec Start 13: test_automorphism_8 8/10 Test #13: test_automorphism_8 .............. Passed 0.12 sec Start 14: test_automorphism_9 9/10 Test #14: test_automorphism_9 .............. Passed 0.10 sec Start 15: test_automorphism_10 10/10 Test #15: test_automorphism_10 ............. Passed 0.12 sec 100% tests passed, 0 tests failed out of 10 Total Test time (real) = 4.45 sec
2 comments:
If you take a look at the add_test documentation (cmake --help-command add_test will give you that if you have CMake installed), you should prefer the new signature as that will resolve targets rather than executable names. So your add_test command would look something like,
add_test(NAME name_to_show_up_friendly
COMMAND target_name arg1_test_name)
The test runner code is really useful, and something that can speed up project builds as it reduces the amount of linking etc that must be done.
Thanks Marcus - I might well add some friendly names.
As you can see, we are finally getting around to including some CMake 2.6 features. Anything else we should be thinking about?
Post a Comment