test

CamiTK CMake macros used for testing. More...

Modules
	level
	CamiTK CMake macros used generating the mediawiki tab containing the list of all functionnal tests, classified by CamiTK extensions with their given LEVEL.

Functions
	camitk_add_integration_test ()
	camitk_add_integration_test is a macro to add a new test to the CTest infrastructure It encapsulates CMake add_test and use the action state machine to run a set of actions. More...
	camitk_add_test ()
	camitk_add_test is a macro to add a new test to the CTest infrastructure It encapsulates CMake add_test and adds useful way of testing programs. More...
	camitk_additional_action_test ()
	camitk_additional_action_test is a macro to create additional level1 test of CamiTK action to the CTest infrastructure. More...
	camitk_init_test ()
	camitk_init_test is a macro to initialize a group of test (for the same command) It is used to initialize a series of tests for a given target. More...
	set (CAMITK_TEST_COMMAND_FILE ${CAMITK_TEST_OUTPUT_DIR}/command) set(CAMITK_TEST_COMMAND_RESULT_FILE $
	This CMake script run the action state machine and compare the output files to the expected files The intended use is to run this script from the camitk_add_test_actionstatemachine. More...

Detailed Description

CamiTK CMake macros used for testing.

Function Documentation

camitk_add_integration_test()

camitk_add_integration_test

(

)

camitk_add_integration_test is a macro to add a new test to the CTest infrastructure It encapsulates CMake add_test and use the action state machine to run a set of actions.

The set of actions are described in a CamiTK SCXML document. This macro is used to add integration tests.

Details on the runned test can be found in directory ${CMAKE_BINARY_DIR}/Testing/Temporary/${TYPE_EXTENSION}-${EXTENSION_NAME}-integration-test

It is typically usued inside the camitk_extension(..) macro and expects the following variable to be set beforehand:

• TYPE_EXTENSION action or component)
• EXTENSION_NAME name of the current action or component

It does add a test if and only if:

• ${CMAKE_CURRENT_SOURCE_DIR}/integration-testdata exists
• ${CMAKE_CURRENT_SOURCE_DIR}/integration-testdata contains an file called "asm-input.scxml"
• ${CMAKE_CURRENT_SOURCE_DIR}/integration-testdata contains at least one file name "output-*.*"

Usage:

camitk_add_integration_test()

camitk_add_test()

camitk_add_test

(

)

camitk_add_test is a macro to add a new test to the CTest infrastructure It encapsulates CMake add_test and adds useful way of testing programs.

It cannot operate on its own, you need to call camitk_init_test first (and only once) before calling camitk_add_test

Details on the runned test can be found in ${CMAKE_BINARY_DIR}/Testing/Temporary/target# where target is the executable name (see camitk_init_test() macro), and # is the test order number.

Usage:

camitk_add_test(EXECUTABLE_ARGS "arg1 arg2 arg3"
                PASS_FILE_OUTPUT pristineOutput
                PASS_REGULAR_EXPRESSION regexp
                FAIL_REGULAR_EXPRESSION regexp
                PROJECT_NAME name
                TEST_SUFFIX name
)

Parameters

EXECUTABLE_ARGS	(optional) The executable arguments (all in double quotes), typically each test will have different arguments. Can be empty
PASS_FILE_OUTPUT	(optional) If specified the test to perform is to compare the output of the command to this file. It the output is the same, then the test is passed, otherwise it is failed.
PASS_REGULAR_EXPRESSION	(optional) This is equivalent to "PASS_REGULAR_EXPRESSION regexp" property for the test, see http://www.cmake.org/Wiki/CTest:FAQ#My_test_does_not_return_non-zero_on_failure._What_can_I_do.3F
FAIL_REGULAR_EXPRESSION	This is equivalent to "FAIL_REGULAR_EXPRESSION regexp" property for the test, see http://www.cmake.org/Wiki/CTest:FAQ#My_test_does_not_return_non-zero_on_failure._What_can_I_do.3F
PROJECT_NAME	Base name for the test, can be used for ctest -S
TEST_SUFFIX	Suffix added to the test name, just after PROJECT_NAME and before the test index

If no argument are given, it does the equivalent as add_test(name target)

Note

You can only choose one between nothing, PASS_FILE, PASS_REGULAR_EXPRESSION and FAIL_REGULAR_EXPRESSION

Example invocation:

add_executable(myprogram)

...
# Start the test series for myprogram
camitk_init_test(myprogram)
camitk_add_test(EXECUTABLE_ARGS "-a inputfile.xml -c" PASS_FILE pristineOuputToCompareTo.xml) # will be called myprogram1
...
camitk_add_test(...) # myprogram2

See also

camitk_init_test

camitk_additional_action_test()

camitk_additional_action_test

(

)

camitk_additional_action_test is a macro to create additional level1 test of CamiTK action to the CTest infrastructure.

It should be used when a CEP introduces a new component (with new test files) and an existing action should be run using this new component and test files.

Usage:

camitk_additional_action_test(ACTION_EXTENSIONS action1 action2 ...
                              [TEST_FILES  file1 file2 ...]
                              [TEST_SUFFIX name]
)

Parameters

ACTION_EXTENSIONS	List of the action extension that will be additionnaly tested with the given test files. Only the action extension name (not the full library/dll name nor the path) should be given. This macro checks the existence of the extension library/dll in the following directories (in this order) : build dir user install dir and global install dir. The first one that is found is used for the tests.
TEST_FILES	Optional. List of files to use for testing. If not set, then all the files in the testdata dir are used. If provided, only the filenames are required (not the absolute paths) and all the given files should be in the testdata subdir.

This will add tests named as follow: "component-[name of the current component extension]-additional-[name of the action extension]-level1-[index]"

See also

camitk_extension

camitk_init_test()

camitk_init_test

(

)

camitk_init_test is a macro to initialize a group of test (for the same command) It is used to initialize a series of tests for a given target.

Usually this is placed in the same CMakeLists.txt as the add_executable() cmake instruction or camitk_application() macro

It does few useful things:

• check if the application is properly defined
• initialize test id (then automatically incremented in camitk_test_declare
• initialize test output directory

Usage:

camitk_init_test(applicationname)

Parameters

applicationname

(required) The name of the test application (in camitk context, name of the project

Example invocation:

camitk_application()

...
# Start the test series for myprogram
camitk_init_test(myprogram)
camitk_add_test(...) # will be called myprogram1
...
camitk_add_test(...) # myprogram2

See also

camitk_add_test

set()

set

(

CAMITK_TEST_COMMAND_FILE ${CAMITK_TEST_OUTPUT_DIR}/

command

)

This CMake script run the action state machine and compare the output files to the expected files The intended use is to run this script from the camitk_add_test_actionstatemachine.

Usage:

add_test(NAME ${CAMITK_TEST_NAME}
         COMMAND ${CMAKE_COMMAND}  
                        -DCAMITK_TEST_COMMAND=${CMAKE_BINARY_DIR}/bin/camitk-actionstatemachine
                        -DCAMITK_TEST_COMMAND_ARG=${CAMITK_TEST_EXECUTABLE_ARG}
                        -DCAMITK_TEST_EXPECTED_FILES=${CAMITK_TEST_EXPECTED_FILES}
                        -DCAMITK_TEST_OUTPUT_DIR=${CAMITK_TEST_OUTPUT_DIR}
                        -DCAMITK_TEST_NAME=${CAMITK_TEST_NAME}
                        -P ${CAMITK_CMAKE_MACRO_PATH}/CamiTKTestActionStateMachine.cmake
)

Parameters

CAMITK_TEST_COMMAND	should always be camitk-actionstatemachine
CAMITK_TEST_EXECUTABLE_ARG	should always contains -a and have the proper -f and -o options
CAMITK_TEST_EXPECTED_FILES	is a list of files that are to be compared
CAMITK_TEST_OUTPUT_DIR	is the temporary directory where to run the test

Considering the value of CAMITK_TEST_OUTPUT_DIR, running the test creates

• file "command" which holds the command-line to execute for the test
• file "command-result" which holds the result status (exit value) of the command. The exit result can be 0=success, 1=failure or other value = failure
• file "command-output" which capture all the std output of the command
• file "test" which command is run to check the file (i.e., diff the files)
• file "test-output" the standard output of the diff cmake command
• file "test-result" contains the exit result (0=success, 1 or other=failure)