How to use Test-Suite

Test-Suite is a micro-benchmark suite designed for validating various static analysis algorithms (particularly pointer analyses and static bug checkers) developed in SVF. It includes around 400 hand-written programs and code snippets. Test-Suite provides flexible and extendable interfaces for users to add their own tests for validating the correctness of different static analyses.

Download SVF and execute the following commandline under SVF's root directory. build.sh will build SVF, downloand Test-Suite and run SVF's analyses against the test cases.

git clone https://github.com/SVF-tools/SVF.git
cd SVF
source ./build.sh

The following gives the Test-Suite's folders and the corresponding SVF's options for validation.

Folder	SVF option	Description
basic_c_tests	wpa -ander -stat=false	basic test cases for C programs (flow-insensitive and field-sensitive analysis)
basic_cpp_tests	wpa -ander -stat=false	basic test cases for C++ programs (flow-insensitive and field-sensitive analysis)
fs_tests	wpa -fspta -stat=false	flow-sensitive tests
cs_tests	dvf -cxt -print-pts=false -stat=false	context-sensitive tests
path_tests		path-sensitive tests
complex_tests	wpa -ander -stat=false	complex test cases simplified from real programs
mta		multithreaded test cases
mem_leak	saber -leak -valid-tests -mempar=inter-disjoint -stat=false	memory leak test cases
fstbhc_tests	wpa -fstbhc -stat=false	FSTBHC tests
fstbhc_tests/fs_tests	wpa -fstbhc -stat=false	FSTBHC tests sourced from fs_tests

To compile the tests in fstbhc_tests and fstbhc_tests/fs_tests, ctir Clang is required, which SVF's build.sh will download.

Scripts

./generate_bc.sh builds all test cases in the aforementioned folders into test_cases_bc.

./generate_cmake_test.sh will produce the add_test calls required by CMake and output them to stdout. The output can be appended to CMakeLists.txt with >> CMakeLists.txt. It takes two arguments: the directory containing .c/.cpp test cases, and the command to run the test. For example, ./generate_cmake_test.sh basic_c_tests "wpa -ander -stat=false" will produce:

add_test(
  NAME basic_c_tests/CI-funptr.c
  COMMAND wpa -ander -stat=false ${CMAKE_CURRENT_SOURCE_DIR}/test_cases_bc/basic_c_tests/CI-funptr.c.bc
  WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/${CMAKE_BUILD_TYPE}-build/bin
)
add_test(
  NAME basic_c_tests/CI-global.c
  COMMAND wpa -ander -stat=false ${CMAKE_CURRENT_SOURCE_DIR}/test_cases_bc/basic_c_tests/CI-global.c.bc
  WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/${CMAKE_BUILD_TYPE}-build/bin
)
add_test(
...
...