Classification of software tests

The classification presented here can be found in many publications on software testing. The ones I consulted were B. Beizer, "Black-Box Testing" (1995) and R. V. Binder, "Testing Object-Oriented Systems" (2000). To have productive discussions about testing with your colleagues it is important to have a clear common understanding of the following concepts.

Dirty test (fault-directed):
We try to break the software by doing a falsification, i.e. to demonstrate that the software does not meet requirements.

Clean test (conformance-directed):
Doing a validation, i.e. to show that requirements are met.

Unit testing:
Testing classes or other "small" units. Called and calling components are either assumed to work correctly or replaced by stubs or mocks. This tests should execute very fast and are done by the programmer.

Integration testing:
Testing the interaction and consistency of  units A and B which are aggregated in a new component C.
We assume that the components A and B passed already successfully unit and integration testing.
The scope is a subsystem of software and hardware units which are physically dependent or are required to cooperate.


System testing:
Testing system behaviour that can not be done by unit or integration testing. Examples: performance, reliability. It focuses on capabilities that are present only in the entire system.

---

Code Coverage with gcc

Gcov is code coverage tool from the gnu compiler suite.

To use gcov we have to set the following compiler and linker flags (CMAKE):

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fprofile-arcs -ftest-coverage")
set(CMAKE_EXE_LINKER_FLAGS="-fprofile-arcs -ftest-coverage")

After compiling you have to run the program and then call:

gcov sourcex.cpp.o

This creates a logfile called sourcefile.gcov.

Create HTML files of the coverage data:
lcov --directory . --capture --output-file app.info

Generate a HTML view:
genhtml app.info