Applying Test Case Coverage

In this documentation article, learn all about test case coverage and negative testing in Test Modeller.

Test Case generation can focus on certain parts of a system's functionality, optimising testing based on time and risk. This uses the notion of "coverage", where tests exercise or "cover" areas of a system's logic to a required level, from being ignored to tested exhaustive.

Test Modeller offers a granular approach to test coverage, controlling the level of testing rigour for sets of blocks or subprocesses in a given Model. Testing can then focus on new or critical functionality, without having to exhaustively test surrounding logic to the same degree. Testing confidence can increase as a result, while keeping run times low.

Profiles

The required level of coverage is set by feature within a given Coverage Profile. Several Profiles can be created for one model, in order to re-use Profiles and quickly regenerate tests that focus on certain features in future.

To create a new Profile, navigate to the "Test Generation" tab of the menu, and click "New" in the "Profiles" section of the menu:
Enter a name for the New Profile and click "Save":
Previously created profiles can be toggled between using the drop-down menu in the "Profiles" section of the menu, and their name can be edited by clicking "Edit". You can Delete the selected profile by clicking "Delete".

Feature Tags

In a given Model, blocks and Subprocesses can be tagged as "Features". This enables coverage levels to be set within a given Coverage Profile for each Feature, generating Test Case to exercise the tagged logic with varying degrees of rigour.

To tag blocks or subprocesses as a given Feature, navigate to the Test Generation tab of the menu and click "Features". This will open the Feature Tags sidebar. Click a block or Subprocess, enter a name for the Feature, and click "Apply":
Blocks assigned the same Feature Tag within a given Model will be treated as one Feature during test case generation.

Features assigned to a block can be browsed and editing by right-clicking a block, and navigating to "Features".

Setting the Coverage for Features within a Profile

Coverage is set for Features by Profile. To set the level of test coverage for tagged Features, navigate to the Test Generation tab of the menu and click "Coverage".
This will open the Coverage Profile Window:
Click "Reference" in the "Features" box and select the relevant Feature Tag:
Use the drop-down menu to assign a measure of coverage from "Ignore" to "Exhaustive" for each feature:
The "Default Coverage" drop-down sets the level of coverage for every block without a Feature Tag assigned. This is useful for focusing on new or critical functionality, while testing the surrounding logic as much as possible based on time.

Test cases can be generated to exercise the broader model only insofar as it is required to reach the tagged features, or the broader model can be tested more rigorously if there is time.

Once Features and a Coverage Profile have been created, navigate to the Home tab of the menu and save the model. Test Case generation will now be performed based on the Coverage Profile assigned to the model. 

Note: The "Ignore" Coverage metric will exclude any paths through a model that do not contain a tagged Feature. To use "Ignore", a Start block for each path to be generated must be tagged as a Feature with a Coverage measure assigned to it.