Coverage in Test Case Design

4.1 Align Collaborative Effort

Welcome back to the Art of Modelling series! Let's talk about collaborative effort and how it can impact your Test Case design and Coverage that you’re able to achieve within your tests. We want to build high quality tests while confirming that we’ve got good Coverage levels. To do this, we need to align our collaborative efforts.

We’ve previously spoken about Decision Gates and defined processes around making a cup of tea. We use these decision gates to add key journeys within our testing process. Things like having a kettle – does it boil – having sugar? Do we want that or not? And the same for milk, for example. All of these are different journeys, and the choice that we make affects those journeys. We’ve referred to this as the Must-Haves and the Wants.

Now it’s time to look at your Criteria and how it relates to Coverage specifically. To do this, we must recognize that these Criteria will change depending on the team that you’re interacting with and how that team interacts with the requirement at hand.

Working at team level will generally get an aligned set of Must-Haves and some Wants with a little careful thought; but integrating that Subflow into a wider picture might change what’s deemed as the highest priority in terms of what’s tested, when and how often. As you can see here, any isolated change in regard to a wider system, potentially changes the most important part – the group together has altered the priority.

The Model allows us to see that, and cater for that, but it’s an important context to have an understanding of. Determining the critical points and allowing teams in unison to work together whilst minimizing the impact of change or the risk of that particular change is essential. Now, with many teams working together on a single system, you can see how dependencies and risks can be understood. Using Coverage Profiles, for example, can really help us understand the impact of a wider change, especially when part of a bigger narrative - multiple teams prioritizing their tests, prioritizing their work in different orders.

What we want to do is use our Coverage system to allow us to derive an assured set of Test Cases; and that can be from a team’s perspective, it can be from a group’s perspective. But as long as we’ve got priority and flexibility, it allows us to pinpoint where we want our testing to focus. So let’s get into some real examples that we want to talk through and show this in action -  making sure that we maintain high quality, a good Criteria, and a defined Test Objective.

4.2 Specify Your Criteria

A glance at how we can use Coverage Profiles on a simple Model, but also to make sure that we're getting highly efficient tests. Coverage and Coverage Profiles inside Test Modeller are a really important way of being able to make sure that we are testing all the Scenarios that we want to, whilst ensuring that those combinations are correct and sensible.

Okay. We've modelled out a straightforward inventory, model here, that's got a few Scenarios in it. So, you can see that we've got a series of products that are part of our inventory. Depending on the stock level of those products, we'll then declare the amount of discount that is potentially available. If we've got a high amount of that stock, our discount will be less.

If we've got a medium amount of that stock or a low, low number of that stock, it will vary the amount of discount. So here I've got a few things in play; one, I've got some Tags. So if we go and look at our 10% discount, for example, we can see that when we get a high stock item and that will change depending on the amount of discount that we want to go and provide. Then depending on, other Scenarios that we could model later on, your loyalty status, card numbers, will all have impacts.

The Coverage Profile that we're using here - so we've got a low discount and a high discount. What this is going to essentially give us are the specific amounts of tests and really reduce potentially the number of combinations, making sure that our low discount model is enforced; and you'll see the Model highlighting those details for us.

Equally, if we switch that round to a high discount model - it's already been generated - we can see that we are still testing all the various Coverage combinations for each of those products. But also we are changing the amount of discount by selecting the Tags that we associate onto our Coverage Profile.

4.3 Define The Test Objective

Here’s end-to-end test flow, where we can use Coverage in order to target the kind of tests we want, for a very specific Use Case. Done in the context of a Master Model. Including the use of Subflow, Tags and setting Coverage level to create a Test Suite

For this online store, we've added a new store branded credit card, and that store branded credit card gets an additional 2% for any purchase that is made. So first we're going to Tag some of these nodes. Has either store-branded or non-store branded: that doesn't change anything in this model; but if I go back to the end-to-end model and I look at my Coverage Profiles, I can define a specific Coverage profile. So, for anything that is Store-Branded-Card I have High Coverage and anything that isn't Store-Branded-Card, I just want to Exclude for now.

So then, when I Generate my Coverage and look at the different Test Cases I can see – if I open up the Subflow here – that what I have is only Store-Branded-Cards; and then if we open up the previous module, which Ben (previous clip) was working on, I can see that they flow through all the different discounts that are possible. So I'm combining a Full Coverage of the types of discounts with the very specific Coverage of the Store-Branded-Card.