Core Concepts - Test Data Allocation
Table of Contents:
Overview of Concepts
Test Modeller supports the concepts of distinct Projects and Versions within Test Modeller workspaces. Test Data Allocation, however, is separate and independent from these Projects. The result is that project teams can use the same Test Data Catalogues and Test Data Allocation pools for different tests, the organization of which is up to the teams themselves.
You may find that many projects are adjacent and share common test data lookups or test in the same test environments, in these cases it makes more sense to share the test data definitions and allocation. Some examples of these are:
The HR development team are using automated scripts to test holiday allocation and need to select managers who are not currently on holiday to test the front end system works correctly. They will allocate unique Manager Ids and Employee Ids to each automated test.
Concurrently, the Payroll development team need to test their holiday entitlements detection system to identify if extra holiday has been taken and to be able to manually add holidays to employees. They will create manual tests and need to add holidays to employees.
The Payroll team needs to use the same test criteria as the HR team and will therefore use the same Data Allocation pool so they don't interfere with the HR automation framework.
A migration team is testing the migration process from the existing banking application to a new application. The accounts are split up into different types which require different transformations to ensure they are standardised.
There are over 200 developers and testers working in 20 scrum teams, with each scrum team using a core set of models (lookup new address formats) and individual transforms. Each scrum team has different Test Modeller projects, however there is only one development and testing environment.
Both the manual tests and the automated tests need to use the same allocation pool to ensure teams do not transform the same account using different transforms.
Test Data Allocation Pools
Allocating Values in Test Modeller does not mean the keys are locked to the application under test. Therefore, if you are in your HR system and Person 987 is allocated to you, there is nothing to stop another developer or tester editing or using Person 987.
The allocated values need to be managed as part of the development team's processes. It is quite common to see teams that have a central spreadsheet where they manually enter keys once they have found them. The Test Modeller allocation, in effect, replaces this spreadsheet.
For automated testing it is much easier as you will just need to run the allocation process just before you run the test automation, see the section 'Running Test Allocation as Part of Your Automation Framework'.
Unique and Non-Unique allocated values. When you set an allocated test to Unique you are saying that returned value is for this test only, the value will not be assigned to ANY other test. If you set a value to Non-Unique, the assigned value could be set in another non unique test or assigned an unused value.
Unique Test: SuiteHuw_TestDeleteCreditCard has Card ID 7 assigned
Non-Unique Test: SuiteHuw _TestReport_1 has Card ID 4 assigned to it
Non-Unique Test: SuiteHuw_TestDashboard_1 has Card ID 4 assigned to it as well
To define uniqueness, Test Modeller needs more information to work out how the returned values are matched with other values. There may be an allocation pool with hundreds of automated and manual tests used for many different parts of the application, you may be using multiple different types of tests (data lookups) but they may well be using the same logical primary keys.
Test Data Catalogues
The Test Data Catalogue is a list of standard test data lookups that can be used by project teams to find and make data during testing. Often, teams, developers and testers create lookups using SQL and scripts to find data. The Test Data catalogue allows users to share best practice across teams, well developed SQL and test scripts which are controlled centrally. The other advantage is that if you change a query for example, every automated test would automatically get the updated SQL, allowing changes to be quickly propagated across tests.
Above is an example list of test criteria stored in a Test Data Catalogue within Test Modeller.
Each test criteria can be run against multiple test environments once the criteria is linked to a test. For example, you may want to find a unique live account you wish to close (a feature under test) across a range of different test environments. Whether it be in your development, QA or UAT environments, each allocated account will most likely be different in each of the three. So, although the same test with the same test data criteria is used, each environment would allocate different account numbers once the test is ready to be run.
You can create multiple Test Data Catalogues if you wish and separate them amongst different project teams and environments. The organization of this is up to the local project teams and environment managers.
Above is an example list of Test Data Catalogues in Test Modeller.
Initially it makes more sense to just have one Test Data Catalogue that is shared amongst all development teams, however after a while you may wish to separate into different Catalogues. In the example above the Business Intelligence team have decided to set up their own Test Data Catalogue, their SQL queries are much more complicated than the Development teams and there is little cross sharing.
There is also a batch utility to export and import Test Criteria, see Exporting Allocated Tests.
SQL and VIP Lookups
When you define a test type there are two types, the first is SQL and the other, is a VIP(Flow):
SQL Lookups typically return lists of columns from tables, these are used to define uniqueness so they can be eliminated as already being used. For example, Test 1 needs a USD Credit card account. Whilst Test 2 needs an account with a fraudulent transaction. These lookups could be expressed as:
SELECT CARD_ID FROM CARD WHERE CURRENCY IS 'USD'
SELECT CARD_ID FROM CARD WHERE CARD_ID IN (SELECT CARD_ID FROM TRANS WHERE FRAUD = 'Y')
When the SQL is defined, the Table Name and the List of Columns act as the logical key that will identify whether another test is already using the same combination of columns.
For the example above, within an allocation pool, when looking for a CARD_ID for Test 2, the value that has been allocated with a TABLE_NAME of CARD and key name of CARD_ID will be checked. Therefore, when looking for the currency USD for Test 1, the allocated value will not be used by Test 2 when looking for CARD_IDs.
VIP Lookups are the second type of lookup here. VIP is a powerful Integration engine that can make call outs via multiple methods to find and make data. See the section 'Using VIP to Lookup and Make Data'. When defining the VIP flow in Test Modeller you still need to be able to segment different flows, so they do not use logical keys from other flows (or SQL). Consequently, when you define a flow, you need to enter a Data Identifier as well as the list of keys.
For example, you may have two tests:
Test 1 which retrieves a CUSTOMER_NUMBER using a rest call to SalesForce. Here, the Data Identifier is set to SFCUSTOMER and Key CUSTOMER_NUMBER.
Test 2 could read a flat file and pass a list of multiple CUSTOMER_NUMBER from the file. If you set the Data Identifier to the same value, SFCUSTOMER, then Test 2 will not use any values from the file which have already been allocated by the rest call in Test 1.
When you set up an allocation pool any values assigned to any manual or automated test cases that call the pool are stored within that pool. Remember, each test environment you are working with will have different data in it so you will get different allocated values in each pool if you set up a specific pool for each individual test case and development environment.
For automated tests it is possible for them to run in multiple environments, in this case, you will need to create a pool for each environment. Developers may also have their own personal test databases so they would need to set up their own allocation pools.
There is a batch utility to export and import Test Data Allocation Pools, contact us if you need to use this.
TEST DATA AUTOMATION TEST DATA ALLOCATION INTRODUCTION TO ALLOCATION