Speaking about test design, where do we start?

For many years, testers were told that they should start their quest for test cases by finding test conditions. If today a more modern approach talks about “test models”, the difference in practice is not that huge: you figure out some idea how your test item should behave and identify several test cases to prove or disprove your assumptions.

The inconspicuous term in this sentence is “several test cases”. Not “one test case”.

Design One Test Case

Let’s check a simple example, based on the freely available sample application “Virtual Showroom”. Using that application, you can configure a car from a fictional brand, and the system in return, e.g., displays a price for that car.

The requirements for this system tell us that depending on your choice of different available vehicles, add-ons, and other parameters, the rules for calculating the price will be different. In our heads, or on paper, based on what the requirements tell us, we derive a model describing these rules. That is our test model for one particular aspect of the test item.

Wouldn’t it be a good idea to create at least one test case for each of those rules? We do, and doing this in a structured way is precisely what textbooks call “equivalence partitioning”. That is, in my opinion, the most fundamental test design technique.

So your first test case might read:

  • Select vehicle “I5”
  • Select Engine “Engine 1”
  • Select package “None”
  • Check that the price equals “29.000,00 €”

Design Even More Test Cases…

That’s fun. So far! By writing down the next few test cases, I figure out that the basic structure of all of them is always the same. Only the data values change slightly. Different engines, different packages… Copy & paste, then adjust the details is a solution for now – but once you need to do maintenance on your test cases, it gets tedious again.

 

The obvious solution is to extract the variable bits into a data table and keep the common structure as a template. In testing, we call that common structure an “abstract test case”. The test cases derived from that abstract test case, based on the test-data, are called “concrete test cases”.

The approach to split data from test steps and, based on the resulting abstract test cases and the test data, derive concrete test cases is what we call “data-driven testing”.

As elegant as the approach is, running such data-driven test cases can be tricky indeed because you then need to recombine the abstract test case with each test data row.

Design ONE Test Case and Create A Data Table

In TestBench, you find sophisticated support for data-driven testing.

It is easy to add parameters in a test case. Just write the name of the parameter – e.g., “total” for the total price like this: ${total}.

Create your parameters.

Do you notice? This parameter is used in checking the result, so it’s an expected value. Parameters can be used for both, triggering the test item and specifying the expected behavior. You can do this in the title of the test case, description, test steps, checklist items, or in custom fields…

After inserting the parameters, they are in your abstract test case highlighted with a yellow mark-up and a bold font.

Highlighted parameters.

To specify the details for concrete test cases, just fill in the cells of a data table. Each column in the table reflects a parameter; each row defines one concrete test case.

Enter your concrete test data into the data table.

So far, you have done what we have discussed above.

If you do this using pen and paper, you could be in trouble if you wanted to see the actual concrete test cases now. Traditionally, testers combine the data in the table with the abstract test case in their heads: “OK, for test case 2, the parameter “vehicle” needs to be “Rassant”…”.

In TestBench, you don’t have to do that.

See any concrete test case and the changes you make in the data table.

No matter whether you just want to check how a change in the data affects the concrete test case. Or if you want to run the test cases and thus need the actual values. All concrete test cases are available at any time, parameters are replaced by the actual values, and you can see them in their real context.

From now on, adding new test cases based on a new equivalence partition or combination of values is a piece of cake: Just add a new row in the data table!

Also, maintenance is greatly simplified. Using data-driven testing, you can update all your concrete test cases for one abstract test case by adjusting the abstract test case. And if only one concrete test case needs an update, e.g., because the price has changed, no need to edit more than the respective row in the table.

It’s all about the data!


Matthias Daigl

Matthias Daigl studied computer science. He spent most of his professional life as a tester, trainer, and consultant at imbus. For many years, he has been active in various committees, including ISTQB and ISO. Today, he is the Product Owner for TestBench CS and works with three international teams.