Test varieties: what is it and why do you need them?

What are test varieties?

Test varieties are the different varieties of testing out there, organized together to choose the right test varieties for each specific situation.

Within software testing, as we mentioned in the introduction to this blog, there is no one-size-fits-all. Because every software, project and system is different, every situation requires a different type of test. So you need variations in your tests for each specific situation. You take care of that with test varieties.

The term is also meant to make everyone involved in software testing aware that there are always different needs for testing. That you need to organize different varieties of testing for each situation. Only then can you be sure that everything has been tested correctly and that the final product has high quality.

Test varieties are defined based on relevant quality characteristics

Test varieties are based on relevant quality characteristics. Quality characteristics include:

• Reliability
• Usability
• Efficiency
• Maintainability
• Safety

These are just a few quality characteristics. A particular situation may call for other quality characteristics. You can imagine that when testing software for a chatbot that "personality" should probably be an important characteristic as well.

What do you need test varieties for?

If you are going to test software, the main goal is as follows: all testing activities should test all important areas and aspects of the system. Test varieties are needed here to have an overview of the needs and expectations of everyone on your team and the relevant testing activities to meet them. Especially within a DevOps or cross functional team, this is very important. Only then can you take the right steps per cycles in the development process to achieve a high-quality end product.

How do you determine test varieties?

So, as a software tester, you need to determine test varieties within a cross functional team or DevOps team. You then follow the following steps:

1. Review your test areas. What exactly needs to be tested? Is this about testing whether a numeric value can be put into a numeric field and an alphabetic value cannot be put into a numeric field? Or should there be more testing such as testing whether an order can be placed and paid for in an online shopping system;
2. Then look at the levels in a so-called test pyramid to decide what kind of testing you will automate.
3. Finally, see if the test quadrants add additional variety to your tests. When testing a few small modules, you have probably already covered most of the test quadrants. However, you may need to add some security testing, for example, to ensure that the IT system complies with security regulations.

What is a test pyramid?

A test pyramid consists of three levels: The first level contains small individual tests. The second level focuses on the integration of modules or systems. These tests often involve the interfaces, for example, the application programming interfaces (APIs). The third level usually uses the graphical user interface to perform the tests. These tests are performed the slowest because these tests focus on complete business processes, for example, a complete user transaction.

Below you will see such a pyramid illustrated.

What are test quadrants?

Test quadrants help software testers determine test variants. There are many different types in this but we adhere to the original version by Brian Marick [Marick 2003] and the adaptation by Lisa Crispin and Janet Gregory [Crispin 2008]. It is also used in the book Quality for DevOps teams.

There are four testing quadrants:

1. Here the focus is on technology and where testing activities guide the team in the creation of the product. The test variants performed here should be automated.
   
2. This one focuses on the business and is also about guiding the team. For example, it is about testing a business process. If possible, it is best to automate these as well.
   
3. This focuses on evaluating a product. These tests are mainly performed manually.
   
4. The fourth focuses on technology and assesses the non-functional aspects of a product. This mainly focuses on verifying whether the software is fit for purpose and finds out how the software responds to unexpected behavior. Hence, this can only be validated once the product is ready. The performance test is a good example of this.
   

The testing strategy

With your test varieties clear, it can be useful to draw up a test strategy. In it, you describe how all tests will be performed. However, this is not always needed in great detail. For example, in a small team, the test strategy may consist of just a few agreements discussed among team members and not documented. Are you working in a large and complex process? Then the testing strategy may need to be documented extensively.

TMap® Quality for cross-functional teams: learn all about test varieties

So with test varieties, you clarify what testing activities are needed to meet the needs and expectations of everyone on your team. Do you want to learn all about this? Then our e-learning TMap® Quality for cross-functional teams is for you. It covers everything about test varieties.

Want to know more about Testlearning?

Would you like to be kept up to date with developments around our test training courses? Then follow us on LinkedIn, sign up for the monthly newsletter or read our blogs!