Exhaustive Testing – How is it different from Effective Testing?

Complete testing or more commonly known as exhaustive testing is a testing approach that aims at trying all data combinations to leave no scope of further testing. In other words, it can be said that, after exhaustive testing, all testing parameters are covered.

The exhaustive testing comes under quality assurance, this means under no circumstance the application will cease to system crash or any other malfunctioning. It is also said that exhaustive testing is a myth and is just not feasible.

Even if you have tried all combinations and re-combinations of test cases, complete testing is just not possible. However, this condition is not applicable to a few trivial cases.

In this article, we are going to unravel some important characteristics of exhaustive testing and understand the process of performing it.

What is Exhaustive testing? Exhaustive Testing Example

Despite knowing the fact that exhaustive testing is physically impractical, the testing team tries to achieve it and tries to get it as close as possible. Let us understand this through an example-

1. Suppose, a calculator accepts integers ranging from -1000000 to +1000000. The tester will try to input all the possible numbers in between the range. No matter how much time is consumed, the tester will test to its maximum ability to leave no scope. This will be exhaustive testing in this case.
2. Another example can be, a device can be accessed only through a code with combinations of numbers in pairs of 3. The maximum possible input combination in this case, will be 9 numbers from 0-9(no repetitions allowed) and 3 input permutations. So the number of test cases will look like- 9*9*9 which is possible to execute. Here we have achieved what we call exhaustive testing.
3. The process applies to not just any software but to other machines as well. For example, a wi-fi router has internet coverage up to 500-600 meters. So the test data inputs will be tested within and outside the limit and include external factors like weather, wire breakage, etc. which is easy and very much possible.

However, such cases are possible for only small projects with limited scopes. Creating an exhaustive test case database for large projects where hundreds of inputs will be involved, exhaustive testing will be time-consuming as well as next to impossible.

Such scenarios can be commonly seen during gaming launches. No matter to what extent the game application is tested, there are always at least one or two glitches noticed by the end-user. Compatibility issues, issues in graphics, movement of the character, etc. are very common.

Though, the company can not be blamed for it as they always strive to achieve complete testing.

Strategies for Exhaustive testing

We have till now understood that barring a few cases, exhaustive testing is just not possible. In fact, one of the seven software testing principles says, that exhaustive testing is impossible. But what we can focus on is- how to achieve a complete test coverage.

We are listing some of the test strategies that one can use during an attempt at exhaustive testing.

1. Focus testing – we can focus and prioritize critical and crucial test scenarios by means of automation testing.
2. Probability of failure – this parameter is important and decides which functionality has greater risks of failure. Suppose in any travel site the maximum inputs and clicks will be on the to and from the column and the search button. So regression testing on that module should be priortized.
3. Regression – the tester should identify which units get impacted the most when any change is made to the application under test. Regression testing over those parts becomes important.
4. Recovery – the recovery rate of a system from a failure or error has to be noted. The recovery rate of each functionality has to be noted by the tester.
5. Review – fellow testers, business analysts, team managers everyone should continuously review the system thus keeping a user point of view.
6. Refreshing test cases – each time new bugs are found tester/developer should take special care to create new test cases while keeping old cases in mind.
7. Test environment – a stable and effective test environment should be created to review the AUT. The environment should be a replica of the real world.
8. Ad hoc testing – at the end of the testing cycle, it should be made sure that ad hoc testing should also be conducted.

Exhaustive testing vs Effective testing

Despite seeming similar both the terms have distinguishable characteristics. Let us look at them one by one.

Conclusion

Despite being labeled as an impractical and impossible method, testers yet strive to achieve whatever is close to exhaustive testing. The method might be time-consuming or high in cost, but it is 100% effective.

It leaves no straw for the user to pick and complain of. If the above-mentioned strategies are followed one can achieve something near to exhaustive testing.

However, for large projects mart work should always be prioritized and that is where software techniques like effective testing come into play. An experienced tester will always choose the right technique for different projects.

Еще об этом:

Manual Testing Interview Questions and Answers Types of Testing | Software Testing Types Every QA Should Know Shift Left Testing and Shift Right Testing Approaches Scripted Testing – How is it different from Exploratory Testing? Integration Testing Top Software Testing Books Every Tester Should Read What is Robustness Testing in Software Testing? Top 10 ETL Tools for Streamlined Data Management What is Manual Testing? Big Data Testing – The Complete Guide