Thoughts on pairwise testing
With pairwise testing we're able to reduce exhaustive testing to testing all combinations of pairs. I'm new to a group of software testers testing an huge amount of functions with many input variables. Now my task is to reduce test cases if possible. My problem is with pairwise testing tools I would create more tests than they are already doing, because they're not testing exhaustively (or anything near exhaustive), but just a "few" cases they think are mostly used and know have showed off bugs in the past.
Now I'm thinking about writing down the tests being done and checking whether they already covered all pairwise combinations of this test with some other tests. This would really be a lot of unsystematic work I believe, and I can't imagine there is no better way to help them optimize their work. Furthermore I'm not quite sure if this would work or not. I would appreciate any contribution to my problem.
consider this scenario: you got to test a software and you already got some testcases with decent coverage for your Problem. Pairwise tools generate testcases covering all pairwise combinations, but thats not what you are looking for, because you dont have the time to test all combinations pairwise tools suggest. Instead I'm thinking about weather its possible or not to drop some of the tests my colleagues created by checking for tests with all pair combinations being already tested in other tests. p.s. does anyone know a tool that could help us with this problem?
O.K. now after analysis of the testing-process, I figured out that they are just testing control samples. For example they got 1000 test to do, 4 systems (e.g. first system is Windows XP French, second is Windows 7 englisch etc.) now anyone grabs any feature he wants to and tests it on his mashine (he or she tests that function in just one scenario and if the test passes nobody else has to do it anymore).
I can't imagine how pairwise testing could help them drop any of their tests when they are already testing just control samples while pairwise would suggest to test way more combinations (and testing more or slightly more for better product quality is no option they say). So does maybe anyone see something i might've overlooked or didn't think about?
Pairwise (or combinatorial) testing is a technique to help us systematically produce a subset of tests that test the combinations of variables for 2 (or more) parameters out of all possible combinations of variables. Today there are many tools to help us derive that subset of tests (see http://www.pairwise.org).
Applying this technique assumes the set of tests that are currently applied are providing inadequate coverage, and that the complete set of combinations is impossible to test (mostly due to time/resource constraints).
Pairwise/combinatorial testing is most valuable in complex systems, and these tests are generally very good candidates for data-driven automated tests.
If the total number of combinations is huge (generally the Cartesian Product of all variables), then mapping current tests against the exhaustive list may be more trouble than its worth. Mapping it against a set of tests derived from a tool is likely more feasible.
If your testers already have a set of combinations based on intuition and historical failure indicators that is a good start. An analysis of other 'pairwise' combinations may provide some additional insight.
You might also refer to these posts on combinatorial testing