How to organise test cases for configurable application
With multiple clients I’m stuck how to document different varieties:
Take for example cancellation:
- one client has cancellation period x
- another y
- third one does not have cancellation option.
Situations would be i.e.:
- under setup 1 cancellation shouldn’t be possible,
- under setup 2, cancellation is possible depending on the parameter X
Edit: I am editing original post as i can not add comment this big.
Our app is very configurable. Config options go to tens of options and there are potentially tens of clients. So lots of permutations.
You are basically suggesting duplication of tests for every configurable option for every new client.
Something like this:
- Duplicate existing test cases and modify them according to new client specifics. Do that for each new client. Keep test cases for each client in their “client suite”.
For this approach i see following pros:
- it is easy to do
- all client data is contained within the test case
- only client data which is relevant for the test case is included in the test case
and following cons:
- when a part of the application changes we have to modify or add the same test cases in all suites
- client specifics are divided between multiple test cases (bad overview)
- if we want to test sub-set across multiple clients, we have to select sub-set in each suite
How about something like this:
- Keep one set of test cases, make them generic and somehow link specific client configurations to them
- no duplication, easier maintenance
- all client specifics are in the same place (good overview)
- easy to select sub-set of cases for testing
- I don’t quite know how to do it so we don’t end up with a list of requirements in one place and a generic set of test cases where it might not be clear which test case refers to which part/configuration.
- More time will be spent looking for expected results in the list of requirements
- Test cases could become too generic / vague
I believe option 2 looks better on the long run, if there is a good way to do it. Any opinion or advice from someone who experienced this would be very welcome. Tool wise: we use Excel for test case tracking but we are thinking of transitioning to TestLink.
It would depend on your tools.
For documentation, you would have two types of tests in this case:
Scenario: Client with cancellation option Given .... Then the user has a cancellation period of <period> Examples: | period | | x | | y | Scenario: Client without cancellation option Given .... Then the user has no cancellation period
And for the testing code, you would have a hierarchy:
abstract CancellationPeriodTest // Common setup and tests WithCancellationPeriodTest extends CancellationPeriodTest // Specific setup and tests WithoutCancellationPeriodTest extends CancellationPeriodTest // Specific setup and tests