Criteria for deciding what is more important to test

  • Which criteria do you usually use to decide what is more important to test and what is less important? I mostly have two parameters: what's the likelihood that the user will encounter the scenario under test, and what is the impact to the user if the scenario breaks. I will tend to test those scenarios with the highest occurrence rate and with the highest impact. Sometimes I would also add to the mix the probability that there is a bug in the scenario. So I would usually put more testing resources to areas that were not tested enough, have a more complex functionality or have a higher occurrence of bugs.

    So which criteria do you apply to decide what to focus your testing efforts on?

  • You want to save your team time and pain, and you do that by focusing on potential issues that are going to be much less costly if we catch them sooner rather than later. I usually start by asking myself "what feature(s), if broken will…”

    • block me from the majority testing,
    • be particularly difficult/time-consuming for the developer, or
    • require fixes that are potentially wide-reaching and will therefore result in me needing to re-test other scenarios in my test plan.

    The faster I get those bugs back to my developer, the faster he or she can commit a fix. The earlier in the process I do that, the less likely it will be that I’ll need to re-test things that the new commit might have impacted.

    1. Test the most basic happy path surrounding NEW code, for 10 minutes. If you aren't able to perform the most basic functions of a new feature, you need to get the code back in the hands of your developer as soon as possible. Chances are, if something is broken here, it could be a longer fix.
    2. Test functionality that impacts a large percentage of the code base as a whole. This is where we find dependencies that we didn't think could possibly be impacted by our changes. It’s much better to find these early, because they could indicate a need for deeper code reviews, and also wider regression testing and an adjustment to your test plan.
    3. Test the critical functionality, and the most commonly used areas. What does your app NEED to be able to do? What do most people use it for? What are the potential blockers that are the only reason some users use your site or app?

    4. Test the potential areas of high risk. Data loss. Load/Performance issues. This should be part of a conversation you have as a team - what are the biggest risks involved in this sprint? What are the worst-case scenarios? This is also the time where it’s most beneficial to test in a stage environment that replicates production as closely as possible.

    5. Then normal regression testing on down to edge cases and thinking of creative ways to break stuff (my favorite part).

Log in to reply

Suggested Topics

  • 2
  • 2
  • 3
  • 2
  • 4
  • 2
  • 2
  • 2