Should all test cases be in test suite?



  • Should all test cases be reusable and in the test suite? Or are there scenarios where you would want to create a specific test case for a single user story? (or a combination of both)

    e.g. a bug was solved for a specific OS. In this case I would actually like to test that specific OS and maybe one extra OS and focus mostly on the bug testing. Should you use a generic test that covers way more scenarios or could you just copy/create a test for this specific user story and use this as a throw-away functional test?

    Other scenarios I could imagine are partial functionality or single time edge cases.

    I tried googling for answers, if anyone could give me a source as to how this would work, that would be great.



  • Generally what you "should" do is highly dependent on your context. If you're in a regulated industry, for example, you would be required to keep a lot more documentation around the testing you perform than if you're a startup building todo apps.

    Even in regulated scenarios, doing unstructured testing outside the context of your formal testing is likely acceptable and even encouraged, but you may also need to provide some form of objective evidence of the testing performed. This may not, however, have to be in the form of a test case in a test suite. I can add more detail if you're in that situation but it's not as common.

    Beyond whatever the external requirements imposed by your context, there is the question of what provides value to you and your company. After all, good testing isn't about checking a box, it's about finding problems!

    If the only requirement for your testing is that you gain some confidence in your bug fix, then it's really up to you how you gain that confidence. If it's a bug that you're not really worried about having regressions for, maybe you only need to do informal testing, and not embed the test in a formal suite that you run repeatedly. If you would get value out of writing a regression test to keep around, do it! You might also find some of the answers to my similar question helpful (note that I wrote that question before I learned anything about context-driven testing and would not take the approach we were taking on that project for my future projects).

    For some additional reading, check out Michael Bolton's 6-part breaking the test case addiction blog series.


Log in to reply
 

Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2