Can testers benefit from knowing the status of unit tests?



  • If a unit test passes, what you can infer is - barred exceptions - that the unit tested works as intended. Nothing can be said about all other components involved since they might be (and usually are) faked.

    Testers' confidence in the quality of a product, on the other end, is boosted by looking at acceptance tests. Such tests operate at a higher level and should cover the behaviour of the system when used by the stakeholder.

    The question: Say you, as a tester, are about to release a new version of the software. You can choose whether to look at the status of the unit tests - which cover pretty extensively the system, unit-wise - or not.

    Does looking at the tests have any effect on you? For instance: - how is your confidence in the quality of the release affected? - would you test differently (different approach, more in-depth testing of some component with respect to others, ...) if you didn't check the status of those tests?



  • Here are a few considerations:

    There are multiple reasons why a unit test might fail.

    Unit tests have to be maintained just like the system under test does. When a unit test breaks, it might point to a bug in the system under test, or it might point to a bug in the unit test (e.g. a timing issue), or it might even point to a unit test that no longer makes sense.

    No one here can tell you what your process should be.

    There is no universal right answer for your question because it depends on the circumstances. Development and test processes vary from one company to the next. If I were in you, I would discuss it with your team and figure out what works for everyone involved.

    You and your developers should agree on how unit tests fit into your development process.

    If you aren't sure whether a failing unit test is a problem, you need to talk to your developers (or your manager, or their manager) about it. For example, you might agree that all unit tests need to pass before you start testing -- although that policy could have unintended consequences. Or you might agree there should be a bug opened for every unit test that's broken at the beginning of the QA cycle so that broken unit tests can be triaged like any other kind of bug. Or you might decide that unit test failures are an internal developer issue that has no relevance to testing because the developers are mature enough to fix the tests that really matter.



Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2