How much should a tester understand and rely on implementation details?
carriann last edited by user
I am a test manager in my organization. We do mostly manual tests. Before that I was a developer. I find that my understanding of software implementation and architecture is very important for my role. Since I know how the architecture works, how the various components affect each other and in which areas to expect which bugs, I understand better where are the riskier areas and what to test and what not.
However perhaps I rely too much on the implementation details, and I tend not to test places where I think the implementation has lower risk of failing. But what if the implementation changes? Perhaps a more black box approach is in order.
Also, most testers in my group don't have development experience, so often they test the same module in different locations, even though if you are familiar with the code you understand that its the same module and its very unlikely that it will behave differently in different places. Often they don't test timing or performance issues because they don't understand how multithreading or memory leakages can cause bugs. There are plenty of examples like this.
So I have two questions
- Should testers understand the code?
- How much should testers rely on implementation details?
Short answers: 1. Yes they should understand the structure, 2. It depends.
If you ask a software tester why they are testing a specific function or decided to use a specific test technique/approach, they will probably give you the Willy Sutton answer: "Because that is where the defects are."
Of course that is going to be different for each organization, but in general the testers should take a "trust but verify" attitude about assurances and assumptions from anyone (not just developers or managers). If someone told me that the application will work the same in five different areas of the application, I will immediately run a fast smoke test to verify the assumption and then plan my approach based on it.
The primary concern is context. Even though the "intent" of the implementation is the same in each area of the application, the actual "experience" may be different. One simple example would be a Java application that implements a Java tree rule that always sorts the tree data in alpha order. However, one area of the tree lists recently accessed files, such that inverse chrono order is more appropriate. If the testers didn't check the assumptions in that implementation, that defect may never have been caught.