Is refreshing multiple times a test case for web applications?
I have been faced with a QA test case where refreshing a page multiple times almost instantly (6x, 7x, 8x) creates a bug. This is totally random and every time the number of refreshes require to reproduce changes.
So my question is that should this be even a test case?
Edited It is caused due to uninterrupted page load. Further it is suspected due to DOM elements not rendered fully
carriann last edited by
Yes, while rather crude and possibly not the most graceful of test cases, such 'abusive' test cases are an important tool in stepping off 'happy path' thinking and testing.
- Effectively a test like this is a crude load/response test.
While in theory it shouldn't happen in the real world environment, the fact that a bug was discovered with such abusive input should be raising red flags for investigation even if ultimately no resources ever get devoted to fixing the issue. ["Known issue, won't fix" is totally a legit 'solution' to some problems. I guess...]
- Why does this non-standard input/interaction cause an unexpected state/bug to be entered?
- Does the above answer expose potential security risks [Such as the system grabbing incorrect, possibly sensitive, data for the wrong user?] or performance spirals? [Such as momentary excessive loads corrupting sessions for some users, who then all end up running multiple hard-refreshes to clear the issue, which in turn creates more server load and causes more user sessions to corrupt?*]
Tests like this go right along with things like navigating back and forth between two screens/states over and over again. The software isn't supposed to be able to fail on things like that, and the low level unit tests should catch it, but at the end of the day what something should do isn't nearly as important to QA as what it actually is doing.
"Something goes wrong when..." are important points to find in the software, and while we may prefer to find more graceful ways to discover such issues the truth is that at times the easiest way is to simply hit things 'at random' with a metaphorical hammer to see if anything breaks.
As a project matures and testing methodologies improve, then such a case may become defunct in favour of more targeted and graceful solutions.
It is also important to remember the dangers of thinking "the user will never do something like that in the real world..." because users are mythical creatures and will seemingly do everything in their power to do things no one on a dev team could ever imagine them doing...
- Are they 'aggressively impatient', and hammering on the refresh button out of anger if something goes wrong?
- Do they have a mobility issue or faulty hardware that results in a surge of unexpected or unintended clicks?
- Does the 'rapid refresh' bug kick in if the server is under load, and the user is actually refreshing fairly casually once a minute or so trying to get your site to respond?
At the same time realities of resource management need to be considered. Test cases like this may not be well suited to being constant 'front and centre' operations that all get run day to day.
- Maybe group them as a pool, then periodically run a subset of them against a given build rather than the entire pool.
- Possibly useful as part of morale events: Pull the team together with their food and drinks of choice, and make a team game of 'who can break things in the weirdest way'.
* I have seen one client reduce their server requirements to nearly an 8th of what they had been using after resolving the software issue behind a problem like this, rather than continue to throw hardware money at it.