Automated testing: focusing on testing controls vs. workflow
Mystic last edited by user
I'm in charge of regression automation on my team, and I need to add some new tests into our regression. Before I get started, though, there's an issue I'm struggling with: where is the line between focusing on automating user workflow through a system, and focusing on only ensuring that the controls are working as they should be?
For example: imagine a system can set up new users, and later edit their information. On the screen that accesses users, there's a list of recently added users at the top, and there's a search function at the bottom. The workflow of a client using our system would be to create a user, then access this screen to edit the user's information immediately afterward. I could write my automation that way, but I would like to write short tests that only test one thing each. I'm trying to stay away from behemoth tests that take a long time to run and test many things, and when one fails you need to wade through many many checkpoints to see exactly which one failed - this is inefficient. A test that follows this workflow would have to either test two things (creating the user, then editing the user's information), which would make it one of those behemoth tests I'm trying to stay away from, or I would have to write two tests that have a dependency between them (Test1 creates the user, Test2 depends on the user being on that screen to click).
It seems to me the better solution is to write a test that focuses on creating a user and write a separate test that uses the search function I mentioned to look for an existing user (we always start our tests with a pre-set database of users) to edit their information. The problem is that this tests purely the controls, not the workflow. I could always throw in a third test that tests the functionality of the recent users screen, so that's covered by the automation too. It seems to me that manual testers should focus more on workflow, whereas my automation should focus more on purely how the controls work. Is my reasoning correct?
Welcome to SQA, Ana. I see at least two questions:
- Does it make sense to distinguish between focused tests and workflow tests? Yes, it does. A focused test concentrates on a particular feature, including things like edge conditions. It attempts to answer the question, "Does this feature work in all circumstances?" A workflow test verifies that the system as a whole is usable for its intended purpose. Both kinds of tests are necessary, but it is a better use of time to execute them separately.
- Should focused tests tend to be automated, and workflow tests executed manually? That depends on your circumstances. Here are some question that might help you decide what to automate. Which tests tend to be error-prone? Which tests do you want to run after every build? Which tests would developers run before every check-in if they were fast enough? Which tests are easy to automate and maintain? (The maintenance part is admittedly hard to predict.) Which tests would you use for smoke-testing?