Selenium Webdriver - handling error messages that are shown to user

  • I've made an automated test case that creates a user inside an app. It's pretty simple, the test enter username, email, password and presses OK.

    But how should I handle scenarios if user enters an username or email that is already taken? The app shows a message to the user if username or email is taken but I don't know how I should handle such scenarios where Selenium tests enter info that cannot be used.

    Should I check if a message pops up after clicking "OK" button? If yes, should the test show a note in test report and end the test or try entering different values?

    Another idea - should I have a separate test scenario if user enters username or email that is taken?

    I really need some ideas that would be optimal.

  • It depends

    The best way to handle your scenario depends on the way your application handles the scenario.

    • If the app notifies the user that username/email is taken as soon as they move to the next field: In this scenario, you should have a separate test for each possible situation. The minimal set of tests I'd use are:
      • Everything works - the username and email are unique and meet whatever other criteria is used.
      • Username is taken - for this I'd take the test as far as checking that for an already taken username the expected error text is displayed.
      • Email is taken - again, the test would be that for an already taken email, the expected error text is displayed.
      • User chooses unique username after being notified of already taken username - for this scenario, I'd want to verify that the error message is cleared once a unique username is selected.
      • User chooses unique email after being notified of already taken email - again, the verification is that the error message is cleared once a unique email is selected.
    • If the app does not notify the user until after they submit: In this situation, the way I'd test would depend on the way the notifications work and how they display. In general, if notifications call out each invalid field, I'd have a test per field. If they are less helpful (which I've seen entirely too many times), then a single test of invalid values might suffice.

    In general it's better to test each potential path or scenario separately. That said, there is a point where it's better to simply stop because the cost of time to create and maintain automated tests exceeds the value they provide. Theoretically you could cycle through duplicate usernames or emails an infinite number of times before the user finds one that is unique. In practice, there's probably going to be a point where a user said "It's not worth it" and stops trying.

    If you have sufficient logging around the app, you can start with the basic scenarios and use the logging to determine the average number of tries it takes for a user to select a unique value, the maximum number of tries, and any other statistical values you care to use. That will allow you to make an informed decision on how far into the "User chooses duplicate, user chooses another duplicate, user chooses another duplicate..." loop you need to go.

    Security implications: You should be aware that telling the user they need to choose another username or email is giving hackers the ability to find usernames/email addresses that are valid for your app, and consider some of the risks in tightening the number of times someone can enter a duplicate vs the risks in allowing someone to continue indefinitely.

Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2