Testcase fails in Jenkins but passes locally via testNG.xml in IDE

  • I've come across a strange problem when I execute my testcases via testNG.xml they are fine and passed but whenever I execute the same testcase in Jenkins I got the following exception

    Element is not clickable at point (663.6666870117188, 30). Other element would receive the click:

    Note there is no scroll in the page and no loading of element for which driver should have to wait.

  • UI feature tests passing in Jenkins, CircleCI (or other Continuous Integration servers) that fail locally or vica-versa is not uncommon.

    In a nutshell they are basically due to different environments behaving differently.

    The environments may vary in regards to things such as:

    • OS - your local might be a Mac, Jenkins might be Ubuntu
    • memory type and size
    • hard drive type and speed
    • location
    • network speed

    The combination of all these and other factors can ultimately lead to the tests running under different conditions and what works (in terms of timing) to load pages, javascript, css etc. and execute ajax requests can be quite different and ultimately leading to the case that what works on one system... sometimes doesn't work on another.

    The end result, unfortunately, is that you will need to work on these failure to make sure they pass in both environments before considering the work 'done'.

    You will need to work with:

    • explicit waits
    • implicit waits
    • ajax issues

    Most of my UI automation work time over the past 2 years has been spent dealing with issues like this and in particular, intermittent failures.

    The problems are exacerbated by the fact that the selenium drivers for different browsers can in themselves have different issues. I am currently working with a test suite that gets different failures in Chrome, Firefox, Safari 8, Safari 9, IE 11 and IE 13 and in each case most of the failures are specific to the browser driver and pass in all the others browser drivers.

Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2