Does codeless automation tools work for large test suites?



  • AI-based codeless automation tools are on the rise. There are plenty of them that even claim to have flexibility and reusability.

    Are they good enough for large complicated test suites with many thousands of test cases?
    Is strong programming skills not required for automation when using these tools?

    Three examples:

    Leapwork: https://www.leapwork.com/blog/codeless-test-automation
    "codeless selenium with AI maintenance"

    Perfecto: https://www.perfecto.io/codeless-automation
    "AI-driven test automation creation"

    Applitools: https://applitools.com/
    "AI powered functional and visual testing"



  • In many cases, no

    The first automation case is easy to write - no page objects, no need to use DRY, etc.

    By the 100th test, you need a lot of programming - or codeless automation suites.

    The problems really start when you then continue to use the

    add or change functionality or bug discovered / fixed
      \|/
    write new UI test
      \|/
    repeat
    

    The test suite starts to grow and before long takes first hours and then days to run. The number of test cases keeps growing (I've seen tens of thousands), as do the intermittent errors that destroy the confidence of the tests adding value. Multiply test cases by browsers and devices and versions and orientation and the Universe might end. Ultimately the original intent of rapid feedback disappears and QA becomes a separate function that does checking and verification but doesn't directly influence quality in the context of application code creation by developers. Testing gets done for the sake of testing and quality does not improve long term.

    It also becomes apparent that you should be following the principles behind the Agile Testing Pyramid with the Unit, Integration and UI layers and just using a tool to write ever more UI cases does not promote this conversation and action within the organization.

    Currently approaches such as 'keep the test suite under 15 mins by deciding which end to end UI tests are needed and which existing ones could be moved to integration or unit tests' are not approaches that AI test automation can handle as far as I'm aware. Maybe google have it internally.



Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2