TestNG how to build a queue of test cases to be retried



  • Currently, we have logic in place which retries all the failing test cases immediately after the failure, what could be any approach where if a test fails that test should go in a queue and should be retried when all other test cases are executed at least once.

    our current logic works like following-

    private static int MAX_RETRY_COUNT = ExecutionConfig.TEST_CASE_RETRY_COUNT;
        AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);
    
    @Override
            public boolean retry(ITestResult result) {
                boolean retry = false;
                if (count.intValue() > 0) {
                    retry = true;
                    count.decrementAndGet();
                }
                return retry;
            }
    


  • This is what TestNG does not support "natively". However here is some example that shows how you can approximately approach to your problem:

    DISCLAIMER: There is no warranty that this would be working for any other real use-case except of the given example:

    package click.webelement.testng.rerun;
    
    import org.testng.*;
    import org.testng.annotations.AfterClass;
    import org.testng.annotations.Test;
    
    import java.util.ArrayList;
    import java.util.List;
    import java.util.stream.Collectors;
    
    public class RerunTest{
    
        static int retries = 0;
        final static int MAX_RETRIES = 2;
    
        @Test
        public void successfulTest(){
            System.out.println("Success");
            Assert.assertTrue(true);
        }
    
        @Test
        void failedTest(){
            System.out.println("Failed test");
            Assert.assertTrue(false);
        }
    
        @Test
        public void anotherSuccessfulTest(){
            System.out.println("Another one success");
            Assert.assertTrue(true);
        }
    
        @Test
        void anotherFailedTest(){
            System.out.println("Another failed test");
            Assert.assertTrue(false);
        }
    
    
        @AfterClass
        public void retry(ITestContext cntx){
            if(retries < MAX_RETRIES){
                TestNG testng = new TestNG();                        // This is the tricky thing
                testng.setTestClasses(new Class[]{this.getClass()}); // If you know how to obtain current executor, you will work around this
                testng.setMethodInterceptor(new IMethodInterceptor() {
                    @Override
                    public List intercept(List methods, ITestContext context) {
                        List toRerun = new ArrayList<>();
                        for(IMethodInstance methodInstance: methods){
                            if(cntx.getFailedTests().getAllMethods().stream().anyMatch(t -> t.getMethodName().equals(methodInstance.getMethod().getMethodName()))){
                                toRerun.add(methodInstance);
                            }
                        }
                        return toRerun;
                    }
                });
                retries++;
                System.out.println("Retrying: " + cntx.getFailedTests().getAllMethods().stream().map(t -> t.getMethodName()).collect(Collectors.toList()));
                testng.run();
            }
        }
    
    }
    ```
    
    
    

    This will likely use different context for your rerun tests so that you will likely run into issues with reporting and many other things.



Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2