Combining coverage from unit tests and functional tests/Selenium
Has anyone had experience getting a combined code coverage measure across both unit tests and functional/UI automation tests? I'm in a Java stack with Cobertura, Jenkins and Ant. I can successfully get the unit test coverage report in Jenkins already, and I am looking to get that build to also spin up an app server with an instrumented version of the app, and include the coverage from the UI automation along with the unit tests. What I'm hoping to do is identify the code that is covered by the UI automation tests, so I could eliminate some unit tests that don't test any meaningful behavior on their own and are there mostly to increase coverage. In other words, I'd like to have a way to report that I don't need a unit test for a controller class (for example) in a Spring MVC/Grails application because it is already covered by the UI automation, and because there is no real business logic in the controller by itself.
Yes, we do just that. We use Python and Coverage. I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path. OK, after comment from OP: yes, it is valid reason to eliminate unit test: you mocked the heck out of controller without really testing much. Especially if maintaining mocks is brittle, tedious and does not buy you much because you need to test it with real life examples anyway. If you have some specific business logic for controller, unit-test it. But 100% code coverage is wrong goal. Better goal is to get the best value for the time of developers and testers spend on writing and maintaining both production code and test code. Code (production, unit-tests, system-tests) is code, you need resources to maintain it, modify it in response to changes in requirements. What is best business value for resources spent? What makes your customers happy? If your code is covered twice over, by both unit and system tests, maybe you need channel more resources for development to have more features to be tested