Tool to track the testing pyramid state
irl last edited by
Is there any tool for tracking testing pyramid state which visualises each stage of pyramid and where I can track disproportions of that?
p.s. I did a quick search and as I can see there are a plenty of tools for code coverage (this is mostly for unit tests) but nothing for tracking a whole pyramid.
I like the guidance from https://automationpanda.com/tag/testing-pyramid/: each layer of tests should be an order of magnitude smaller than below.
The Testing Pyramid should be a guideline, not a hard rule. Don’t require hard proportions for test counts at each layer. Why not? Arbitrary metrics cause bad practices: a team might skip valuable end-to-end tests or write needless unit tests just to hit numbers. W. Edwards Deming would shudder!
Instead, use loose proportions to foster better retrospectives. Are we covering too many input combos through the Web UI when they could be checked via service tests? Are there unit test coverage gaps? Do we have a pyramid, a diamond, a funnel, a cupcake, or some other wonky shape? Each layer’s test count should be roughly an order of magnitude smaller than the layer beneath it. Large Web apps often have 10K unit tests, 1K service tests, and a few hundred Web UI tests.
The problem generally is that Unit tests, Integration tests and UI tests are written in separate efforts. With different frameworks, languages, reporting, etc.
I think your two options are:
Use an complete testing solution vendor and get feeds from all the different testing areas to construct a visualization of your pyramid.
Manually have each layer communicated test totals to another small application that aggregates them for display.