To enable DevOps, teams are focusing on delivering high value with high quality fast. This translates into efficient Continuous Integration (CI) that builds new software fast and validates its quality in each step of the development process – that’s Continuous Testing (CT). Only by having reliable CI and CT, teams can push software fast and automatically to their end users, as part of their Continuous Delivery/Deployment (CD).
The QA manager role in assuring the fluent process of these 3 pillars is fundamental. QA management should have maximum quality visibility in each stage starting from design, implementation, build acceptance, integration testing, and production testing. Such visibility means, that the manager needs actionable test data to base a smart decision on.
Continuous Test Measurement & ImprovementDevOps and software development, in general, is a continuous process. Innovation does not stop at the end of a sprint, nor does the test management and measurement.
QA managers need to define metrics to help them measure efficiency from one sprint or release to the next. Such metrics would include things like:
Test productivity – which is the most efficient test scenarios?
What is the Pass/Fail rate between cycles?
Is the defect backlog declining or increasing?
How is the CI process progressing – is the cycle length increasing? Are builds becoming flaky?
Does my test coverage and platform coverage align with the market and business requirements? Is it aligned with product analytics/usage?
What is my overall test automation % vs. manual?
Each QA manager would need to have such metrics with criteria set to each. Such metrics ought to be relevant to his domain. When there is a degradation or miss to some of these metrics, he needs to communicate with his counterparts and build a plan to improve on them.
It is on the modern QA manager shoulders (it is a great opportunity as well) to take the lead, and drive the triangle of CI, CT, CD to make the overall software delivery pipeline as smooth, reliable, and efficient as possible