What are Test Metrics? Parameters to objectively measure the software testing process on various aspects
Why do we need Test Metrics? • To quantitatively analyze the current level of maturity in testing and set goals/objectives for future • To provide means for managing, tracking and controlling the status of testing • To provide a basis for estimation (Today’s data is tomorrow’s historic data) • To objectively measure the effectiveness and efficiency of testing • To identify areas for process improvements • To give insight into the quality of the product
Base Metrics 1.Project Management Metrics 2. Test Progress Metrics 3.Defects Metrics Derived Metrics 1.Test Efficiency Metrics 2. Test Effectiveness Metrics 3. Group Standard Testing Metrics
Project Management Metrics - Effort Variance - Schedule Variance - Defect Density - Residual Defect Density Test Progress Metrics - Test Case Preparation Status - Test Execution Status - Planned Vs Actual Execution Defect Metrics - Defect Distribution by Severity/Status - Defect Distribution by Root Cause - Defect Removal Efficiency(DRE) Test Efficiency Metrics - Test Case Preparation Productivity - Test case execution Productivity - Test Case review Productivity - Test Efficiency - Review Efficiency - Review effort - Rework effort - Rejected Defects - Resource Utilization factor Test Effectiveness Metrics - Defect Removal Efficiency - Requirements Coverage - Test Coverage - Cost of Quality Group Standard Testing Metrics - Testing Effort - Total Effectiveness Indicator
Schedule Variance Description: This metrics quantifies the amount of slippage from the original planned schedule of the project. Formule:
Schedule variance = (Actual duration in days – Planned duration in days) --------------------------------------------------------------* 100 Planned duration in days
Effort variance Description: This metrics quantifies the slippage from the original estimated effort of the project. Formule:
Effort variance = (Actual effort – Original estimated effort) --------------------------------------------------------------* 100 Original estimated effort
Defect Density Description: This metrics gives the number of defects detected divided by a suitable measure of product size such as lines of code or FP. Formule:
Defect Density = Total no of defects ------------------------- Size unit
Residual Defect Density Description: This metrics gives the number of defects detected after release to customer by a suitable measure of project or product size Formule:
Residual Defect Density = # of defects found after system testing (QA testing) ------------------------------------------------------------ Size
Defect Removal Efficiency(DRE) Description: This metrics quantifies the efficiency of review and testing process Formule:
DRE = # of defects found until and including system testing ---------------------------------------------------------------- * 100 Total # of defects Review Efficiency Description: Thus metrics quantifies the efficiency of review process Formule:
Review Efficiency = # of defects found until review -------------------------------------- * 100 Total # of defects
Review Effort Description: This metrics quantifies the effort spent on review Formule:
Review effort = Total review effort ------------------------ * 100 Total actual effort
Rework Effort Description: This metrics quantifies the effort spent on rework Formule:
Rework = Total rework effort -------------------------- * 100 Total actual effort
Test Case Preparation Productivity No of test cases prepared per person months Formule:
TC Preparation Productivity = No of test cases prepared ---------------------------------------- Total time spent in person months
Test Case Execution Productivity Description: No of test cases executed per person months Formule:
TC execution productivity = No of test cases executed ---------------------------------------- Total time spent in person months
Test Case Review Productivity Description: No of test cases reviewed per person months Formule:
TC Review Productivity = No of test case reviewed ----------------------------------------- * 100 Total time spent in person months
Test Efficiency Description: This metrics quantifies the efficiency of the testing process Formule:
Test Efficiency = # of defects found upto and including system testing ----------------------------------------------------------------- * 100 # of defects found during and after testing
Resource Utilization factor
Description: This metrics is an indicator of how well the resources are being utilized and will help take a decision whether the resources present are sufficient or not. Formule
Resource utilization Factor = Actual Effort in hrs ------------------------------------------------ * 100 No of resources * No. of days
Bug Rejection rate (internal & external) Description: This metrics gives the no of bugs re-opened by QA and Customer Formule: Bug Rejection rate = No of bugs re-opened ---------------------------- * 100 Total no. of Bugs
Rejected Defects Description: This metrics gives the no of invalid defects logged by QA Formule:
Rejected Defects = Number of defects rejected (invalid) ----------------------------------------------- * 100 Total number of defects logged
Requirement Coverage Description: Indicates the distriburion of requirements covered by test casesalong with status Formule: Prepare traceability matrix along with test case execution result.
Test Coverage Description: this metrics gives howmany defects were logged by QA by performing adhoc/exploratory testing Formule: # Valid defects not mapped to test cases Vs # valid defects mapped to test cases
Cost of Quality Description: Formule: Total testing effort (person days) ------------------------------------------- # Valid defects found Testing Effort Description:
Formule: Actual Testing Effort ----------------------------- Total Project Effort (in person hours)
Total Effectiveness Indicator Description: Measures the proportion of the number of defects discovered in all formal testing stages i.e System testing, UAT, performance testingetc, with the discoveredin the first month of production operations.
Formule: Total # of defects found in all test stages ------------------------------------------------------ Number of latest defects during first month after implementation
Note: Template is under preparation for all the above metrics and graphs with dummy data. Share over here soon