An important role in the project development lifecycle
The QA metrics and specific project KPIs have a very important role in the project development lifecycle, as they are used to:
Asses the Quality
of the delivered services
Continuous Improvement
through analysis and act to fix and improve
Determine the Effectiveness
of the quality assurance strategy and activities
Determine the Evolution
of the team work outcome – productivity & quality improvements
We use the following main categories of QA metrics and KPIs (grouped by functional or domain area), to identify the project areas that needs improvement, and also to guide the improvement activities:
Systems High-availability
- Modules/services availability
- Uptime numbers: average uptime and variance
- Continuity of services
System Performance
- System components performance targets compliance (response time to various operations)
Change Management
- Successful and failed changes
- Changes backlog timeline and variance
- Mean RFC (Request for Change) turnaround time
Monitoring and Auditing
- System monitoring and auditing quality targets compliance
- Incidents & issues management
Incidents & Issues Management
- Number of incidents reported per module/service area
- Number of escalations
- Number of repeated Incidents
- Mean time between incidents are registered
- Mean time between high-priority incidents are registered
- Incidents severity/priority distribution (timeline)
- RCA (Root Cause Analysis) incidents distribution
- Average period for open issues
- Average time to fix an issue
- Average issues fix cost (man-hours)
- Reopened issues rate
- Ratio of accepted/rejected/postponed issues
- Incidents resolved remotely
Service Validation and Testing
- Failed Release Acceptance Tests
- Incidents caused by new releases
Service Asset and Configuration Management
- Incidents due to inaccurate configuration management information
- Number of unauthorized changes
- Configuration management coverage
- Average effort for configuration management verifications
Backup and Recovery
- Successful completion of systems backup activities
- Successful completion of systems restore activities
Delivery Performance
- Operations/tasks completed on time
- Completeness
As a general rule, metrics are customized for each activity and each project. The number of metrics deployed must be kept as small as possible and they must complement each other.