Continuous Quality Engineering in DevOps: Optimizing Automated Testing for Rapid Enterprise Software Delivery

Abstract

The software delivery of enterprises has become reliant upon the use of continuous integration and continuous delivery pipelines. With conventional regression testing, large sequential suites are used and this imposes delay and poor feedback to the development teams. This paper proposes a quality improvement model that considers testing a continuous engineering effort within the DevOps pipeline as opposed to a bottleneck during the release phase. The framework is a combination of change impact analysis, historical test failure behaviour with business criticality that is converted into a single risk score being used to select and prioritize tests. A test case based simulated enterprise scenario involving 1240 test cases indicates that the risk aware approach is quicker to execute, finds faults better than random and coverage-based techniques. The five quality engineering indicators, which relate testing efficiency to the delivery performance are defined in the paper. The experiment shows that risk conscious prioritization provides quicker feedback and quality releases.