Accelerating Business Value and Improving Quality Through Large-Scale Test Automation

Description

Intel IT planned the deployment of the Vistex rebate management solution, based on the SAP platform, over three releases. The solution was part of a portfolio of modernization initiatives through which Intel sought to standardize its contract structures, rebate accruals, and settlement and payment processes. After Release 1 of the program was deployed, new business requirements dictated that internal users and external customers needed the solution sooner than initially planned.The team accelerated deployment of Release 2 and Release 3 by applying agile methodologies including test quality practices like end-to-end integration testing. As a result, Intel IT was able to meet the new schedule commitments. Compared to Release 1, the team reduced the testing time of Release 2 by 50 percent and Release 3 by 79 percent. The team achieved these results even as the number of test cases rose in each release: compared to Release 1, test cases increased by 25 percent in Release 2 and by 24 percent in Release 3.The team attributed much of the reduction in delivery time to automation. By Release 3, Intel IT had automated 66 percent of the end-to-end integration tests and 94 percent of the regression tests. Test-automation efficiency (defined as the ratio of effort saved by automating testing to the effort required for manual testing) was 74 percent for both Releases 2 and 3. Intel IT achieved these levels of accelerated delivery and test efficiency while remaining within the original budget and resources. The team also created a reusable testing framework that can now be applied to other programs. Post-program efficiency was calculated at 75 percent.By delivering Release 2 and Release 3 on schedule and with high quality, Intel’s internal stakeholders were able to onboard customers faster and gain access to real-time financial information sooner. This allowed Intel’s business groups to respond to market demands more rapidly with faster business decisions.This paper shares how Intel IT accelerated business value and improved quality through a large-scale test automation program. It describes the team’s use of possibility thinking to jumpstart the pivot to agile methodologies, and it outlines the six test quality practices that the team adopted. The paper also describes Intel IT’s development of a test-automation framework, highlighting both automation efficiency and end-to-end integration testing. By sharing details of the program, Intel IT hopes to enable other teams to build a framework that delivers quality software faster, leading to more rapid realization of business value.