運用軟體測試覆蓋率資訊於歷史值方法為基礎之測試個案優先順序方法

Applying Software Testing Coverage Information to Historical-Value-Based Approach for Test Case Prioritization

劉韋男(Wei-Nan Liu)

指導教授 ： 黃慶育

國立清華大學/電機資訊學院/資訊工程學系/碩士(2009年)

摘要

軟體開發過程中，回歸測試是經常被執行來維護軟體的測試程序，它用來驗證修改過後的軟體系統。然而回歸測試的測試個案的數量大小會隨著時間的推移與新功能的加入而顯著的增加。所以回歸測試所花費的時間也會隨之愈來愈大，為了降低回歸測試所花費的成本，軟體測試者可以根據一些測量基準或是考量在有限的時間限制中執行較重要的測試個案。雖然目前的有許多測試個案優先排列技巧可以改善回歸測試的效率，但可惜的是既存的測試個案優先排列的方法把每次的回歸測試視為獨立的過程。執行回歸測試的過程並未考量過去執行的經驗，使得某些較重要的測試個案不斷重覆的被執行，而某些較不重要一點的測試個案永遠沒有被執行的機會。雖然Kim和Porter為了克服這個問題有提出過一個根據歷史資料的回歸測試優先排列方法，不幸的是這種方法反而只考慮歷史資訊而忽略了其他在回歸測試優先排列的方法中既存的重要的考量。在此論文中，我們提出一個方法來改良在資源受限的環境中以歷史資訊為基礎的優先排列法。通過調查其他重要的量測以及考量，我們將覆蓋率加入我們所提出的方法中，使得新的方法不僅會考慮到歷史資訊更會把程式碼為基礎的測試個案優先排列法的優點帶入其中。此外，我們還分析了測試個案選擇的比例與測試個案的錯誤偵測能的關係，從而找出適當的測試個案的選擇大小或比例的關係。

關鍵字

軟體測試 ； 軟體測試個案優先順序 ； 歷數值優先順序

並列摘要

Regression testing is a frequently executed maintenance process used to revalidate a modified software system. However, the test suite size of regression testing may grow significantly with new functionality over time. Thus it would take a lot of time to perform regression testing over and over again. To reduce the cost of regression testing, software testers may prioritize their test cases according to some measurements and run more important test cases within existing time constraints. However, most existing prioritization techniques regard regression testing as independent process. They didn’t take any advantage from past execution experience. Some important test cases would be executed repeatedly whereas some test cases would never be executed. To overcome this problem Kim and Porter proposed a history-based prioritization technique that can execute test cases based on the history execution profile. Unfortunately this approach only takes into account the history execution data without pertain some other important criteria in regression testing prioritization. In this thesis, we propose a method to improve the performance of the history-based prioritization technique for resource constraint environments. By investigating other important measurements and criteria, we incorporated the coverage information into the proposed method so that it will consider not only the history data but also the power of code-based prioritization techniques. Moreover, we analysis the relationship of the ratio of test case selected and executed with the fault detection ability of our proposed method to find out what is the appropriate test case selection ratio for regression testing.

並列關鍵字

software testing ； software testing case prioritization ； historical value based priorization

參考文獻

[1] “IEEE Standard Glossary of Software Engineering Terminology,” IEEE Std 610.12-1990, 1990.

[1] P.C. Jorgensen, Software Testing: A Craftsman's Approach, CRC Press, 2002.

[4] Y. Chen, D.S. Rosenblum, and K. Vo, ”TestTube: A System for Selective Regression Testing,” Proceedings of the 16th international conference on Software engineering, pp. 211-220, May 1994.

[5] M. Harrold and M. Souffa, “An Incremental Approach to Unit Testing During Maintenance,” Proceedings of the Conference on Software Maintenance, pp. 362-367, Oct 1988.

[6] J. Hartmann and D.J. Robson, “Techniques for Selective Revalidation,” IEEE software, vol. 7, No.1, pp. 31-36, 1990.