軟體測試與除錯階段之可靠度量化分析與管理

Reliability Analysis and Management during Software Testing and Debugging

林楚迪(Chu-Ti Lin)

指導教授 ： 黃慶育

國立清華大學/電機資訊學院/資訊工程學系/博士(2009年)

https://doi.org/10.6843/NTHU.2009.00137

摘要

在開發穩固且高品質軟體產品的過程中，軟體可靠度的測量扮演相當重要的角色。過去三十幾年來，已有專家學者提出許多軟體可靠度成長模型，並應用於軟體開發流程中以評估產品的可靠度。然而，存在的可靠度模型大多假設所偵測到的錯誤會立即被移除。因為移除錯誤所需的時間和除錯人員的技術與經驗、除錯團隊的規模...等因素息息相關，這個假設在實務上顯然是不合理的。此外，軟體測試的實務經驗亦告訴我們，若錯誤之間不具關聯性，則錯誤可直接被偵測到並分別移除。但若錯誤之間存在相依性，某些錯誤只能在其前置錯誤被移除後才能被移除。因此，具相依性的錯誤無法被立即移除，並且錯誤移除的流程必定會落後於錯誤偵測流程。本文將討論錯誤之間的關係性，並且研究如何將錯誤關係性和除錯時間延遲這兩項重要因素導入軟體可靠度模型。
我們亦注意到多數軟體可靠度成長模型並不考慮測試心力消耗速率可能產生的變動。在某些情況下，測試資源的配置可能會產生急速地變動或調整。因此，我們將多重變動點的概念加入韋伯型式函數中。亦藉由所提出的模型分別就軟體可靠度和花費成本的角度來討論最佳的軟體釋放時機。目的是在滿足特定可靠度需求的前提下，將軟體開發所需成本降到最低。
除了軟體可靠度模型之外，過去十年內已有數個應用失效率的模擬方法被用於軟體失效過程的評估。但這些方法大多忽略除錯人員的數目限制(即假設除錯人員的數目是無限的)。在實務上，專案管理者會考慮到所花費之人事成本，除錯人員的數目不可能是無限制的。因此，當錯誤被偵測到時有閒置的除錯人員，則除錯人員能夠立即著手尋找錯誤的根源，並在找出錯誤後加以修正；但是，當錯誤被偵測到時已無閒置的除錯人員，則該錯誤將先被存放於佇列中。本文將應用排隊理論來敘述與解釋軟體開發過程中的除錯行為。研究主體將分別以G/G/∞和G/G/m兩種佇列模型為基礎，提出兩個模擬程序。我們並以真實的錯誤資料來說明所提出的模擬方法。藉由所提出的模擬程序，軟體專案管理者將可了解除錯人員的數目增減對軟體錯誤修正效率所造成的影響。最後，以分析所得的結果決定最符合成本效益的除錯團隊之規模。

關鍵字

軟體測試 ； 軟體可靠度 ； 軟體失效過程 ； 錯誤偵測流程 ； 錯誤移除流程 ； 測試心力 ； 變動點 ； 排隊理論

並列摘要

Software reliability measurement plays a very important role in developing a robust and high quality software product. Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of products during software development processes. Usually it is assumed that detected faults are immediately corrected when mathematical models are developed. This assumption may not be realistic in practice since the time to remove a detected fault depends on the skill and experience of personnel, the size of debugging team, and so on. Besides, during software testing, practical experiences show that mutually independent faults can be directly detected and removed, but mutually dependent faults can be removed if and only if the leading faults have been removed. Thus, the dependent faults may not be immediately removed and the fault removal process lags behind the fault detection process (FDP). In this dissertation, we will discuss the software fault dependency and study how to incorporate both fault dependency and debugging time lag into software reliability modeling.
We also noticed that most SRGMs do not take into account possible changes of testing-effort (TE) consumption rates. However, in some cases, the policies of testing-resource allocation could be changed or adjusted. Thus, we will incorporate the concept of multiple change-points (CPs) into Weibull-type testing-effort functions (TEFs). Based on the proposed SRGMs, constructive rules are developed for determining optimal software release times (OSRTs) in terms of cost-reliability criterion. The main purpose is to minimize the cost of software development when a desired reliability objective is given.
In addition to SRGM, the attractive rate-based simulation approaches have been proposed to describe the fault detection and correction processes during software testing phase. Thus far, it appears that most existing simulation approaches do not take into account the number of available debuggers. In practice, the number of debuggers will be carefully controlled. If all debuggers are busy, newly detected faults may have to wait (for a long time to be corrected and removed). Therefore, in this dissertation, we will apply the queueing theory to describe and explain possible debugging behavior. Two simulation procedures are developed based on G/G/∞ and G/G/m queueing models, respectively. The analysis through the proposed framework may greatly help project managers to assess the appropriate staffing level for the debugging team from the standpoint of performance and cost-effectiveness.

並列關鍵字

Software testing ； Software reliability ； Software failure process ； Fault detection process ； Fault correction process ； Testing effort ； Change point ； Queueing theory

參考文獻

[Hua04a] C. Y. Huang, C. T. Lin, S. Y. Kuo, M. R. Lyu, and C. C. Sue, “Software Reliability Growth Models Incorporating Fault Dependency with Various Debugging Time Lags,” Proceedings of the 28th Annual International Computer Software and Applications Conference, pp. 186-191, September 2004, Hong Kong, China.

[Lin07] C. T. Lin, C. Y. Huang, and C. C. Sue, “Measuring and Assessing Software Reliability Growth through Simulation-based Approaches,” Proceedings of the 31st IEEE Annual International Computer Software and Applications Conference, pp. 439-446, July 2007, Beijing, China.

[Sch02] N. F. Schneidewind, “An Integrated Failure Detection and Fault Correction Model,” Proceedings of 18th International Conference on Software Maintenance, pp. 238-241, October. 2002, Montreal, Quebec, Canada.

[Rae03] M. Raetzmann and C. D. Young, Galileo Computing Software Testing and Internationalization, LISA/Lemoine International, 2003.

[Ant04] G. Antoniol, A. Cimitile, G. A. Di Lucca, M. Di Penta, ”Assessing Staffing Needs for a Software Maintenance Project through Queuing Simulation,” IEEE Trans. on Software Engineering, Vol. 30, No. 1, pp. 43-58, January 2004.