雲端平行測試策略效能評估之研究

A Study on the Performance Evaluation of Cloud Testing Strategies

林憲良(Hsien-Liang Lin)

指導教授 ： 劉建宏

國立臺北科技大學/電資學院/資訊工程系研究所/碩士(2012年)

https://doi.org/10.6841/NTUT.2012.00358

摘要

軟體測試通常占軟體開發成本的30%至50%，因此如何有效提升軟體測試效率與降低測試成本一直是重要的研究課題之一。近年來由於雲端運算蓬勃發展，透過雲端強大的計算能力與虛擬化的特性來提升測試效率和降低測試成本逐漸受到廣泛的注意。有鑑於此，本論文嘗試辨識與分析影響雲端測試效率和成本的因素，並根據不同因素的組合，提出數個雲端平行測試策略，以及探討不同的測試策略對雲端測試效率和成本的影響。特別是本論文分析測試工作大小與測試工作分配方式對測試效率和成本可能的影響，並提出不同的測試工作分割基準以及工作分配策略機制，以減少測試的執行時間。此外，我們亦考慮雲端測試平台可能的資源限制，作為測試工作分割與分配的參考。為了評估不同雲端平行測試策略的執行效能，我們開發一個基於OpenStack平台的雲端測試框架，並進行數個實驗來觀察不同測試工作分割與分配，以及資源限制下，測試執行時間的變化。實驗結果顯示，當虛擬機器數量增加時，效率並不會以等比率提升；當完成測試所需時間較長時，選擇以平衡測試時間方式分割和動態分配測試工作之策略，會隨著虛擬機器數量增加而有較佳的測試效率，反之若完成測試所需時間較短時，則以平衡測試時間方式分割和靜態分配測試工作之策略，有較佳的測試效率；另外，當採用動態分配工作策略時，將一個測試分割成較多的平行工作時，隨著虛擬機器數量增加，相對於將該測試分割成較少的平行工作時，其測試效率曲線提升幅度較快。

關鍵字

雲端測試 ； 雲端運算 ； 雲端平行測試策略

並列摘要

Software testing usually makes up 30% to 50% of software development cost. Thus, how to increase testing efficiency and reduce test cost has been considered as an important research issue. In recent years, due to the emergence of cloud computing, leveraging the characteristics of cloud computing, such as elastic provision and virtualization, for increasing testing efficiency and reducing test cost has drawn much attention. Therefore, this thesis aims to identify and analyze the factors affecting cloud testing efficiency and cost, proposes several cloud testing strategies based on the variations of the factors, and studies the influences of these testing strategies to the testing efficiency and cost. In particular, the possible effects caused by testing task size and dispatching policy are analyzed. Several testing task partitioning and dispatching policies are presented in order to minimize the overall test execution time and cost. Moreover, the resource constraints imposed by cloud testing platform are also considered when partitioning and dispatching testing tasks. To evaluate the performance of the proposed cloud testing strategies, a Cloud Testing Framework based on the OpenStack is developed and several experiments are conducted. The experimental results suggest that increasing the number of virtual machines does not linearly improve the overall testing efficiency. For a testing job that takes a long time to finish, the strategy with balancing test execution time and dynamic task dispatching policy has a better performance than others when the number of virtual machines increases. In contrast, for a testing job that takes a short time to complete, the strategy with balancing test execution time and static task dispatching policy generally outperforms others. Besides, when using the dynamic dispatching policy, the curve of testing efficiency will rise more quickly as the number of virtual machines increase for a job partitioned to more parallel testing tasks than that partitioned to fewer parallel testing tasks.

並列關鍵字

Cloud Computing ； Cloud Testing ； Cloud Testing Strategy

參考文獻

Ji Wu, Chunhui Wang, Yang Liu and Li Zhang, “AGARIC-a Hybrid Cloud Based Testing Platform,” In Proceedings of the International Conference on Cloud and Service Computing (CSC), pp. 87-94, December 2011.

Niclas Snellman, Adnan Ashraf and Ivan Porres, “Towards Automatic Performance and Scalability Testing of Rich Internet Appications in the Cloud,” In Proceedings of the 37th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 161-169, September 2011.

Huan Liu and Dan Orban, “Cloud MapReduce a MapReduce Implementation on top of a Cloud Operating System,” In Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 464-474, May 2011.

Selenium Web Application Testing System, http://seleniumhq.org/

Amazon Elastic Compute Cloud, http://aws.amazon.com/ec2/