提升嵌入式系統圖形化介面測試之精確度

基於省電與成本考量，嵌入式系統所配有的運算資源通常有限，要將現有的圖形界面測試技術套用在嵌入式系統軟體開發流程上是非常困難的。隨著智慧型手持裝置(如手機、平板)的普及，在嵌入式系統上進行自動化圖形化介面測試的需求也大幅提高，但現有的測試工具在使用時會有自動化不精確的問題。我們基於現有的測試工具Sikuli以遠端測試架構為基礎實作了圖形化介面自動化測試工具SPAG，以提升測試事件重播的精確度。SPAG將遠端手持式裝置的畫面顯示在本機端，讓使用者可以透過本機端畫面操作手持式裝置並錄製自動化測試項目。我們設計批次事件傳送與智慧等待兩種方法並實作在SPAG中，用以改善滑動等手勢的重播精確度，同時避免待測程式延遲造成的測試失敗。SPAG會自動評估待測程式是否產生延遲，並等待額外的時間讓待測程式完成反應，以提高測試指令的完成機率。基於輔助功能技術，SPAG能錄製流程中自動偵測待測程式狀態的改變，產生驗證指令，並在測試時自動驗證手持式裝置之狀態。我們選定一常用自動化測試工具monkeyrunner來比較測試精確度。在指定待測系統負荷工作量的情況下，SPAG自動化測試的精確度仍然達到90%以上。若依照一般測試流程估計，SPAG 將可以比monkeyrunner少用11%~71%的測試回數。

關鍵字

嵌入式系統；圖形化介面；自動化測試； Android

並列摘要

The embedded systems are built with limited computation resources. Therefore, it is hard to apply conventional GUI testing tools on embedded systems. However, with the popular of smart handheld device, e.g. smart phone and pads, more software run on handheld devices and rising the need of software testing on handheld device. Current testing tools on embedded systems all have defect on reproducing GUI operations, so we design a Smart Phone Automated GUI testing tool (SPAG) based on Sikuli, in order to reduce testing defects. We design Batch Event and Smart Wait method to reproduce GUI operations accurately and ensure the device under test (DUT) can fully process all test operations. We also design the Hybrid Verifying method based on accessibility technology to generate most verify operations and verify them automatically. We use an experiment to compare our SPAG with monkeyrunner on test accuracy and time efficiency, in which SPAG archives 90% testing accuracy with all types of workload while monkeyrunner only has test accuracy of 27%~88%. We use the second experiment to analyze the contribution of our methods. The testing result shows our Smart Wait method brings more improvement on test accuracy.

並列關鍵字

embedded system ； GUI ； automated testing ； Android

參考文獻

[2] T. Yeh, T.-H. Chang, and R. C. Miller, "Sikuli: using GUI screenshots for search and automation," presented at the Proceedings of the 22nd annual ACM symposium on User interface software and technology, Victoria, BC, Canada, 2009.

[3] T.-H. Chang, T. Yeh, and R. C. Miller, "GUI testing using computer vision," presented at the Proceedings of the 28th international conference on Human factors in computing systems, Atlanta, Georgia, USA, 2010.

[4] T.-H. Chang, T. Yeh, and R. Miller, "Associating the visual representation of user interfaces with their internal structures and metadata," presented at the Proceedings of the 24th annual ACM symposium on User interface software and technology, Santa Barbara, California, USA, 2011.

[5] Q. Xie and A. M. Memon, "Using a pilot study to derive a GUI model for automated testing," ACM Trans. Softw. Eng. Methodol., vol. 18, pp. 1-35, 2008.

[9] J. Bo, L. Xiang, and G. Xiaopeng, "MobileTest: A Tool Supporting Automatic Black Box Test for Software on Smart Mobile Devices," presented at the Proceedings of the Second International Workshop on Automation of Software Test, 2007.

國際替代計量

提升嵌入式系統圖形化介面測試之精確度

全文下載

主題瀏覽