Translated Titles

GUI Testing with Component Abstraction





Key Words

GUI測試 ; 測試驅動開發 ; 錄影播放 ; 元件抽象化 ; 測試腳本 ; 測試案例產生 ; GUI testing ; TDD ; Component Abstraction ; test-case generation ; test scripts



Volume or Term/Year and Month of Publication


Academic Degree Category




Content Language


Chinese Abstract

圖形使用者介面(GUI)已被廣泛的使用在各種應用軟體中,因此GUI測試(GUI testing)是一個很重要的研究議題。一般而言,進行GUI測試必須先撰寫(或以工具產生)測試腳本(Test Script),以便自動執行大量的使用者事件與驗證的動作,本論文首先提出一個整合測試驅動開發與錄影播放測試的方法,讓測試者能獲得兩種方法的好處,我們研究成果也顯示這樣的整合方法比直接寫程式的方式來的有效率多了。其二,是現今的測試腳本技術,其實只有對GUI的行為進行抽象化,並沒有考慮到GUI結構,因此當GUI發生變動時,測試腳本就會變的難以維護。因此本論文提出了元件抽象化(Component Abstraction)方法,以物件基礎來描述GUI的結構為行為,研究結果也顯示使用元件抽象化,的確能提昇腳本的可讀性、可維護性、可重複使用性。其三,我們將元件抽象化與使用案例契約進行整合,來對GUI進行自動化的系統化測試,利用此方法可以有效的自動化產生測試案例,對GUI各種行為的互動,進行完整的系統測試,研究結果我們的方法能有效的找到GUI上的缺陷。

English Abstract

Most of today's software users interact with the software through a graphical user interface (GUI). Hence, GUI testing plays an important role in ensuring the correctness of software applications. To perform GUI testing, a test script must be prepared so that massive user interactions and verifications can be conducted automatically. Capture/replay (CR) is a common method that creates test scripts by recording user interactions of an application under test (AUT). However, a deficiency of CR technologies is that test scripts cannot be generated before the AUT is ready for testing. Thus, specification-based approaches have been studied as alternatives. This dissertation studies how a CR technology can be integrated with a specification-based approach and show that the integration is beneficial by a case study. Ideally, the actions of a test script should be organized based on the structure of the GUI so that the actions are easier to extend and maintain. Unfortunately, current methodologies and tools fall short in supporting such an organization. Thus, even a tiny change in GUI may break a large number of existing test scripts (cases). To overcome this problem, this dissertation proposes an “object-based” approach, called component abstraction, to model the structure of a GUI. A GUI testing modeling language, GTML, is defined and a systematic approach in applying component abstraction is described. Three case studies are conducted to show that a GTML script is both easier to maintain and extend than an ordinary test script. To perform a comprehensive GUI testing, a large number of test cases are needed. We propose a GUI test-case generation approach that is suitable for system testing. The key idea is to extend high-level GUI scenarios with contracts and use the contracts to infer the ordering dependencies of the scenarios. From the ordering dependencies, a state machine of the system is constructed and used to generate test cases automatically. A case study is conducted to investigate the quality of the test cases generated by the proposed approach. The results showed that, in comparison to creating test cases manually, the proposed approach can detect more faults with less human effort.

Topic Category 基礎與應用科學 > 資訊科學
機電學院 > 資訊工程系所
  1. M.-A. D. Storey, Eds. IEEE Computer Society, 2003, pp. 260–269.
  2. 155, 2006.
  3. [5] B. Westgarth. Crossword sage. Jul. 1, 2008 [Online]. Available:
  4. [6] R. Odell. Sudoku mania. Jul. 1, 2008 [Online]. Available:
  5. [7] Y.-S. Ma, J. Offutt, and Y. R. Kwon, “MuJava: an automated class mutation system,”
  6. 97–133, 2005.
  7. [8] A. M. Memon, A comprehensive framework for testing graphical user interfaces.
  8. Ph.D. dissertation, 2001, advisors: Mary Lou Soffa and Martha Pollack; Committee
  9. [11] B. Beizer, Black-Box Testing: Techniques for Functional Testing of Software and
  10. Systems, 1st ed. Verlag John Wiley Sons, Inc, 1995, mR: referenziert aus
  11. Manning Publications, 2007.
  12. [16] Q. Xie and A. M. Memon, “Designing and comparing automated test s for GUIbased
  13. [17] M. Fewster and D. Graham, Software test automation: effective use of test execution
  14. improvement through test driven development: results and experiences of four
  15. industrial teams,” Empirical Software Engineering, vol. 13, no. 3, pp. 289–302,
  16. June 2008. [Online]. Available: http://dx.doi.org/10.1007/s10664-008-9062-z
  17. and C. Stienstra, “Presenter first: Organizing complex gui applications for
  18. http://doi.ieeecomputersociety.org/10.1109/AGILE.2006.43
  19. [20] A. Ruiz and Y. W. Price, “Test-driven gui development with testng and abbot,”
  20. IEEE Software, vol. 24, no. 3, pp. 51–57, 2007.
  21. [23] J. Andersson and G. Bache, “The video store revisited yet again: Adventures
  22. LNCS, 2004, pp. 1–10.
  23. pp. 8–12, Apr. 2003.
  24. programming environment,” in Proceedings of the 2nd International Conference on
  25. Extreme Programming and Flexible Processes in Software Engineering (XP2001),
  26. [30] J. Chen and S. Subramaniam, “Specification-based testing for GUI-based applications,”
  27. [31] J. Chen, “Expressing graphical user’s input for test specifications,” in EDCIS, ser.
  28. Lecture Notes in Computer Science, Y. Han, S. Tai, and D. Wikarski, Eds., vol.
  29. 2480. Springer, 2002, pp. 347–359.
  30. GUI-based applications in java,” in HICSS, 2001.
  31. [33] Y. Sun and E. L. Jones, “Specification-driven automated testing of GUI-based java
  32. Available: http://jfcunit.sourceforge.net/
  33. capture/replay tools for graphical user interface testing,” in Proceedings of
  34. [38] M. Miura and J. Tanaka, “A framework for event-driven demonstration based on
  35. record/replay environment for multi-threaded java applications,” Softw., Pract. Exper.,
  36. [42] J. S. P. Ltd. Marathon. Sep. 17, 2009. [Online]. Available: http://www.
  37. [43] T.-H. Chang, T. Yeh, and R. C. Miller, “Gui testing using computer vision,” in
  38. [44] T. Yeh, T.-H. Chang, and R. C. Miller, “Sikuli: using gui screenshots for search
  39. and automation,” in UIST, A. D. Wilson and F. Guimbreti`ere, Eds. ACM, 2009,
  40. pp. 183–192.
  41. Conference on Reverse Engineering, Nov. 2003.
  42. [46] R. K. Shehady and D. P. Siewiorek, “A methodology to automate user interface
  43. testing using variable finite state machines,” in Proc. The Twenty-Seventh Annual
  44. International Symposium on Fault-TolerantComputing (FTCS’97). Seattle,Washington,
  45. USA: IEEE Computer Society, June 1997, pp. 80–88.
  46. [47] L. J. White and H. Almezen, “Generating test cases for GUI responsibilities using
  47. Society, October 2000, pp. 110–123.
  48. [48] F. Belli, “Finite-state testing and analysis of graphical user interfaces,” in Proc.
  49. 12th International Symposium on Software Reliability Engineering (ISSRE’01),
  50. 2001, p. 34.
  51. May 2000.
  52. Conference on Artificial Intelligence Planning and Scheduling. AAAI Press, Apr.
  53. 2000, pp. 226–235.
  54. [51] ——, “Hierarchical GUI test case generation using automated planning,” IEEE
  55. [52] A. Memon, A. Nagarajan, and Q. Xie, “Automating regression testing for evolving
  56. GUI software,” Journal of Software Maintenance and Evolution, vol. 17, no. 1, pp.
  57. [53] A. M. Memon, “An event-flow model of GUI-based applications for testing,” Software
  58. 884–896, 2005.
  59. C. Stienstra, “Presenter first: Organizing complex gui applications for test-driven
  60. development,” in AGILE, 2006, pp. 276–288.
  61. Available: http://fit.c2.com/
  62. [57] P. Klarck. Robot framework: a keyword-driven test automation framework. June.
  63. [58] A. M. Memon and M. L. Soffa, “Regression testing of GUIs,” in Proc. 9th European
  64. New York, NY, USA: ACM Press, 2003, pp. 118–127.
  65. [59] L. J. White, “Regression testing of GUI event interactions,” in Proc. 1996 International
  66. Conference on Software Maintenance (ICSM’96). Monterey, CA, USA:
  67. IEEE Computer Society, 1996, pp. 350–358.
  68. [60] L. J. White, H. Almezen, and S. Sastry, “Firewall regression testing of gui sequences
  69. and their interactions,” in ICSM. IEEE Computer Society, 2003, pp.
  70. [61] A. M. Memon, “Automatically repairing event sequence-based GUI test suites for
  71. Methods for Interactive Systems (FMIS 2007), ser. Electronic Notes in Theorectical
  72. and formal models for GUI testing,” Electr. Notes Theor. Comput. Sci., vol. 190,
  73. [66] J. Newmarch, “Testing java swing-based applications,” in TOOLS ’99: Proceedings
  74. [69] Z. Yin, C. Miao, Z. Shen, and Y. Miao, “Actionable knowledge model for GUI
  75. Computer Society, September 2005, pp. 165–168.
  76. [70] S. McMaster and A. M. Memon, “An extensible heuristic-based framework for
  77. Workshop on TESTing Techniques and Experimentation Benchmarks for
  78. Event Driven Software. Washington, DC, USA: IEEE Computer Society, 2009.
  79. Improving the Design of Existing Code, 1st ed. Addison-Wesley Professional,
  80. July 1999. [Online]. Available: http://www.amazon.com/exec/obidos/redirect?
  81. [72] H. Zhu, P. A. V. Hall, and J. H. R.May, “Software unit test coverage and adequacy,”
  82. ACM Comput. Surv., vol. 29, no. 4, pp. 366–427, 1997.
  83. [73] A. Memon, M. Soffa, and M. Pollack, “Coverage criteria for GUI testing,” in 8th
  84. European Conference and 9th ACM SIGSOFT Foundation of Software Engineering,
  85. 2001, pp. 256–267.
  86. [76] ——, “DART: A framework for regression testing nightly/daily builds of GUI applications,”
  87. in Proceedings of the International Conference on Software Maintenance
  88. 2003, Sep. 2003.
  89. of the 20th IEEE International Conference on Software Maintenance.
  90. Washington, DC, USA: IEEE Computer Society, 2004, pp. 8–17.
  91. [78] Q. Xie and A. M. Memon, “Rapid ”crash testing” for continuously evolving GUIbased
  92. tional Conference on Software Maintenance (ICSM’05). Washington, DC, USA:
  93. IEEE Computer Society, 2005, pp. 473–482.
  94. in ICSM ’06: Proceedings of the 22nd IEEE International Conference on Software
  95. Maintenance. Washington, DC, USA: IEEE Computer Society, 2006, pp. 145–
  96. [80] ——, “Studying the characteristics of a ‘good’ GUI test suite,” in Proceedings
  97. of the 17th IEEE International Symposium on Software Reliability Engineering
  98. (ISSRE 2006). IEEE Computer Society Press, Nov. 2006.
  99. [81] ——, “Designing and comparing automated test oracles for GUI-based software
  100. applications,” ACM Transactions on Software Engineering and Methodology,
  101. [82] R. C. Bryce and A. M. Memon, “Test suite prioritization by interaction coverage,”
  102. in Proceedings of The Workshop on Domain-Specific Approaches to Software Test
  103. [83] S.McMaster and A.M.Memon, “Fault detection probability analysis for coveragebased
  104. Conference on Software Maintenance (ICSM’07). Paris, France: IEEE
  105. Computer Society, 2007.
  106. [84] X. Yuan, M. Cohen, and A. M. Memon, “Covering array sampling of input event
  107. sequences for automated gui testing,” in ASE ’07: Proceedings of the 22nd IEEE
  108. international conference on Automated software engineering. Washington, DC,
  109. USA: IEEE Computer Society, 2007.
  110. [85] S. McMaster and A. M. Memon, “Call-stack coverage for GUI test-suite reduction,”
  111. [86] Q. Xie and A.M.Memon, “Using a pilot study to derive a gui model for automated
  112. testing,” ACM Trans. Softw. Eng. Methodol., vol. 18, no. 2, 2008.
  113. [87] P. Brooks and A. M. Memon, “Automated gui testing guided by usage profiles,”
  114. software engineering. Washington, DC, USA: IEEE Computer Society, 2007.
  115. [88] X. Yuan and A. M. Memon, “Generating event sequence-based test cases using
  116. [89] J. Strecker and A. M. Memon, “Relationships between test suites, faults, and fault
  117. IEEE Computer Society, 2008.
  118. Spoon, and A. Gujarathi, “Regression test selection for java software,” in OOPSLA
  119. [92] K.-Y. Cai, L. Zhao, and F. Wang, “A dynamic partitioning approach for GUI testing,”
  120. in COMPSAC (2). IEEE Computer Society, 2006, pp. 223–228.
  121. [93] J. Takahashi and Y. Kakuda, “Extended model-based testing toward high code
  122. coverage rate,” in ECSQ, ser. Lecture Notes in Computer Science, J. Kontio and
  123. [97] J. Bennett. Autoit. Sep. 17, 2009. [Online]. Available: http://www.autoitscript.
  124. [98] P. Kaluski. Win32:guitest. Sep. 17, 2009. [Online]. Available: http://www.
  125. [102] I. Vista Software. vTask. Sep. 17, 2009. [Online]. Available: http://www.
  126. [104] A. Q. Corp. TestComplete. Sep. 17, 2009. [Online]. Available: http:
  127. [105] V. C. Software. Vermont High Test. Sep. 17, 2009. [Online]. Available:
  128. [107] J. Offutt, S. Liu, A. Abdurazik, and P. Ammann, “Generating test data from statebased
  129. [108] J. H. Andrews, L. C. Briand, and Y. Labiche, “Is mutation an appropriate tool for
  130. for assessing and comparing testing coverage criteria,” IEEE Trans. Software
  131. [1] M. Robinson and P. Vorobiev, Swing, Second Edition. Manning Publications,
  132. 2003.
  133. [2] Abbot. June. 1, 2010. [Online]. Available: http://abbot.sourceforge.net
  134. [3] A. M. Memon, I. Banerjee, and A. Nagarajan, “Gui ripping: Reverse engineering
  135. of graphical user interfaces for testing,” in WCRE, A. van Deursen, E. Stroulia, and
  136. [4] C. Nebut, F. Fleurey, Y. L. Traon, and J.-M. J’ez’equel, “Automatic test generation:
  137. A use case driven approach,” IEEE Trans. Software Eng., vol. 32, no. 3, pp. 140–
  138. http://sourceforge.net/projects/crosswordsage/. [Online]. Available:
  139. http://crosswordsage.sourceforge.net
  140. http://sourceforge.net/projects/sudokumania/.
  141. Journal of Software Testing, Verification and Reliability, vol. 15, no. 2, pp.
  142. members: Prof. Rajiv Gupta (University of Arizona), Prof. Adele E. Howe
  143. (Colorado State University), Prof. Lori Pollock (University of Delaware).
  144. [9] (2010) White-box testing. [Online]. Available: http://en.wikipedia.org/wiki/
  145. White-box testing
  146. [10] (2010) Black-box testing. [Online]. Available: http://en.wikipedia.org/wiki/
  147. Black-box testing
  148. Binder1999 (u.a. im Hinblick auf data flow testing strategies).
  149. [12] S. Mcconnell, Code Complete, 2nd ed. Microsoft Press, 2004.
  150. [13] J. Shore and S. Warden, The Art of Agile Development. O’Reilly Media, Inc.,
  151. October 2007. [Online]. Available: http://www.amazon.com/exec/obidos/redirect?
  152. tag=citeulike07-20&path=ASIN/0596527675
  153. [14] L. Koskela, Test Driven: Practical TDD and Acceptance TDD for Java Developers.
  154. [15] K. Beck, Test-Driven Development: by Example. Boston, MA, USA: Addison-
  155. Wesley Longman Publishing Co., Inc., 2002.
  156. software applications,” ACM Trans. Softw. Eng. Methodol., vol. 16, no. 1,
  157. 2007.
  158. tools. New York, NY, USA: ACM Press/Addison-Wesley Publishing Co.,
  159. 1999.
  160. [18] N. Nagappan, E. Maximilien, T. Bhat, and L. Williams, “Realizing quality
  161. [19] M. Alles, D. Crosby, C. Erickson, B. Harleton, M. Marsiglia, G. Pattison,
  162. test-driven development,” agile, vol. 0, pp. 276–288, 2006. [Online]. Available:
  163. [21] K. Beck and E. Gamma. JUnit Framework. June. 1, 2010. [Online]. Available:
  164. http://www.junit.org
  165. [22] NetBeans. Jemmy module. June. 1, 2010. [Online]. Available: http://jemmy.
  166. netbeans.org
  167. in GUI acceptance testing,” in 5th International Conference (XP’04), vol. 3092.
  168. [24] J. Steven, P. Chandra, B. Fleck, and A. Podgurski, “jrapture: A capture/replay tool
  169. for observation-based testing,” in ISSTA, 2000, pp. 158–167.
  170. [25] S. Dutta, “Abbot-a friendly JUnit extension for GUI testing,” Java Developer Journal,
  171. [26] IBM. IBM rational robot. June. 1, 2010. [Online]. Available: http://www-306.ibm.
  172. com/software/awdtools/tester/robot
  173. [27] HP. Hp WinRunner. June. 1, 2010. [Online]. Available: http://www.mercury.com/
  174. us/products/quality-center/functional-testing/winrunner
  175. [28] M. Finsterwalder, “Automating acceptance tests for gui applications in an extreme
  176. 2000.
  177. [29] H. Interactive. Quicktest professional (QTP). Sep. 17, 2009. [Online]. Available:
  178. https://h10078.www1.hp.com/cda/hpms/display/main/hpms content.jsp?zn=
  179. bto&cp=1-11-127-24%5E1352 4000 100
  180. Software Quality Journal, vol. 10, no. 3, pp. 205–224, 2002.
  181. [32] J. Chen and S. Subramaniam, “A GUI environment to manipulate fsms for testing
  182. programs,” in ACM Southeast Regional Conference, S.-M. Yoo and L. H. Etzkorn,
  183. Eds. Huntsville, Alabama, USA: ACM, April 2004, pp. 140–145.
  184. [34] G. Booch, R. A. Maksimchuk, M. W. Engel, B. J. Young, J. Conallen, and
  185. K. A. Houston, Object-Oriented Analysis and Design with Applications (3rd
  186. Edition), 3rd ed. Addison-Wesley Professional, April 2007. [Online]. Available:
  187. http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=
  188. ASIN/020189551X
  189. [35] C. Kaner and J. Bach, “The nature of exploratory testing,” in The Nature of Exploratory
  190. Testing, 2004.
  191. [36] K. L.Wilson, M. Caswell, and V. Aravamudhan. JFCUnit. June. 1, 2010. [Online].
  192. [37] E. Sjosten-Andersson and L. Pareto, “Costs and benefits of structure-aware
  193. the Sixth Conference on Software Engineering Research and Practice in
  194. Sweden (SERPS06), 2006. [Online]. Available: http://www.ituniv.se/program/
  195. sem research/Publications/2006/SP06
  196. the java toolkit,” in APCHI. IEEE Computer Society, 1998, pp. 331–336.
  197. [39] A. Georges, M. Christiaens, M. Ronsse, and K. D. Bosschere, “Jarec: a portable
  198. vol. 34, no. 6, pp. 523–547, 2004.
  199. [40] C. Spannagel. Jacareto. Sep. 17, 2009. [Online]. Available: http://jacareto.
  200. [41] M. Pekar. Pounder. Sep. 17, 2009. [Online]. Available: http://pounder.sourceforge.
  201. net/
  202. marathontesting/
  203. CHI, E. D. Mynatt, D. Schoner, G. Fitzpatrick, S. E. Hudson, K. Edwards, and
  204. T. Rodden, Eds. ACM, 2010, pp. 1535–1544.
  205. [45] A. M. Memon, I. Banerjee, and A. Nagarajan, “GUI Ripping: Reverse engineering
  206. of graphical user interfaces for testing,” in Proceedings of The 10th Working
  207. complete interaction sequences,” in Proc. 11th International Symposium on Software
  208. Reliability Engineering (ISSRE’00). San Jose, CA, USA: IEEE Computer
  209. [49] A.M.Memon,M. E. Pollack, andM. L. Soffa, “A planning-based approach to GUI
  210. testing,” in Proceedings of The 13th International Software/Internet Quality Week,
  211. [50] ——, “Plan generation for GUI testing,” in Proceedings of The Fifth International
  212. Trans. Softw. Eng., vol. 27, no. 2, pp. 144–155, 2001.
  213. 27–64, Jan. 2005.
  214. Testing, Verification and Reliability, 2007.
  215. [54] A. M. Memon and Q. Xie, “Studying the fault-detection effectiveness of GUI test
  216. cases for rapidly evolving software,” IEEE Trans. Softw. Eng., vol. 31, no. 10, pp.
  217. [55] M. Alles, D. Crosby, C. Erickson, B. Harleton, M. Marsiglia, G. Pattison, and
  218. [56] W. Cunningham. Fit: framework for integrated test. June. 1, 2010. [Online].
  219. 1, 2010. [Online]. Available: http://code.google.com/p/robotframework
  220. software engineering conference held jointly with 11th ACM SIGSOFT international
  221. symposium on Foundations of software engineering (ESEC/FSE’03).
  222. 398–409.
  223. regression testing,” ACM Trans. on Softw. Eng. and Method., 2008.
  224. [62] J. L. Silva, J. C. Campos, and A. Paiva, “Model-based user interface testing with
  225. spec explorer and concurtasktrees,” in 2nd International Workshop on Formal
  226. Computer Science, P. Curzon and A. Cerone, Eds., vol. 208, 2008, pp. 77–93.
  227. [63] A. Paiva, J. C. P. Faria, and R. F. A. M. Vidal, “Towards the integration of visual
  228. no. 2, pp. 99–111, 2007.
  229. [64] W.-K. Chen, Z.-W. Shen, and T.-H. Tsai, “Integration of specification-based and
  230. cr-based approaches for GUI testing,” Journal of Information Science and Engineering,
  231. vol. 24, no. 5, pp. 1293–1307, 2008.
  232. [65] W.-K. Chen. GTT. June. 1, 2010. [Online]. Available: http://gtt.sourceforge.net
  233. of the 31st International Conference on Technology of Object-Oriented Language
  234. and Systems. Washington, DC, USA: IEEE Computer Society, 1999, p.
  235. 156.
  236. [67] W.-K. Chen, Z.-W. Shen, and C.-M. Chang, “GUI test script organization with
  237. component abstraction,” in IEEE International Conference on Secure System Integration
  238. and Reliability Improvement (SSIRI 2008), 2008, pp. 128–134.
  239. [68] C. Larman, Applying UML and Patterns, 3rd Ed. Prentice Hall PTR, 2004.
  240. regression testing,” in Proc. the 2005 IEEE/WIC/ACM International Conference
  241. on Intelligent Agent Technology, A. Skowron, J.-P. A. Barth`es, L. C. Jain, R. Sun,
  242. P. Morizet-Mahoudeaux, J. Liu, and N. Zhong, Eds. Compiegne, France,: IEEE
  243. GUI test case maintenance,” in TESTBEDS’09: Proceedings of the First International
  244. [71] M. Fowler, K. Beck, J. Brant, W. Opdyke, and D. Roberts, Refactoring:
  245. tag=citeulike07-20&path=ASIN/0201485672
  246. [74] A. Memon. Terpoffice. June. 1, 2010. [Online]. Available: http://sourceforge.net/
  247. projects/terpoffice/
  248. [75] A. M. Memon, I. Banerjee, and A. Nagarajan, “What test oracle should I use for
  249. effective GUI testing?” in Proceedings of the IEEE International Conference on
  250. Automated Software Engineering. IEEE Computer Society, Oct. 2003, pp. 164–
  251. 173.
  252. [77] A. M. Memon and Q. Xie, “Empirical evaluation of the fault-detection effectiveness
  253. of smoke regression test cases for GUI-based software,” in ICSM ’04: Proceedings
  254. software applications,” in ICSM ’05: Proceedings of the 21st IEEE Interna-
  255. [79] ——, “Model-based testing of community-driven open-source GUI applications,”
  256. 154.
  257. vol. 16, no. 1, p. 4, 2007.
  258. Automation (DoSTA 2007); co-located with The 6th joint meeting of the European
  259. Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations
  260. of Software Engineering, Dubrovnik, Croatia, Sep. 2007.
  261. test suite reduction,” in ICSM ’07: Proceedings of the 21st IEEE International
  262. IEEE Trans. Softw. Eng., 2008.
  263. in ASE ’07: Proceedings of the 22nd IEEE international conference on Automated
  264. GUI run-time state feedback,” IEEE Transactions on Software Engineering,
  265. vol. NN, no. N, 2010.
  266. detection in gui testing,” in ICST ’08: Proceedings of the First international conference
  267. on Software Testing, Verification, and Validation. Washington, DC, USA:
  268. [90] A. Paiva, N. Tillmann, J. C. P. Faria, and R. F. A. M. Vidal, “Modeling and testing
  269. hierarchical guis,” in Abstract State Machines, 2005, pp. 329–344.
  270. [91] M. J. Harrold, J. A. Jones, T. Li, D. Liang, A. Orso, M. Pennings, S. Sinha, S. A.
  271. ’01: Proceedings of the 16th ACM SIGPLAN conference on Object-oriented programming,
  272. systems, languages, and applications. New York, NY, USA: ACM,
  273. 2001, pp. 312–326.
  274. R. Conradi, Eds., vol. 2349. Springer, 2002, pp. 310–320.
  275. [94] A. Balkan. Autotestflash. Sep. 17, 2009. [Online]. Available: http://osflash.org/
  276. autotestflash/
  277. [95] J. Layat. jDiffChaser. Sep. 17, 2009. [Online]. Available: http://jdiffchaser.
  278. sourceforge.net/
  279. [96] V. Prahlad. frankenstein. Sep. 17, 2009. [Online]. Available: http://frankenstein.
  280. openqa.org/
  281. com/autoit3/
  282. piotrkaluski.com/
  283. [99] Q. F. S. GmbH. QF-Test. Sep. 17, 2009. [Online]. Available: http://www.qfs.de/en
  284. [100] Borland. SilkTest. Sep. 17, 2009. [Online]. Available: http://www.borland.com/
  285. us/products/silk/silktest
  286. [101] B. GmbH. GUIdancer. Sep. 17, 2009. [Online]. Available: http://www.bredex.de/
  287. en/guidancer
  288. vtaskstudio.com
  289. [103] M. F. Ltd. TestPartner. Sep. 17, 2009. [Online]. Available: http://www.microfocus.
  290. com/products/TestPartner
  291. //www.automatedqa.com/products/testcomplete
  292. http://www.vtsoft.com/vcsproducts
  293. [106] I. Instantiations. WindowTester. Sep. 17, 2009. [Online]. Available: http:
  294. //www.windowtester.com
  295. specifications,” The Journal of Software Testing, Verification and Reliability,
  296. vol. 13, pp. 25–53, 2003.
  297. testing experiments?” in ICSE, G.-C. Roman, W. G. Griswold, and B. Nuseibeh,
  298. Eds. ACM, 2005, pp. 402–411.
  299. [109] J. H. Andrews, L. C. Briand, Y. Labiche, and A. S. Namin, “Using mutation analysis
  300. Eng., vol. 32, no. 8, pp. 608–624, 2006.
Times Cited
  1. 李玠錞(2013)。Web應用程式之雲端效能測試系統。臺北科技大學資訊工程系研究所學位論文。2013。1-89。 
  2. 陳怡靜(2012)。一個支援元件抽象化的Android GUI測試工具。臺北科技大學資訊工程系研究所學位論文。2012。1-61。