電腦視覺為導向的機器人原型設計：能以滑鼠使用電腦的機器人CUBot

Computer-Vision-Oriented Robot Prototype Design: a Mouse-Controlling-Based Computer-Using Robot (CUBot)

林坤立(Kun-Li Lin)

指導教授 ： 陳永盛

元智大學/電機通訊學院/電機工程學系/碩士(2012年)

摘要

電腦網路資訊量的累積規模時至今日已經相當龐大，透過認知電腦螢幕與操作電腦所能獲得的「電腦世界中的資訊」和在我們所處的「真實世界裡的資訊」一樣重要。偌大的資料量讓我們在許多時候需要自動化的機制幫助我們分析與過濾電腦世界中的資訊，電腦視覺領域便可運用在認知顯示在電腦螢幕上的訊息。輔以參考智慧型機器人直至目前在於各種不同型態的機器人架構之發展與應用，這樣的自動化機制需求可以透過一個類似人類行為的操作系統來幫助我們篩選及過濾資料。

有鑑於前述，我們考慮在以電腦視覺的知識領域為研究前提之下，結合認知螢幕上所顯示的物件和以機械機構控制滑鼠移動及點擊目標物件，使機器人具有像「人類以滑鼠操作電腦」的能力。本論文闡述以此為目標的機器人原型設計，並稱呼其為「使用電腦的機器人」(Computer-Using Robot或簡稱CUBot)。從觀察自身和一般人類操作電腦時候的使用經驗，我們了解到，操作電腦的過程一般來說可分為觀看電腦螢幕、識別顯示內容及操控滑鼠點選物件等三個步驟，從擬人觀點出發的CUBot也該是如此。它本身不能夠具備任何與被操控電腦之間有線或無線的資料傳輸能力，因此完全以其視覺認知自己正在使用的電腦螢幕所顯示之訊息，並且用其機器手機構控制滑鼠進而操作該電腦。從視覺能力認知螢幕上的物件、經由手控制滑鼠、再藉由視覺能力認知螢幕上的游標與目標物件作為回饋的過程稱為「眼手協調」，在此研究中是一項CUBot能夠成功運作的核心技術。CUBot對於其所認知到的物件必須建立並更新「心智地圖」，以供後續操作電腦的過程使用。然而時下電腦作業系統介面設計較為複雜，並非目前能夠完全以電腦視覺認知分析的能力範圍。所以在此原型設計中我們以簡化的訊息及操作環境界面提供CUBot操作，並給它予二項電腦操作任務以驗證其各項系統功能的正確性及穩定性。

關鍵字

機器人 ； 電腦視覺 ； 眼手協調 ； 人機互動 ； 使用電腦

並列摘要

The quantity accumulation scale of computer network information till these days is immense. Information that perceived in computer world through a computer screen is as important as that perceived in our physical world. Automation for analyzing and sifting enormous pieces of information in computer world is indeed necessary in many situations. Computer vision techniques are proper to perceive information on a computer screen. Besides, concerning development and application of various modern intelligent robot architectures, such kind of automation requirement can be achieved by a human-behavior-like operating autonomous system.

Therefore, taking the premise of computer vision research topic and combining perception of objects that showing on screen with a mouse controlling device, this thesis proposed a robot prototype that possesses ability to operate a computer with a mouse like a human. This robot is named ``Computer-Using Robot" or briefly ``CUBot.' By observing the experience of human being, computer operating can be generally distributed into three phases, i.e., watching the screen, recognizing the objects, and controlling the mouse to click the target. Thus CUBot perceives information through its vision and controls the mouse by its robotic hand without any physical data communication with the operated computer. The path which starts from perceiving objects through vision, via controlling mouse by hand, and use the positions of the cursor and the target as control feedback is called ``eye-hand coordination." It is the key point in this research. And also, CUBot builds its mind map about what it has recognized and where it is and continuously updating this information. However the user interfaces of common operating systems are too complex that is over the topic in this study. Therefore we designed a simplified user interface and provide two missions for examining the abilities and stability of CUBot.

並列關鍵字

Computer Using ； Robot ； Computer Vision ； Eye-Hand Coordination ； Human-Computer Interaction

參考文獻

[2] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and K. Fujimura, “The intelligent asimo: system overview and integration,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, 2002, pp. 2478–2483 vol.3.

[5] L. Geppert, “Qrio, the robot that could,” IEEE Spectrum, vol. 41, no. 5, pp. 34–37, 2004.

[7] S.-H. Liao, P.-H. Chu, and P.-Y. Hsiao, “Data mining techniques and applications – a decade review from 2000 to 2011,” Expert Systems with Applications, vol. 39, no. 12, pp. 11 303–11 311, 2012.

[8] H.-D. Lin and C.-H. Chien, “Automated detection of color non-uniformity defects in tft-lcd,” in Proceedings of the International Joint Conference on Neural Networks, 2006, pp. 1405–1412.

[9] L.-F. Chen, C.-T. Su, and M.-H. Chen, “A neural-network approach for defect recognition in tft-lcd photolithography process,” IEEE Transactions on Electronics Packaging Manufacturing, vol. 32, no. 1, pp. 1–8, 2009.