無人機環境感知與階層式手勢控制於人機協作任務應用

UAV Environment Perception and Hierarchical Gesture Control in Human-Robot Collaboration Applications

邱庭毅(Chiu Ting-Yi)

指導教授 ： 劉吉軒

國立政治大學/理學院/資訊科學系/碩士(2021年)

https://doi.org/10.6814/NCCU202101454

摘要

無人機應用由早期軍事任務的範疇，逐漸拓展到現今民生服務的領域，鑑於無人機具有易於專案部署、低保養成本、高機動性的特點，因此廣受各領域人士的歡迎，但是實體搖桿控制對操作人員並不友善，需要專業培訓以熟練所有操作技能，是屬於高學習門檻的人機互動方式。除此之外，無人機的自動化控制是難以有效地被應用於現實任務，主要原因是現實環境往往是非結構化的，對於自動化控制而言，可能存在未被定義或者未能被準確定義的例外狀況。
為了建立自然直觀的無人機操控方式，本研究提出無人機環境感知與階層式手勢控制的人機協作方法，採用階層式框架以手勢進行半自動化飛行控制的調控，是基於Mediapipe的手部追蹤與定位技術，提出由幾何向量計算手指開合狀態與指向方位作為手勢辨識的方法；此外也基於ORB-SLAM2的即時定位與地圖構建與Detectron2的實例分割技術，提出可以根據自訂義資料集進行特定目標的感知，透過圖片的實例分割進行3D物體的體積與座標估計。最後，經由數名受試者的實驗資料結果分析，得以證實本研究提出的控制方法更優於實體的搖桿控制，可以更快更高效率地完成任務，而且在環繞飛行時目標的檢視畫面也更為平穩。

關鍵字

無人機 ； 手勢辨識 ； 人機協作 ； 基於視覺的即時定位與地圖構建 ； 實例分割

並列摘要

The application of UAVs has gradually shifted from military missions to civilian services. UAVs are popular in various fields due to their convenient deployment, low maintenance cost, and high maneuverability. However, the joystick control is not friendly to the operator, because the joystick is a human-computer interaction with the high learning threshold, and requires professional training to be proficient in skill. In addition, since real-world conditions are usually unstructured, and there may be undefined or inaccurately defined exceptions, it is difficult for the automated control of UAVs to be applied to real-world tasks.
In order to create an intuitive drone control method, we propose a human-robot collaboration method of UAV environment perception and hierarchical gesture control, using a hierarchical framework to adjust flight procedures through gestures to achieve semi-automatic control. In hierarchical gesture control, we adopt flexion state of fingers and pointing direction of hand as the features of gesture recognition, based on the hand tracking technology of Mediapipe. Furthermore, we provide customizable target perception based on combining ORB-SLAM2 and Detectron2, which can estimate the volume and coordinates of 3D objects by instance segmentation. Finally, through the analysis of the experimental results of the participants, given that our proposed control method can complete the task more efficiently and provide a more stable image during surround inspection, we can confirm that our proposed control method is better than physical joystick control.

並列關鍵字

UAV ； Gesture Recognition ； Human-Robot Collaboration ； v-SLAM ； Instance Segmentation

參考文獻

[1] R. Austin, Unmanned Aircraft Systems: UAVS Design, Development and Deployment, John Wiley & Sons, 2011.

[2] S. G. Gupta, M. M. Ghonge and P. M. Jawandhiya, "Review of Unmanned Aircraft System (UAS)," International Journal of Advanced Research in Computer Engineering & Technology, vol. 2, no. 4, pp. 1646-1658, 2013.

[3] A. P. Cracknell, "UAVs: regulations and law enforcement," International Journal of Remote Sensing, vol. 38, no. 8-10, pp. 3054-3067, 2017.

[4] PwC, "Global market for commercial applications of drone technology valued at over $127bn," 2016. [Online]. Available: https://pwc.blogs.com/press_room/2016/05/global- market-for-commercial-applications-of-drone-technology-valued-at-over-127bn.html. [Accessed Nov 2020].

[5] N. Smolyanskiy and M. Gonzalez-Franco, "Stereoscopic First Person View System for Drone Navigation," Frontiers in Robotics and AI, vol. 4, no. 11, 2017.