基於影像視覺伺服控制之無人飛行器行人監視系統

Person Surveillance System by Unmanned Aerial Vehicles Using Image-Based Visual Servoing Control

吳睿哲(Jui-Che Wu)

指導教授 ： 連豊力

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201903642

摘要

本篇論文提出了一個基於單眼視覺來進行伺服控制的無人飛行器行人監視系統。此系統可以跟隨任何有紋理的物體,像是衣服上的圖案。所以此系統也可以被用來監視較易受傷害的人，像是小孩以及老人等。當他們發生危險狀況時，此監視系統便會發出警報來降低損傷。
這個系統的主要架構可以分成兩個部分：移動物體追蹤與自動跟隨目標物。物體追蹤演算法可以提供物體在影像中的位置，於是影像伺服控制器便可以根據此資訊來進行自動跟隨目標物。
在物體追蹤的部分，主要是使用基於特徵點方法的追蹤器來找出物體在二維影像中的位置，且為了達到實時的效果，本篇論文使用了一個較快的特徵點偵測方法。為了在目標物上能取得足夠的特徵點，光流演算法被用來追蹤特徵點，並且跟之前配對在目標物上的特徵點結合。結合後的特徵點使用了特徵點之間的幾何關係來預測物體的旋轉以及尺度，並且使用一個異常值移除演算法來決定目標物的中心點。為了減少影像模糊以及無人機突然震動造成的影響，本篇論文使用了卡爾曼濾波器來預測目標物的邊界框大小。
在目標物跟隨的部分，追蹤器提供的目標物資訊可以被用來當作姿態控制器的輸入量測值。首先，追蹤器預測的目標物邊界框大小可以用來控制無人機與目標物之間的相對距離，邊界框的中心點資訊可以用來維持目標物於前視相機影像的正中心。最後，為了解決單眼相機無法取得絕對尺度的問題，針孔相機模型結合了從超聲波取得的高度資訊來預測無人機與目標物之間的相對距離。於是無人機便可以根據使用者給的目標物與無人機之間的預期距離，來進行絕對距離的控制，解決了基於影像視覺伺服控制無法對絕對距離進行控制的問題。
本系統同時在模擬與真實環境下進行評估。模擬環境是在gazebo模擬器中進行操作，一個脆弱的人可能產生的移動行為像是走路與轉彎等…都會被測試。為了測試此系統的跟隨性能與穩健性，與在模擬中一樣的移動行為模式也會在室內場景中進行實驗。

關鍵字

目標跟隨 ； 物體追蹤 ； 視覺伺服 ； 四旋翼 ； 姿態控制 ； 無人飛行器

並列摘要

In this thesis, a UAV person surveillance system mainly using on-board monocular vision for image-based visual servoing control is proposed. The system can follow any textured object, such as pattern on the clothe. Therefore, it can be utilized to oversee the fragile people, such as children and old people, etc. Whenever these people are under risk, the person surveillance system will send an alarm to reduce the damage.
The system is composed of two main processes: moving object tracking and autonomous target following. Moving object tracking algorithm can locate the target position on the image. Thus, target information can be given to the image-based visual servoing controller for autonomous target following.
For object tracking, a keypoints-based tracker is designed to locate the object in the 2-D image space. To achieve real time tracking performance, a faster keypoints detection method is utilized. To capture sufficient keypoints on the target, an optical flow algorithm is also utilized for keypoints tracking and fused with the keypoints matched on the target. Fused keypoints are mainly utilized for scale and rotation estimation using geometric relationship between keypoints. Then, the target center can be estimated. To further reduce the influence of image blur and sudden movement of the drone, a Kalman filter is utilized to predict the size of the target bounding box.
For autonomous target following, target information provided by the tracker is utilized as measurement for the pose controller. First, the size of the bounding box predicted by the tracker can be utilized to control the relative distance between the drone and the target, and the center of the bounding box can be utilized to maintain the target at the center field of view of the on-board camera. To solve the problem of absence of the absolute scale in pure monocular system, a pinhole camera model is combined with the height information from on-board ultrasound to estimate the distance between drone and the target. Thus, the drone can be controlled to maintain a desired distance relative with the target given by user, solving the problem that IBVS can’t control absolute distance.
The proposed system is evaluated both in simulation and experiment. Simulation environment is operated under the gazebo simulator, and different action modes a fragile person may conduct such as walking and turning, and so on are tested. To further evaluate the performance of the following result and robustness, same cases are also tested in an indoor environment scenario.

並列關鍵字

Target Following ； Object Tracking ； Visual Servoing ； Quadrotor ； Pose Control ； Unmanned Aerial Vehicle

參考文獻

Mykhaylo Andriluka, Paul Schnitzspan, Johannes Meyer, Stefan Kohlbrecher, Karen Petersen, Oskar von Stryk, Stefan Roth, and Bernt Schiele, “Vision Based Victim Detection from Unmanned Aerial Vehicles,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, pp. 1740-1747, Oct. 18-22, 2010.

T. Ozaslan, S. Shen, Y. Mulgaonkar, N. Michael, and V. Kumar, “Inspection of Penstocks and Featureless Tunnel-Like Environments using Micro UAVs,” in Proceedings of Field and Service Robot Conference, Brisbane, Australia, pp. 123-136, Jan. 2013.

Fredrik Heintz, Piotr Rudol, and Patrick Doherty, “From Images to Traffic Behavior A UAV Tracking and Monitoring Application,” in Proceedings of 10th International Conference on Information Fusion, Quebec, Canada, Jul. 9-12, 2007.

Yang Lyu, Quan Pan, Chunhui Zhao, Yizhai Zhang, and Jinwen Hu, ‘‘Vision-Based UAV Collision Avoidance with 2d Dynamic Safety Envelope,” IEEE Aerospace and Electronic Systems Magazine, vol. 31, no. 7, pp. 16-26, Jul. 2016.

Paul Pao-Yen Wu, Duncan Campbell, and Torsten Merz, “On-Board Multi-Objective Mission Planning for Unmanned Aerial Vehicles,” in Proceedings of IEEE Aerospace Conference, Big Sky, USA, pp. 1-10, Mar. 7-17, 2009.