基於深度強化學習之移動大型重物

Moving Large Size and Heavy Object with Deep Reinforcement Learning

許哲菡(Hanjaya Mandala)

指導教授 ： 包傑奇

國立臺灣師範大學/科技與工程學院/電機工程學系/碩士(2020年)

https://doi.org/10.6345/NTNU202000829

摘要

none

關鍵字

人形機器人 ； 深度強化學習 ； 拖動物件 ； 深度學習

並列摘要

Humanoid robots are designed and expected to work alongside a human. In our daily life, Moving Large Size and Heavy Objects (MLHO) can be considered as a problem that is a common activity and dangerous to humans. In this thesis, we propose a novel hierarchical learning-based algorithm, which we use dragging to transport an object on an adult-sized humanoid robot. The proposed method proves robustness on a THORMANG-Wolf adult-sized humanoid robot, that manages to drag a massive object with a mass of double of its weight (84.6 kg) for 2 meters. Therefore, the algorithms consist of three hierarchical deep learning-based algorithms to solve the MLHO problem and distributed in terms of robot vision and behavior control. Based on this insight, in the robot vision control, first, we propose deep learning algorithms to 3D object classification and surface detection.

For 3D object classification, we propose a Three-layers Convolution Volumetric Network (TCVN). Input data of the TCVN model used a voxel grid representation from point clouds data acquired from the robot’s LiDAR scanner. On the other hand, for surface detection, we propose a lightweight real-time instance segmentation called Tiny-YOLACT (You Only Look at Coefficients) to segment the floor from the robot’s camera. Tiny-YOLACT model is adopted from the YOLACT model and utilized ResNet-18 model as the backbone network. Furthermore, for robot behavior control, as the main part of this thesis we address solving MLHO problem by an adult-sized humanoid robot using the deep reinforcement learning algorithm for the first time. At this part, we proposed a Deep Q-Learning algorithm to train a deep model for control policy in offsetting the Centre of Body (CoB) of the robot when dragging different objects named (DQL-COB). For this purpose, the offset CoB is implemented to keep tracking with the robot’s center of mass. As a result, the robot can keep balance with maintaining the ZMP in the support polygon. DQL-COB algorithm was first trained on the ROS Gazebo simulator to avoid costly experiments in terms of time and real environment constraints, then it was adopted with a real robot on three different types of surfaces.

To evaluate the stability of the THORMANG-Wolf robot with the proposed methods, we evaluated two types of experiments on three types of surfaces with eight different objects. In these experiments, in one scenario we use IMU along with foot Pressure (F/T) sensor, in the second scenario we just use IMU data as learning algorithm input. In the experiments, the success rates of applying the DQL-COB algorithm on the real robot are 92.91% with using the F/T sensor and 83.75% without using F/T sensors. Moreover, the TCVN model on 3D object classifications achieved a 90% accuracy in real-time. Correspondingly, the Tiny-YOLACT model achieved a 34.16 mAP on validation data with an average of 29.56 fps on a single NVIDIA GTX-1060 GPU.

並列關鍵字

humanoid robot ； deep reinforcement learning ； dragging object ； deep learning

參考文獻

Bibliographies

[1] K. Tanie, "Humanoid robot and its application possibility," in Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI2003., 2003, pp. 213-214.

[2] A. Choudhury, H. Li, C. Greene, and S. Perumalla, "Humanoid Robot-Application and Influence," arXiv preprint arXiv:1812.06090, 2018.

[3] S. Kajita, H. Hirukawa, K. Harada, and K. Yokoi, Introduction to humanoid robotics. Springer, 2014.

[4] T. Takubo, K. Inoue, and T. Arai, "Pushing an Object Considering the Hand Reflect Forces by Humanoid Robot in Dynamic Walking," in Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, Spain, 2005, pp. 1706-1711.