REINFORCEMENT LEARNING-BASED IBVS STRUCTURE FOR CONTROL OF POINT-TO-POINT MOTION OF ROBOT MANIPULATORS

Ting-Yu Ye ； Ming-Yang Cheng ； Ya-Ling Chen ； Pin-Hsuan Huang

《Journal of Marine Science and Technology》 28卷5期 (2020/10) Pp. 367-375

https://doi.org/10.6119/JMST.202010_28(5).0006

摘要

In order to facilitate the use of robot manipulators equipped with visual servoing systems so as to enhance the flexibility/functionality of the automatic production line in industry, this paper focuses on applying the reinforcement learning paradigm to the Image-Based Visual Servoing (IBVS) structure. By responding to changes in the environment, the proposed reinforcement learning-based IBVS structure can select the best policy for controlling the position/ pose of the robot manipulator so as to converge the error between the image feature and the desired image feature. This paper exploits Q-learning and a deep Q-network to implement a reinforcement learning-based IBVS structure, respectively. In this paper, the states used in reinforcement learning are the coordinates of the image feature point (or grid points) on the image plane, while the action is the increment in the gain constant of the IBVS structure. Three different IBVS structures-conventional IBVS, Q-learning-based IBVS and deep Q-network-based IBVS-are implemented on a 2-DOF planar robot manipulator to perform a point-to-point motion. Experimental results indicate that the proposed deep Q-network-based IBVS structure has the best performance, while the conventional IBVS yields the worst.

關鍵字

reinforcement learning ； visual servoing ； robotic system ； Q-learning ； deep Q-network