Implementation of a Human-Robot Collaboration System Based On Smart Hand Gesture and Speech Recognition

基於智慧手勢和語音辨識實現之人機協作系統

陳響亮(Shang-Liang Chen) ； 黃立武(Li-Wu Huang) ； 黃仲麒(Chung-Chi Huang) ； 李峰吉(Feng-Chi Lee) ； 黃河銓(Ho-Chuan Huang) ； 陳建佑(Chien-Yu Chen)

《中國機械工程學刊》 41卷6期 (2020/12) Pp. 755-762

摘要

With the development of artificial intelligence technology, the robot arm will be applied in more fields, undertake more work, and become an important assistant of humans. Especially in the manufacturing industry, if a perfect human-robot cooperation (HRC) model can be developed, in which the engineer is responsible for complex operations and monitoring the production process and the robot arm is responsible for highly repetitive and tiring work, the flexibility of the manufacturing process can be improved, the risk of personnel injury can be reduced, and the production line productivity can be improved. To realize the human-robot collaboration system that can be applied in various fields, the technologies of robot arm control, human-robot interaction (HRI), Radio Frequency Identification, machine-to-machine communication, and automatic speech recognition (ASR), etc. are combined in this research. Besides, the HRC in intelligent manufacturing is taken as the demonstration result of this research. In this application example, the human-robot interaction function is realized through smart hand gesture and speech recognition technology, which enables the user to communicate commands to the robot arm most intuitively and easily. The robot arm can execute specific actions, provide processing prompts, or read corresponding motion control command macros according to different commands, to realize various processing contents required by users.

關鍵字

HRC ； deep learning ； hand gesture recognition ； RFID ； ASR

並列摘要

隨著人工智慧技術的發展，機械手臂將在更多樣的應用領域中承擔不同的工作，成為人類重要的助手。尤其在製造業中，若能開發出完善的人機協作模式，由工程師負責複雜的操作並監控生產過程，而機械手臂則負責重複性高的勞力工作，將可以提高製造過程的彈性、減少工作人員受傷的風險，並且提高產線的生產率。為了實現能應用在各個領域中的人機協作系統，本研究結合了機械手臂控制、人機互動、無線射頻辨識、機器對機器通訊和自動語音識別等多項技術，並以智慧製造中的人機協作作為研究成果的展現。在此應用範例中，人機互動功能通過智慧手勢和語音辨識實現，讓使用者能以最為直觀、簡易的方式將指令傳達給機械手臂。根據不同的指令內容，機械手臂可以執行特定的動作、提供加工提示資訊或讀取指定的運動控制巨集，以實現加工過程中使用者的各項需求。