應用視覺化程式語言與人工智慧實現創新人機互動介面

Using Visual Programming Language and Artificial Intelligence to Develop an Innovative Human Machine Interaction Interface

黃立武 ； 陳響亮 ； 李宜靜 ； 黃兆平 ； 董塘筠 ； 陳品修

《機械工業雜誌》 472期 (2022/07) Pp. 73-82

摘要

機台運動控制程式之編輯往往需要艱深的專業知識，且控制程式之程式語言需要專業背景才能閱讀。為了降低編輯運動控制程式之專業門檻，本文針對PC-Based控制器之六軸機械手臂與生產線周邊設備之運動控制進行研究，設計出一套視覺化程式語言（Visual Programming Language, VPL），並透過此VPL實作出一套人機互動介面，使用者可透過拖曳、連結不同功能方塊之方式，實現生產線控制流程之建立與調整。為更進一步降低系統之操作門檻，本系統導入深度學習、電腦視覺、語音識別、眼動追蹤（eye tracking）等技術，開發基於手勢、語音、視線等特徵之人機互動功能，實現創新人機互動VPL編程機制。藉由上述機制之輔助，使用者可透過較為直覺的方式完成控制程式之編輯。

關鍵字

視覺化程式語言 ； 運動控制 ； 人工智慧 ； 語音辨識 ； 手勢辨識

並列摘要

The editing of machine motion control programs often requires professional knowledge, and the programming language of the control program is difficult to read. To reduce the professional threshold for editing motion control programs, this article focuses on the design and development of a human machine interface by using the visual programming language (VPL) which can be applied for the performance testing of the performance of the developed system. The testing examples include the motion control of the six-axis robotic arms with PC-Based controllers and the control of production line peripherals. Through the VPL, a human-computer interaction system is developed, which allows users to establish and adjust the production line control flow by dragging and linking different functional icons. To further reduce the operating threshold of the system, deep learning, computer vision, voice recognition, eye tracking and other technologies are also introduced to develop the human-computer interaction mechanism based on gesture, voice, and vision features. With the help of the above mentioned mechanism, users can realize the editing of production line control programs in a more intuitive way.

並列關鍵字

Visual programming language ； Motion control ； Artificial intelligence ； Voice recognition ； Gesture recognition

參考文獻

S. Alexandrova, Z. Tatlock, and M. Cakmak, “RoboFlow: A flow-based visual programming language for mobile manipulation tasks,ˮ in 2015 IEEE International Conference on Robotics and Automation (ICRA), 5537-5544, 2015.

M. Erwig, K. Smeltzer, and X. Wang, “What is a visual language?,ˮ Journal of Visual Languages & Computing 38(6), 9-17, 2016.

Ilias El Makrini, et al. “Working with walt: how a cobot was developed and inserted on an auto assembly line,ˮ IEEE Robotics & Automation Magazine 25(2), 51-58, 2018.

V. Këpuska and G. Bohouta, “Comparing speech recognition systems (Microsoft API, Google API and CMU Sphinx),ˮ Int. J. Eng. Res. Appl 7(3), 20-24, 2017.

Microsoft Docs, “ 評估和改善自訂語音精確度 ,ˮ 2022. [Online]. Available: https://docs.microsoft.com/zh-tw/azure/cognitive-services/speech-service/how-to-custom-speech-evaluate-data