手勢調控之太極拳多媒體撥放學習系統

Gesture-mediated Multimedia Player for Tai Chi Chuan Instruction

吳儒涵(Ru-Han Wu)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2014年)

https://doi.org/10.6342/NTU.2014.01485

摘要

近年來，除了以傳統的上課學習方式，多媒體教材在各種運動或舞
蹈學習上被廣為使用，人們利用多媒體容易取得的特性，在家中可以
隨時隨地自主練習。然而，多媒體教材雖然方便，卻無法達到像真人
教練引導時的互動性，學習者往往要手動調整多媒體的播放進度來反
覆練習，因為多媒體的播放速度往往是單一且沒有彈性的。另一方面，光靠影片的示範也很難確保所習得的動作之細節及正確性。
為了解決上述問題，我們提出了一套利用姿勢調控多媒體播放進度
之系統：「觀自在」播放器。利用搭載三軸加速器之智慧手錶來感測手部動作資訊辨識動作，依據所計算出來的姿勢速度與進度，來調整學習影片播放之速度，以達到影片隨使用者練習狀態變化而產生對應調整之效果。
在實驗及使用者試用回饋中，我們請使用者依據不同的動作速度測
試動作完整度之辨識，並且得到相當高之正確率。不只如此，在受測
者使用即時影片回饋系統時，使用者認為影片正確隨姿勢調控速度的
時間至少佔七成以上的練習時間，我們的實驗結果顯示我們提出的方
法，能夠達到利用姿勢調控影片之效果。

關鍵字

多媒體互動同步 ； 基於三軸加速器姿勢辨識 ； 自動化姿勢分段 ； 太極拳學習系統

並列摘要

In addition to the traditional way of learning, multimedia learning materi-als are widely used in training of various kinds of exercises and dancing. With the accessibility of these materials, people can do the training any time and any where. Despite the fact that learning by using multimedia is convenient (such as watching videos), the interaction with teacher in training process is hard to be simulated. Learners usually need to manually adjust the playback progress and repeat it again and again since the monotonous and lack of flex-ibility of video. On the other hand, it is difficult to confirm the correctness and details of gestures the user learned.
In order to solve above problems, we proposed the gesture-mediated mul-timedia player application, ”Follow-Me”, to learning Tai Chi Chuan, which built up with accelerometer-enabled smart watches and commercial mobile devices. It provided an interaction between user and multimedia according to progress of user’s hand gesture. We applied an incomplete time series match-ing method to get the progress ,completeness of gestures and fulfil automatic segmentation. The video playback design is based on the automatic segmen-tation to reach the goal of mediating video content with the alteration of gestures.
In experiments and user study, we asked users to perform gestures in var-ious levels of speed to evaluate the relative error time and percentage error of progress prediction. The result demonstrated low percentage error was achieved. Furthermore, users gave us some positive feedback toward our real-time video feedback system. A percentage of 71% was reported when we questioned participants about how much time they felt the adjustment of video according to gestures was correct. Our results show the effectiveness of the gesture-mediated method.

並列關鍵字

multimedia synchronization ； accelerometer-based gesture recognition ； gesture segmentation ； Tai Chi Chuan learning

參考文獻

[3] Paolo Tormene, Toni Giorgino, Silvana Quaglini, and Mario Stefanelli. Matching incomplete time series with dynamic time warping: an algorithm and an application

to post-stroke rehabilitation. Artificial Intelligence in Medicine, 45(1):11–34, 2009.

[4] Liqun Deng, Howard Leung, Naijie Gu, and Yang Yang. Real-time mocap dance recognition for an interactive dancing game. Computer Animation and Virtual Worlds, 22(2-3):229–237, 2011.

[5] Otniel Portillo-Rodriguez, Oscar O Sandoval-Gonzalez, Emanuele Ruffaldi, Rosario Leonardi, Carlo Alberto Avizzano, and Massimo Bergamasco. Real-time gesture recognition, evaluation and feed-forward correction of a multimodal tai-chi platform. In Haptic and Audio Interaction Design, pages 30–39. Springer, 2008.

[7] Xu Zhang, Xiang Chen, Yun Li, Vuokko Lantz, Kongqiao Wang, and Jihai Yang. A framework for hand gesture recognition based on accelerometer and emg sensors. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 41(6):1064–1076, 2011.