基於追踨三維人體關節結構之動作辨識方法及其應用

A Novel Human Action Recognition Approach based on 3D Tracked Body Joints and its Application

游宗樺(Zong-Hua You)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2011年)

https://doi.org/10.6342/NTU.2011.02183

摘要

本論文提出一個使用已追踨到之人體關節三維位置的行為辨識的新演算法。本研究使用行為機率圖來模擬動作的動態，利用人體各關節位置的分布當作行為機率圖之訓練特徵藉以描述一個行為內所含有的姿勢。也就是在動作機率圖中的節點。更進一步的，我們也提出了一個階層式行為辨識架構，用來加速辨識的速度以及增進動作的辨識準確率。在我們的階層式架構中，人體可被分成四個部位(也就是左上肢、右上肢、左下肢和右下肢)。我們的想法是提出一個動態指標，用來評估每一個部位移動程度。因此所有的動作可以透過部位移動程度粗略的分成若干類。由實驗結果可得知，動作辨識正確率在未使用階層式架構時約為80%，而使用階層式系統可以超過90%，而在速度方面使用階層式系統可以增進20%的效能。
此外，本研究應用於人偶以及舞者互動之表演，透過深度攝影機以及現有追踨系統取得人體關節的資訊，並且將這些資訊透過我們所提出的系統對動作分析，達到互動的效果。

關鍵字

動作辨識 ； 動作機率圖 ； 形狀特徵 ； 顯著姿勢 ； 動態指標

並列摘要

This study presents an innovative action recognition approach using tracked human body joint locations from a depth sensor with full-body tracking capability. The proposed method encodes actions in a weighted directed action graph to model the kinematics of actions and models distribution of joints to be a set of salient postures that correspond to the nodes in the action graph. In addition, we propose a hierarchical action seeking framework for increasing recognition performance and raising the accuracy rate. In our hierarchical framework, the human body is divided into four parts ( left upper limb, right upper limb, left lower limb, and right lower limb). We propose a motion indicator to evaluate the degree of movement in each human limb, referred to as motion descriptor. Then, the system classifies observed motion into several a specific action clusters using motion descriptor content. Second, we employ a smaller action data set, which is relative to specific motion, to seek the most appropriate action. Experimental results show that about 80% recognition accuracy were gained in non-hierarchical system, and over 90% recognition accuracy were achieved in hierarchical system.
Moreover, we employ the proposed action recognition framework to an interactive performance between an intelligent puppet and actors. In this performance, the puppet is able to realize the meaning of actor’s behavior by our action recognition framework. Thus, the puppet is able to make related action to the actors.
Experimental results demonstrate the proposed hierarchical-based action recognition approach reduces the computational cost and increases the accuracy rate.

並列關鍵字

Action recognition ； Action graph ； Shape descriptor ； Salient posture ； Motion descriptor

參考文獻

[2] A. Galata, N. Johnson, and D. Hogg, “Learning variable-length Markov models of behavior,” Comput. Vis. Image Understand., vol. 81, pp.398–413, 2001.

[3] A. Bobick and J. Davis. The recognition of human movement using temporal templates. IEEE Trans. PAMI, 23(3):257– 267, 2001.

[4] P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-temporal features. VS-PETS, pp. 65-72, 2005

[5] I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. CVPR, pp. 1-8, 2008.

[6] J. C. Niebles, H.Wang, and L. Fei-Fei. Unsupervised learning of human action categories using spatial-temporal words. Int’l J. Computer Vision, 79(3):299– 318, 2008.