基於機率圖形模型之人體動作辨識

Human Action Recognition based on Probabilistic Graphical Models

林士堯(Shih-Yao Lin)

指導教授 ： 洪一平

共同指導教授 ： 陳祝嵩 ； 林彥宇(Yen-Yu Lin)

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/博士(2016年)

https://doi.org/10.6342/NTU201610023

摘要

無資料

關鍵字

人體動作辨識 ； 機率圖形模型 ； 條件隨機場

並列摘要

Video-based human action recognition is a key component of video analysis. Despite significant research efforts over the past few decades, human action recognition still remains a challenging problem. My dissertation investigates three important and emerging topics in practical action recognition problems.

First, we consider a crucial problem of recognizing actions having similar movements. An approach called Action Trait Code (ATC) for human action classification is proposed to represent an action with a set of velocity types derived by the averages velocity of each body part. An effective graph model based on ATC is employed for learning and recognizing human actions. To examine recognition accuracy, we evaluate our approach on our self-collected action database.

Second, an action video may have many similar observations with occasional and irregular changes, which make commonly used fine-grained features less reliable.
This dissertation introduces a set of temporal pyramid features that enriches action representation with various levels of semantic granularities. For learning and inferring the proposed pyramid features, we adopt a discriminative model with latent variables to capture the hidden dynamics in each layer of the pyramid. Experimental results show that our method achieves more favorable performance than existing methods.

Third, actions taken in real-world applications often come with corrupt segments of frames caused by various factors. These segments may be of arbitrary lengths and appear irregularly. They change the appearance of actions dramatically, and hence degrade the performance of a pre-trained recognition system. In this dissertation, I presented an integrated approach which includes two key components {em outlier filtering} and {em observation completion}. It can tackle this problem without making any assumptions about the locations of the unobserved segments. Specifically, the outlier frame filtering mechanism is introduced to identify the unobserved frames. Our observation completion algorithm is designed to infer the unobserved parts. It treats the observed parts as the query to the training set, and retrieves coherent alternatives to replace the unobserved parts. Hidden conditional random fields (HCRFs) are then used to recognize the filtered and completed actions. Furthermore, we collect a new action dataset where outlier frames irregularly and naturally present. Besides this dataset, our approach is evaluated on two benchmark datasets, and compared with several state-of-the-art approaches. The superior results obtained by using our approach demonstrate its effectiveness and general applicability.

On the other hand, I also develop a unified action recognition framework that can jointly handle the outlier detection and predicting actions.
For each action to be recognized, we explore the mutual dependency between its frames, and augment each frame with extra alternative frames borrowed from training data. The augmentation mechanism is designed in the way where a few alternatives are of high quality, and can replace the detected corrupt frames. Our approach is developed upon hidden conditional random fields. It integrates corrupt frame detection and alternative selection into the process of prediction, and can more accurately recognize partially observed actions. The promising results manifest its effectiveness and large applicability.

並列關鍵字

Human Action Recognition ； Probabilistic Graphical Models ； Conditional Random Fields

參考文獻

[15] Tian Lan, Tsung-Chuan Chen, and Silvio Savarese. A hierarchical representation for future action prediction. In Proc. Euro. Conf. Computer Vision, pages

[73] Simon Fothergill, Helena M. Mentis, Pushmeet Kohli, and Sebastian Nowozin. Instructing people for training gestural interactive systems. In Proc. Int’ Conf.

[2] Nataliya Shapovalova, Arash Vahdat, Kevin Cannons, Tian Lan, and Greg Mori. Similarity constrained latent support vector machine: An application to

[3] Qinfeng Shi, Li Cheng, Li Wang, and Alex Smola. Human action segmentation and recognition using discriminative semi-markov models. Int. J. Computer

[4] Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3d skeletons as points in a lie group. In Proc. Conf.