應用注意力機制於深度學習之行為辨識

Human Activity Recognition Using Attention Mechanism on Deep Learning

陳文輝(Wen-Hui Chen) ； 張瑀軒(Yu-Xuan Zhang)

《國立宜蘭大學工程學刊》 13期 (2019/12) Pp. 30-42

https://doi.org/10.6176/BCE.201912_(13).0003

摘要

近年來，人工神經網路取得突破性的發展，許多研究利用循環神經網路來對序列資料進行編碼，取得上下文的關聯。本研究以行為辨識為主軸，利用長短期記憶網路加上注意力機制，讓模型能聚焦在重要的輸入上，使得編碼後的上下文向量更具代表性。本文提出的方法和未使用注意力機制的模型相比，提升了3.8%的辨識率。

關鍵字

注意力機制 ； 循環神經網路 ； 深度學習 ； 行為辨識 ； 行動裝置

並列摘要

In recent years, deep neural networks have made breakthroughs. Many studies use recurrent neural networks (RNNs) to encode sequence data and obtain context correlation. In this study, behavior recognition is the main axis, and the model can focus on the important input by using the long short-term memory (LSTM) and attention mechanism, which overcomes the problem of model performance degradation caused by too long sequence data, and makes the context vector more representative. Compared with the model without attention mechanism, the proposed method improves the recognition rate by 3.8% on average.

並列關鍵字

Attention Mechanism ； Recurrent Neural Network ； Deep learning ； Activity Recognition ； Mobile Devices

參考文獻

Auli, M.,Galley, M.,Quirk, C.,Zweig, G.(2013).Joint Language and Translation Modeling with Recurrent Neural Networks.EMNLP.(EMNLP).

Meng, Z.,Watanabe, S.,Hershey, J.,Erdogan, H.(2017).Deep Long Short-Term Memory Adaptive Beamforming Networks for Multichannel robust Speech Recognition.International Conference on Acoustics, Speech and Signal Processing.(International Conference on Acoustics, Speech and Signal Processing).

Li, L.,Zhang, Z.L.(2017).Emphasizing essential words for sentiment Classification Based on Recurrent Neural Networks.Journal of Computer Science and Technology.32,785-795.

Gers, F.(2001).Long Short-Term Memory in Recurrent Neural Networks.Ecole Polytechnique Fédérale de Lausanne.

Badhanau, D.,Cho, K.,Dengio, Y.(2014).Neural Machine Translation by Jointly Learning to Align and Translate.ICLR.(ICLR).