Deep Learning for Expressions Recognition on Small Dataset Using the Methods of Dual Transfer Learning and Feature Visualization

Junhuan Lin ； Zhen Liu ； Chih-Chieh Hung ； Ting Ting Liu ； Yan Jie Chai

《電腦學刊》 31卷5期 (2020/10) Pp. 127-145

https://doi.org/10.3966/199115992020103105010

摘要

In this paper, we propose a dual transfer learning framework for image-based facial expressions recognition combining the deep convolutional neural networks(CNN) and feature visualization technique. The framework includes three steps. The first step is visualizing the features of BVLC's CNN to observe the pixels-level images reconstructed by the strongest activated neurons using deconvolutional method. As a result, some useful convolutional layers of the BVLC's CNN can be transferred to the next new targeting CNN immediately. Then the first transfer learning model of CNN is built up by concatenating the convolutional layers from BVLC's CNN to other convolutional layers. The second step is visualizing the features of the first transfer learning model after being trained on a medium dataset which is relevant to attributes of face. According to the results of feature visualization, the second transfer learning model can be built like the first steps. In the last step, the second transfer learning model is fine tuned on the CK+ dataset and used for recognizing the expressions. The testing results of classifying basic expressions demonstrate that our model based on dual transfer learning approach outperforms the current state-of-the-art works. Additionally, we also verify that our model is robust against interferences caused by various occlusions.

關鍵字

convolutional neural network ； feature visualization ； transfer learning