以不平衡資料集之分類技術進行結合人臉與語音之身分確認

Fusion of Face and Voice Information in Person Identity Verification with Class-Imbalanced Dataset

鄭先廷(Hsien-Ting Cheng)

指導教授 ： 洪ㄧ平

共同指導教授 ： 陳祝嵩(Chu-Song Chen)

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2005年)

https://doi.org/10.6342/NTU.2005.02217

摘要

在本論文中，根據更多的資訊將帶來較佳的辨識結果，我們提出了在信心層級來結合了人臉以及語音的資訊進而進行以生物特徵為基礎的身份確認。從系統的觀點，我們建構了一個讓使用者可輕鬆註冊、介面人性化、以及防止偽裝入侵的線上身份確認系統。以方法論的角度，我們使用了目前公認最好、最新的技術來發展人臉跟語音的模組。在整合部分，為了利用所有可使用的人臉資訊，我們提出了”多張人臉/單一語句”的策略進而降低了人臉偵測錯誤或是對位錯誤的風險。支持分類器(Support Vector Machine)被選用來當作二維分類器。

除了個別化的模組以及後端的整合，在此論文中我們更探討了”從一個不平衡資料集中學習”的問題。一般而言，我們都希望可以有越多的訓練資料可供訓練越好，然後若是訓練資料的分佈極不平衡，一般的分類器方法將會受到影響進而偏向資料較多的類別。此問題在分類問題中其實相當常見，但在身份辨識或是身份確認的領域，我們是首次提出並加以解決，實驗結果證明透過不同階層、方式的處理，將會使得不平衡的現象獲得改善。

關鍵字

身份確認 ； 人臉確認 ； 語者確認 ； 多模式整合 ； 支持分類器 ； 不平衡資料集

並列摘要

Based on the idea of more information brings better performance, in this thesis we presents a confidence-level fusion method to combine face and voice information in biometric person identity verification. In systematic aspect, we develop an on-line verification system with light-weight enrollment process, fraud precaution mechanism and an easy-to-use verification interface. While in algorithmic point of view, state-of-the art techniques are used to build the face and voice experts. More-over, a multi-face/single-sentence strategy is proposed to utilize all the available in-formation to reduce the cost of miss-detection and miss-registration of face, and support vector machine (SVM) is employed as the binary fusion classifier.

In addition to individual experts and the fusion work, another important issue proposed in this thesis is learning from a class-imbalanced dataset. To train a good classifier, most of the time we use as many training data as possible. However in lots of fields involving classification jobs, training data is highly imbalanced distributed from class to class, ordinary classification algorithms will favor to the class which has more training samples. In the field of identity verification we are the first one that discover such important issue and try to handle it. Different level approaches are studied and implemented to reduce the influence of imbalanced dataset and lead to better performance.

並列關鍵字

person identity verification ； face verification ； speaker verification ； multimodal fusion ； SVM ； class-imbalanced dataset

參考文獻

[2] T. Kanade. Computer recognition of human faces. Birkhauser Verlag, Stuttgart, Germany 1973

[3] R. Brunelli and T. Poggio. Face recognition: features versus templates. IEEE Trans. on Pattern Analysis and Machine intelligence, 15(10), 1993.

[5] M. Lades, J. C. Vorbr ‥uggen, J. Buhmann, J. Lange, C. von der Malsburg, R. P. W‥urtz, and W. Konen. Distortion invariant object recognition in the dy-namic link architecture. IEEE Transactions on Computers, 42(3):300–311, Mar. 1993.

[6] L. Wiskott, J.-M. Fellous, N. Kr ‥uger, and C. von der Malsburg. Face recog-nition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):775–779, July 1997.

[7] C. Kotropoulos, A. Tefas, and I. Pitas. Frontal face authentication using mor-phological elastic graph matching. IEEE Transactions on Image Processing, 9(4):555–560, Apr. 2000.