具分類/迴歸資格之特徵工程與解釋性萃取以利深度分類/迴歸器建模

Extraction of Classification/Regression-qualified and Explainable Features for Deep Classifier/Regressor Modeling

褚芸(Yun Chu)

指導教授 ： 藍俊宏

國立臺灣大學/工學院/工業工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202101525

摘要

隨著科技不斷的精進與創新，資料分析不論在資料規模、運算技術和分析方式都帶起了一波典範的轉移，面對來源多樣且結構複雜的資料探勘技術中，特徵工程是至關重要的步驟，目前主流使用的降維技術，多為非監督式學習萃取出保留資訊最大化的特徵，然而其中的特徵並不全然適合於建立分類/迴歸模型，因此本論文以研究此問題為出發點，設計以半監督式的卷積自動編碼器分類/迴歸模型，藉著把卷積自動編碼器的中間層攤平後，再送入分類/迴歸器同步訓練，目標是經由一次訓練即萃取出具備分類/迴歸能力之特徵，使這些特徵不僅有重建原始資料的能力，還能維持分類/迴歸模型預測的準確度。
深度學習技術在近年來獲得許多重要突破，在影像辨識、語音處理、特徵診斷等廣泛應用催生了許多成功案例，因此類神經網路的黑盒子問題也重新受到檢視，面對模型中無法理解的推理過程，將使其應用大大受限。而本論文提出之卷積自動編碼器分類/迴歸模型亦屬深度學習模型，也同樣面對黑盒子無法解釋的困境，因此本研究先回顧當前人工智慧可解釋性之發展，發展演算法拆解已訓練完成的深度神經網路模型，以得出一套對應的特徵解析架構，使提出的特徵萃取模型能夠具備基本的解釋性。
本研究最後使用Fashion MNIST、半導體化學機械研磨製程的兩資料集來驗證提出的特徵萃取模型，並透過視覺化呈現特徵重要性，找出影響模型的關鍵變數，達到模型預測性和可解釋性之間的平衡，提升深度神經網路技術的應用性。

關鍵字

卷積自動編碼器 ； 卷積神經網路 ； 特徵萃取 ； 分類器/迴歸器建模 ； 模型可解釋性

並列摘要

With the continuous innovation in science and technology, data analysis has brought about a wave of paradigm shifts in data volume, computing technology and modeling methods. In the face of diverse sources and complex structure data exploration technologies, feature engineering is particularly essential. The current mainstream of dimension reduction technologies is mostly unsupervised learning to extract the features that can retain the maximum information within the data. However, the features are not entirely suitable for constructing classification/regression models. This thesis is thus motivated to design the framework that integrates the convolutional autoencoder with deep classification/regression model. Through flattening the hidden feature layer of the convolutional autoencoder to be the input into the classifier/regressor, the two losses are integrated for simultaneous training. The goal is to extract the classification/regression-qualified features as well as to retain the capability of reconstructing the original data.
Deep learning technology has made many important breakthroughs to be popularly practiced in a wide range of domains, such as image recognition, speech processing, and fault diagnosis. However, the black box problem in artificial neural networks has again been re-examined as understanding the model inference is crucial. The convolutional autoencoder integrated classification/regression model proposed in this research is also of deep learning models, and thus faces the dilemma of the black-box model. To overcome the problem, reviews of AI explainability are made and an algorithm that can disassembly the deep neural network model has been developed to obtain a set of meaningful covariates. Consequently, the extracted features will become explainable.
Two data sets: Fashion MNIST and the data from a chemical mechanical polishing process are used to validate the proposed method. The feature importance is visualized to identify the key variables that affect the model, so as to keep the predictability and explainability in one model. The balance between the two abilities will surely enhance the applicability of deep neural network technology.

並列關鍵字

Convolutional Autoencoder (CAE) ； Convolutional Neural Networks (CNN) ； Feature Extraction ； Classifier/Regressor Model ； Machine Learning Algorithms ； Explainable AI (XAI)

參考文獻

Anuradha, J. (2015). A brief introduction on Big Data 5Vs characteristics and Hadoop technology. Procedia Computer Science, 48, 319-324.

Azarang, A., Manoochehri, H. E., Kehtarnavaz, N. (2019). Convolutional autoencoder-based multispectral image fusion. IEEE Access, 7, 35673-35683.

Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K. R., Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One, 10(7), e0130140.

Bengio, Y., Goodfellow, I., Courville, A. (2017). Deep Learning (Vol. 1). Massachusetts, USA:: MIT press.

Böhning, D. (1992). Multinomial logistic regression algorithm. Annals of the Institute of Statistical Mathematics, 44(1), 197-200.