工業應用之偽標籤特徵半監督式學習

A Novel Approach, Pseudo-Representation Labeling, for Semi-Supervised Learning on Industrial Applications

楊淞博(Song-Bo Yang)

指導教授 ： 于天立

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202003577

摘要

近年來，半監督學習的相關技術大大降低了模型對大量標記數據的依賴，並且在ImageNet、CIFAR-10等著名的數據集上展現了令人驚豔的成果。然而因為資料集型態的差異性太大，造成許多著名的方法（尤其是基於數據增強技術的方法）在工業應用類型的資料集上仍然難以獲得顯著的改善。因此本論文的目標是在工業類型的影像資料上透過半監督學習框架來改進現有的固定架構模型，並且提高模型的準確度。

本論文提出了偽表示標記，一個簡單而靈活的框架，透過批次增加標記數據並結合自監督學習來試圖改進上述的問題。然而，在添加標記數據的同時也會產生一定程度的雜訊，進而影響模型的表現，因此我們提出了方法來衡量資料雜訊對整體準確度的影響。此外為了最小化產生的雜訊，我們傾向於小批次的加入標記數據，並且會對批次量的大小進行探討。相較於其他的半監督學習演算法，這個框架更為彈性，可以根據不同的資料型態來選擇自監督學習的任務，並在大量的未標籤資料上學習到整個資料集的特性。在我們的實驗中，在工業類型的分類問題（例如WM-811K晶圓圖和MIT-BIH 心律不整分類）上，此框架的表現優於當前的最新半監督學習方法。

關鍵字

半監督學習 ； 自我監督學習 ； 偽標籤 ； 工業應用 ； 資料擴充

並列摘要

In recent years, semi-supervised learning (SSL) has shown tremendous success in leveraging unlabeled data to improve the performance of deep learning models, which significantly reduces the demand for large amounts of labeled data. Many SSL techniques have been proposed and have shown promising performance on famous datasets such as ImageNet and CIFAR-10. However, some exiting techniques (especially data augmentation based) are not suitable for industrial applications empirically. Therefore, this work proposes the pseudo-representation labeling, a simple and flexible framework that utilizes pseudo-labeling techniques to iteratively label a small amount of unlabeled data and use them as training data. In addition, our framework is integrated with self-supervised representation learning such that the classifier gains benefits from both labeled and unlabeled data. This framework can be implemented without being limited at the specific model structure, but a general technique to improve the existing model. Compared with the existing approaches, the pseudo-representation labeling is more intuitive and can effectively solve practical problems in the real world. Empirically, it outperforms the current state-of-the-art semi-supervised learning methods in industrial types of classification problems such as the WM-811K wafer map and the MIT-BIH Arrhythmia dataset.

並列關鍵字

Semi-Supervised learning ； Self-Supervised Learning ； Pseudo Labeling ； Industrial Application ； Data Augmentation

參考文獻

[1] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems, pages 5050-5060, 2019.

[2] P. Cascante-Bonilla, F. Tan, Y. Qi, and V. Ordonez. Curriculum labeling: self-paced pseudo-labeling for semi-supervised learning. arXiv preprint arXiv:2001.06001, 2020.

[3] O. Chapelle, B. Scholkopf, and A. Zien. Semi-supervised learning (Chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks, 20(3):542{542, 2009.

[4] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. Randaugment: Practical data augmentation with no separate search. arXiv preprint arXiv:1909.13719, 2019.

[5] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255. IEEE, 2009.