以簡化群體演算法優化卷積神經網路結構及超參數調整

Optimization Convolutional Neural Network Architecture and Hyperparameter Using Simplified Swarm Optimization

鍾杰峰(Chieh-Feng Chung)

指導教授 ： 葉維彰

國立清華大學/工學院/工業工程與工程管理學系/碩士(2020年)

摘要

深度學習是一項功能強大的機器學習技術，其在圖像分類，語義圖像分割等方面均扮演重要的角色。鑑於圖片的高維度及同類圖片間的變異性，圖像分類任務一直是機器學習艱鉅的任務之一；而卷積神經網路（CNN）在處理圖像分類問題上具有優異的表現。然而，手動設計一個優秀的卷積神經網路模型除了需要具備豐富的相關領域知識外，更需耗費大量的時間。神經網路結構搜索（NAS）和超參數優化（HPO）可以通過多種搜索策略自動搜尋網路的結構及超參數的良好組合。本研究提出了一種基於簡化群體演算法 (SSO) 的演算法作為搜索策略，並納入三種不同的更新機制於演算法中以尋找良好的卷積神經網路結構及超參數組合。此外，所提出的演算法在優化過程中亦將變數的重要性及值域納入權衡。為了使卷積神經網路的結構和超參數可以被提出的搜索策略進行優化，需要一種將網路結構及超參數訊息映射到演算法編碼中的編碼策略，故本研究亦提出一種三級式編碼策略，將卷積神經網路結構和超參數編碼成演算法的解。為了驗證所提出方法的表現，本研究於Convex及MNIST-RB資料集上與基於演化算法的神經網路結構搜索方法及著名的卷積神經網路模型進行比較。實驗結果顯示，本研究的提出的方法可以的達到相當具有競爭力的表現。

關鍵字

神經網路架構搜尋 ； 超參數優化 ； 卷積神經網路

並列摘要

Deep learning is one of the powerful machine learning techniques in the world and plays an important role in image classification, semantic image segmentation, etc. Among the tasks, the image classification task is one of the hard jobs for machine learning due to the high dimensionality of images and the variability in the same class. Convolutional Neural Networks (CNNs) take an important place in it. But design a prominent CNN manually requires ample domain knowledge and is a time-consumption job. The Neural architecture search (NAS) and hyperparameter optimization (HPO) can automatically search the good combination of architecture and hyperparameters of the network through a variety of search strategies. In this study, we proposed an SSO-based algorithm as a search strategy that applies three different kinds of update mechanisms to find the good structure and hyperparameters combination of CNN. Moreover, this algorithm took the importance and the range of value of variables into consideration during the search process. For allowing CNN's architecture and hyperparameters can be optimized by the search strategy, an encoding strategy is required which mapping the network's information to a series of encoding. We encoded the CNN structure and hyperparameters into solutions by a three-level encoding strategy and contained different CNN's information at varying levels. To verify our approach's performance, we compare image classification accuracy on the Convex and MNIST-RB datasets with competitors, including evolutionary NAS approaches and famous CNN models. The experiment results indicate our approach could achieve promising performance compare to rivals.

並列關鍵字

Neural architecture search ； Hyperparameter optimization ； Convolutional neural network

參考文獻

[1] I. Sutskever, G. E. Hinton, and A. Krizhevsky, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems,pp. 1097–1105, 2012.

[2] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab:Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence,

vol. 40, no. 4, pp. 834–848, 2017.

[3] A. Barbu, M. Suehling, X. Xu, D. Liu, S. K. Zhou, and D. Comaniciu, “Automatic detection and segmentation of lymph nodes from ct data,” IEEE Transactions on Medical Imaging, vol. 31, no. 2, pp. 240–250, 2011.

[4] F. Sultana, A. Sufian, and P. Dutta, “Advancements in image classification using convolutional neural network,” in 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), pp. 122–129, IEEE, 2018.