卷積神經網路刪減使用以訓練為基礎之重要通道辨識

Convolutional Neural Network Pruning by Training-based Important Channel Identification

樊恩宇(En-Yu Fan)

指導教授 ： 江介宏

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201900897

摘要

卷積神經網路在許多應用上表現良好，然而大量的運算和記憶體需求仍使其受 到諸多障礙。許多文獻均提出以通道為單位來做網路刪減，其中大部分的方法將 通道彼此的相關性和網路訓練分開討論，或只以一層或連續兩層的網路資訊做層 層的網路刪減。因此，本篇文獻將以在訓練過程中引入算分網路和重要通道兩個 概念為基礎，設計一對網路通道進行刪減的方法，具體而言，我們將通道的相互 關係融入訓練過程，並對訓練完的網路進行每一層的通道刪減。在當前的卷積神 經網路架構以及多個數據集上，實驗結果顯示，我們所提出的方法在減少參數和 浮點運算上有良好的結果;此外，刪減後網路的表現下降浮動是可忽略的，有些 甚至超越原來網路的表現。

關鍵字

卷積神經網路、網路刪減、通道刪減、模型壓縮

並列摘要

Despite the tremendous success of convolutional neural networks (CNNs) in various applications, their deployment is greatly obstructed by its high computational cost and its large memory usage. Many approaches have been proposed to prune the network channel-wisely. Nevertheless, most consider the interrelations of channels independently to training or they prune the network in a layer-by-layer manner leveraging the statistics only by an individual layer or two consecutive layers. In this work, we devise a strategy that introduces the concepts of Scoring Network (SN) and Importance of Channels (IofC) into training for channel pruning. Specifically, we take interdependencies of channels into account by combining them into the training phase and jointly prune the channels of every layer based on the trained model. Experimental results evaluated on multiple datasets with several modern CNN models demonstrate that our method can produce promising reductions for modern CNN frameworks in both parameters and floating point operations (FLOPs) while the performance loss is negligible, or even better relative to the unpruned counterparts.

並列關鍵字

convolutional neural networks, network pruning, channel pruning, model compression

參考文獻

[1] K. Simonyan and A. Zisserman. Very deep convolutional networks for large- scale image recognition. abs/1409.1556, 2014.

[2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, 2015.

[3] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. SSD: Single shot multibox detector. In ECCV, 2016.

[4] Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional net- works for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(4):640–651, April 2017.

[5] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In Proceedings of the 2015 IEEE Interna- tional Conference on Computer Vision (ICCV), ICCV ’15, pages 1520–1528, Washington, DC, USA, 2015. IEEE Computer Society.