從能量層面探討卷積神經網絡及其架構

Mining Structures of Convolutional Neural Networks: An Energy Perspective

陳君函(Chun-Han Chen)

指導教授 ： 張世杰

國立清華大學/電機資訊學院/資訊工程學系所/碩士(2017年)

摘要

近幾年來，卷積神經網路除了在計算機視覺領域中扮演著相當重要的角色，也被廣泛的應用在圖像辨識上面。因此，卷積神經網路的運算複雜度和能量消耗量已然成為一個重要的議題，特別是應用在嵌入式系統，或者其他電池供電的行動設備。除了減少網絡之間的運算複雜度之外，如果我們能夠在訓練或者測試階段之前，預先估計給定的卷積神經網路的能量消耗量，我們就能夠知道此卷積神經網路是否適合應用在行動裝置。因此，藉由這個動機，我們提出了一個可以有效地預估卷積神經網路的能量消耗量的模型。在我們的論文中，一開始，我們先詳細地分析不同卷積神經網路內部參數與核心程式之間的關係，接下來根據我們觀察的結果，提出了一個可以有效地在訓練或者測試階段之前，預估卷積神經網路的能量消耗量的模型。我們使用CIFAR-10 資料集作為我們的實驗數據，並且在Caffe 上執行實驗，最後我們的方法在預測卷積神經網路的能量消耗量達到了平均只有14.41%的誤差率。

關鍵字

深度學習 ； 卷積神經網絡 ； 能量消耗

並列摘要

Recently convolutional neural networks (CNNs) have drawn much attentions and been widely applied on image recognition; therefore, the complexity of computation and energy consumption have become a big issue for deploying CNNs, especially on embedded systems or other battery-powered mobile devices. Apart from reducing the complexity of network computations, if we could estimate the energy consumptions of the given network configuration before train or test phases, we would realize that whether the CNNs can be deployed on mobile devices or not. As the result, we propose a predictive energy model to effectively predict the energy consumption of a CNN. In this work, first we analyze the relation between different network configurations and kernel functions operations reported by NVIDIA profiler tool in detail, and then based on the analysis, we propose a predictive energy model that could calculate an estimated energy consumption as we have the architecture of a convolutional neural network before test phases. The experiments are processed with CIFAR-10 dataset and are implemented in Caffe, and the overall error rate of our methodology for predicting energy consumption is 14.41%.

並列關鍵字

deep ； learning ； convolutional ； neural ； network ； energy

參考文獻

[2] S. Changpinyo, M. Sandler, and A. Zhmoginov. The power of sparsity in convolutional neural networks. arXiv preprint arXiv:1702.06257, 2017.

[7] A. Karpathy. Cs231n: Convolutional neural networks for visual recognition. Neural networks, 1, 2016.

[8] K. Kasichayanula, D. Terpstra, P. Luszczek, S. Tomov, S. Moore, and G. D. Peterson. Power aware computing on gpus. In Application Accelerators in High Performance Computing (SAAHPC), 2012 Symposium on, pages 64–73. IEEE, 2012.

[9] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009.

[14] T.-J. Yang, Y.-H. Chen, and V. Sze. Designing energy-efficient convolutional neural networks using energy-aware pruning. arXiv preprint arXiv:1611.05128, 2016.