新型卷積神經網路之研究

Research on New Convolutional Neural Networks

陳建華(Chien-Hua Chen)

指導教授 ： 謝哲光

義守大學/智慧科技學院/電機工程學系/博士(2022年)

摘要

現今深度學習是機器學習領域中最熱門的話題。於影像處理方面最流行的特徵擷取器是使用深層卷積神經網路，它是由許多卷積層堆疊而成。深層卷積神經網路具有很好的特徵擷取能力，並已成功應用於許多的實務問題中。然而加深神經網路的層數亦代表著從資料輸入到輸出之間需要經過較多步驟，導致計算時間不僅較長亦難以利用平行化的方法去減少計算時間；同時由於權重參數變多，因而需要更龐大的資料集。如果不以增加層數的方式去改進卷積神經網路之效能，勢必要考慮提升單一卷積層的功能；而要達成此目的或許可以寄望於一些常用的非線性方法或特殊的機器學習辦法。本研究的目標在於提升單一卷積層的學習能力，因此將建立數種非線性的卷積層，如餘弦卷積層、核卷積層及模糊卷積層等。本研究將以標準的影像資料集 MNIST, Kuzushiji-MNIST, Fashion-MNIST 及CIFAR10 進行驗證以提供客觀的比較結果。

關鍵字

非線性卷積 ； 餘弦 ； 相關係數 ； 核函數 ； 模糊類神經網路

並列摘要

Today, deep learning is the hottest topic in the area of machine learning. The most popular feature extractors in image processing are deep convolutional neural networks (CNNs), which are stacked with many convolutional layers. Deep CNNs have excellent feature extraction ability, and have been successfully applied to many practical problems. However, increasing the number of layers in a neural network also means that more calculation steps are required from the inputs to outputs, resulting in a longer computation time that is difficult to be reduced by parallel computing. Also, due to the increasing number of weightings, much lager datasets are required. In order to improve CNNs' capability without increasing the number of layers, it would be necessary to improve the capability of a single convolutional layer. This may be achieved by some common nonlinear methods or some special machine learning techniques. The goal of this study is to improve the learning ability of a single convolutional layer. To this end, several nonlinear convolutional layers, such as cosine convolution, kernel convolution, and fuzzy convolution layers, will be proposed. Several benchmark image datasets such as MNIST, Kuzushiji-MNIST, Fashion-MNIST, and CIFAR10 will be used to validate the proposed convolutional layers.

並列關鍵字

Non-linear convolution ； Cosine ； Correlation coefficient ； Kernel function ； Fuzzy neural network

參考文獻

[1] J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon, "A proposal for the dartmouth summer research project on artificial intelligence," Aug. 31, 1955. http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html (accessed March 30, 2022).

[2] A. Newell, J. Shaw, and H. Simon, "Report on a general problem-solving program," in Information Processing: Proceedings of the International Conference on Infonnation Processing, UNESCO, Paris, pp. 256-264, 1960.

[3] W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," The bulletin of mathematical biophysics, vol. 5, no. 4, pp. 115-133, 1943.

[4] A. K. Jain, J. Mao, and K. M. Mohiuddin, "Artificial neural networks: A tutorial," Computer, vol. 29, no. 3, pp. 31-44, 1996.

[5] L.X. Wang, A course in fuzzy systems and control. Prentice-Hall, New Jersey, 1996.