Have library access?
  • Theses


A Dimensionality Reduction Layer by Projection in a Convolutional Neural Network

Advisor : 陳素雲



Parallel abstracts

In this research, we proposed a dimensionality reduction method that takes the place of the pooling methods. A pooling layer is usually put after a convolutional layer to summarize the output images from the convolutional layer. At the moment, the max-pooling method or the average-pooling method is widely used on CNN. On the other hand, our proposed method transforms an output image from a convolutional layer into a lower-dimensional image by multiplying truncated orthogonal matrices. We regard the truncated orthogonal matrices as parameters of the neural network, and we derived the derivatives that appear in the backpropagation algorithm. Moreover, we also verified the feasibility of our proposed method by implementing it as a computer program. We compared the performance of our proposed method with the pooling methods under similar conditions. In the experiment, our proposed method achieved better performance than the pooling methods.


[1] Chainer: A flexible framework for neural networks. https://chainer.org/. (Accessed on 02/28/2019).
[2] Pytorch. https://pytorch.org/. (Accessed on 07/05/2019).
[3] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.
[4] J. C. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159, 07 2011.
[5] F. Chollet et al. Keras. https://keras.io, 2015.