通過卷積自編碼器神經網路之小數據整合於化工製程建模研究

Process Modeling With Small Data Integration via Deep Convolutional Autoencoder-based Embedding Model

劉梓堂(Tzu-Tang Liu)

指導教授 ： 姚遠

國立清華大學/工學院/化學工程學系/碩士(2021年)

摘要

隨著數據科學的興起，大數據正在成為學術研究和商業業務發展的流行趨勢。但是，對於高附加價值產業，不一定有足夠的數據可用於建立一個可靠的數據驅動模型。因此如何整合多個不同但類似的任務收集的小數據並通過在任務之間共享信息來構建準確的模型是一個研究挑戰。而其中一個例子是針對不同操作條件配置的雙螺桿擠出機製程進行建模。
卷積神經網絡（CNN）是計算機視覺中常用的深度學習技術。在這項工作中，採用了卷積自動編碼器（一種深度圖像去噪模型）來描述雙螺桿擠出過程中的定性因素，即螺桿元件的幾何形狀。具體而言，通過卷積自動編碼器嵌入來提取這些定性因素中包含的信息；然後將嵌入值以及定量條件合併輸入到前饋全連接神經網絡模型，以實現過程輸出的預測。與傳統的卷積自動編碼器不同，該模型同時考量了自動編碼器的重構損失和最終預測的回歸損失進行迭代訓練，從而確保了模型的可解釋性。
在本次研究中以雙螺桿押出過程的數值模擬用於說明所提出模型的可行性。從研究的結果之下，表現出該模型具有良好的解釋性和預測準確性。特別是對於包含未知定性因素的過程模擬條件下，即使僅收集了有限數量種類的螺桿元件製程數據，該模型仍根據不同螺桿間的相似性而做出合理的預測。

關鍵字

前饋全連接神經網路 ； 卷積自動編碼器 ； 製程建模 ； 小數據 ； 整合分析

並列摘要

Big data is becoming a popular trend of research and business development. Nevertheless, for high-value process industries, sufficient data is not necessarily available for data-driven process modeling. How to integrate small data collected from several different tasks and build an accurate process model by sharing the information between tasks is a challenge research topic. A typical example is the modeling of a twinscrew extruder for screw configuration.
Convolutional neural network (CNN) has been a common deep learning technique used in computer vision. In this work, a convolutional autoencoder, a deep image denoising model, is adopted to describe the qualitative factors, i.e. the geometries of the screw elements, in a twin-screw extrusion process. In detail, the information contained in these qualitative factors is extracted by convolutional autoencoder embedding; then the embedding codes are connected to a fully connected feedforward neural network, together with the quantitative process conditions, to achieve the prediction of the process outputs. Different from the conventional convolutional autoencoders, the proposed model is trained using both the reconstruction loss of autoencoder and the regression loss of final prediction, ensuring the model interpretability.
Numerical simulations of a twin-screw extrusion process are used to illustrate the feasibility of the proposed model. In the studied case, it shows that this model has both good interpretability and prediction accuracy. Specifically, for the process contain qualitative factors with extrapolate values, the model can still make reasonable predictions, given that only a limited amount of data was collected for each screw configuration.

並列關鍵字

Feedforward fully connected neural network ； Convolutional Autoencoder, ； Process modeling ； Small data ； Integrated analysis

參考文獻

1. Qian, Peter Z. G., Huaiqing Wu, and CF Jeff Wu. "Gaussian process models for computer experiments with qualitative and quantitative factors." Technometrics 50.3 (2008): 383-396.

2. G. Hinton, L. Deng, D. Yu, G. Dahl, A.-r. Mohamed, N. Jaitly, et al., "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal processing magazine, vol. 29, 2012.

3. D. Cireşan, U. Meier, and J. Schmidhuber, "Multi-column deep neural networks for image classification," arXiv preprint arXiv:1202.2745, 2012.

4. T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, "Convolutional, long short-term memory, fully connected deep neural networks," in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 4580-4584.

5. W. Sun, S. Shao, R. Zhao, R. Yan, X. Zhang, and X. Chen, "A sparse auto-encoder-based deep neural network approach for induction motor faults classification," Measurement, vol. 89, pp. 171-178, 2016.