開發半監督VAE/GAN模型於非線性靜態與動態程序的軟測量

Developing Semi-Supervised Variational Autoencoder/ Generative Adversarial Network for Static and Dynamic Nonlinear Process Soft Sensor model

陳俊龍(Dave Tanny)

指導教授 ： 陳榮輝

中原大學/工學院/化學工程研究所/碩士(2020年)

https://doi.org/10.6840/cycu202000670

摘要

軟測量模型是現今許多製造業者深感興趣的議題之一，是因為該模型可以推斷出產品的質量以滿足客戶的規格要求。然而，要建立軟測量模型存在許多難題，從品質數據到整體程序特徵都須納入考量，且軟測量模型的準確性和有效性也會受限於模型的假設、結構以及用於訓練模型的數據的品質。
在大多數化工廠的過程中，存在著干擾隨機性及非線性的動態特性，因此難以使用常見的軟測量模型去精準地預測產品的品質。此外，有鑑於化工廠記錄到的數據通常包含了大量雜訊，且品質及過程數據的數量不同，因此，在這種情況下，要訓練出可靠且強大的軟測量模型是十分困難的。因此，透過半監督學習的方法，將無相對應品質數據的過程數據透過預測模型去預測品質數據，藉此彌補數據不足及缺失的問題，便能改善軟測量模型的訓練成效。
本文提出結合生成對抗網絡的半監督變分自動編碼器 (S2-VAE/GAN)，透過解碼器、生成器和鑑別器之間的拮抗，來達到納許均衡(Nash equilibrium)狀態，從而提高解碼器與生成器在學習過程及品質數據真實分佈的能力，如此可以改善模型重構和預測質量數據的能力，並可以同時保有訓練模型時快速收斂的優點。另外，在本模型中皆透過機率分佈的形式表示，使得S2-VAE / GAN能夠捕獲化工過程的非線性特徵並更能代表化工過程中的隨機性質。
然而，為了將軟測量模型擴展到在線預測，必須考慮化工過程的動態特性。但由於大多數軟測量模型是在穩態條件下進行訓練的，因此無法學習過程的動態特性，將使得預測模型的不精準，再加上品質數據不足，進而嚴重降低了模型的訓練性能，導致擬合不足問題。因此，本文提出半監督式潛動態變分自動編碼器（semi-supervised latent dynamic VAE, S2-LDVAE），該模型可以利用所有數據，意指在序列中儘管缺少了對應的品質數據，也可以透過模型預測缺失的數據，使得所有的數據都可以得到充分的運用；並透過編碼器進行了維度的縮減，減少了雜訊及計算機運算的負荷；隨後再經由前向及反向遞歸神經網絡(recurrent neural network, RNN)去學習數據的動態特性。此外，前向RNN預測網絡不僅能夠補足缺失的品質數據，亦能提供在線的預測。
本文所提出的S2-VAE 和S2-LDVAE方法，藉由數學例子和工業案例研究，證明了其預測的結果及模型的可行性。

關鍵字

半監督學習 ； 變分自動編碼器 ； 生成對抗網絡 ； 軟測量模型 ； 靜態和動態過程

並列摘要

Soft sensor model has been a major interest for a lot of manufacturers to infer the product quality at any given time to meet customers’ specification demand. However, building a soft sensor model have a lot of major challenges, ranging from the quality of data to the process characteristics. The accuracy and effectiveness of the soft sensor model are also limited to the assumptions and structure of the model, as well as the quality of data used to train the soft sensor model. Due to the stochastic nature of process disturbance and nonlinear dynamic characteristic inherent in most chemical plants, it is quite hard to implement common soft sensor model to predict the product quality accurately. In addition, given that most chemical plants’ data contain a lot of noises with unequal length of quality data to the amount of process data, it is very difficult to train a reliable and robust soft sensor data with such limited amount of process and quality data. Therefore, semi-supervised learning is proposed to improve the training of a soft sensor model by incorporating the unused process data without quality data into the limited amount of process and quality data. Semi-supervised variational autoencoder integrated with generative adversarial network (S2-VAE/GAN) is developed to enhance the performance of the decoder/generator in learning the true distribution of both process and quality data through the competition between decoder/generator and discriminator to achieve Nash Equilibrium state. This allows the model to improve the reconstruction and prediction quality, while retaining a faster convergence training rate. Through the probability distribution format, S2-VAE/GAN is able to capture the nonlinear feature of the process and represent the stochastic nature of chemical plants.
To extend the soft sensor model to on-line prediction, it is imperative to consider the dynamic behavior of the process. But as most soft sensor is typically trained under steady state condition, the dynamic behaviors of the process are not learnt in the soft sensor model, causing the prediction to be inaccurate. The limited amount of quality data available also severely degrade the training performance of the model, which also causes under-fitting issue. Hence, this paper proposes semi-supervised latent dynamic VAE (S2-LDVAE) that can utilize all data even those with missing quality data in the sequence. To reduce computational load and the noise, dimensional reduction is applied through encoder. Then under reduced dimension, the dynamic properties are learnt through the forward and backward recurrent neural network (RNN). In case of missing quality data, the trained prediction network (with forward RNN layer) will be utilized to provide the data and re-used to provide quality estimate during on-line prediction. The effectiveness of both proposed S2-VAE and S2-LDVAE prediction results are presented by corresponding numerical and industrial case study.

並列關鍵字

Semi-supervised learning ； Variational Autoencoder ； Generative Adversarial Network ； Soft Sensor Models ； Static and Dynamic Process

參考文獻

References

Alex Lamb, Vincent Dumoulin, Aaron Courville. 2016. "Discriminative Regularization for Generative Models." arXiv:1602.03220. February 15. https://arxiv.org/abs/1602.03220.

Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, Brendan Frey. 2016. "Adversarial autoencoders." arXiv:1511.05644. May 25. https://arxiv.org/abs/1511.05644.

Anders Boesen Lindbo Larsen, Soren Kaae Sonderby, Hugo Larochelle, Ole Winther. 2016. "Autoencoding beyond pixels using a learned similarity metric." arXiv:1512.09300. February 10. https://arxiv.org/abs/1512.09300.

Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, Jason Yosinski. 2017. "Plug play generative networks: conditional iterative generation of images in latent space." arXiv:1612.00005[cs.CV]. April 12. https://arxiv.org/abs/1612.00005.