多任務高斯過程在化工製程建模的應用

Multi-task Gaussian process for chemical process modeling

郭晉廷(Jin-Ting Guo)

指導教授 ： 姚遠

國立清華大學/工學院/化學工程學系/碩士(2020年)

摘要

本文探討如何利用遷移學習建模，解決工業上常見數據量不足的問題，採用的機器學習模型為高斯過程，藉由修改高斯過程中的核矩陣(kernel matrix)使原本只適用於單個任務的模型可以延伸至多個任務的處理，修改過後的多任務高斯過程可以將多個任務之間數據點的資訊量藉由模型參數訓練的過程自動決定各個任務之間需要共享的量是多少。
在化學工業製程中我們常需要讓各項變數維持在一個我們所希望的範圍當中，如何能準確的預測變數趨勢在化工廠的控制中相當重要，隨著製程越來越複雜，各種單元的變數不僅個數多，彼此間的關係也漸趨複雜，在實際的工廠數據中往往難以清晰地指出各種變數間的物理關係，而數據驅動模型正好可以在不明白變數間複雜關係的情況下，利用機器學習演算法去提取工業上龐大數據中隱含的關係，並輔助建立製程預測的軟儀表模型，有了基於數據所產生的軟儀表模型再進一步搭配製程物理理論所推論而得的模型，期望能準確地預測出製程的動態行為。
在化學工業中因為同一種化學品會依照不同的品質標準而有不同的應用，像是高分子化學品依照高分子鏈結的程度有不同的使用方式，所以多規格的製程生產方式非常常見，所謂多規格製程就是同一個製程生產線在不同的產品規格中做切換，不同規格有不同的製程參數設定，雖然製程參數設定不同但是基本上用的是同一個化工單元，製程變數的種類以及個數也都相同，通常多規格製程的特色是大部分規格中的數據量都很少，因為這些規格的數據有相同的製程資訊，若單獨建模會因為數據量缺少使模型預測效果不好，所以我們希望設計一個模型可以同時對各個規格的數據建模，這個多任務模型有機制可以使各個任務之間的數據分享訊息，我們期望藉由多任務的模型從各個任務中提取出相似的製程資訊以補足單個任務製程資訊缺少的問題。
高斯過程是一種基於貝葉斯定理(Bayes’ theorem)的機率模型，它的好處是在假設數據機率分布的前提下可以建立出目標數據(target data)可能的數據機率分布，而非單純訓練出一個預測的目標數據，因為結果是一個機率分布，我們可以藉由信賴區間分析出各數據有可能出現的範圍，而且相較於深度學習的神經網絡需要大量的參數訓練，高斯過程的訓練參數遠小於神經網絡因此不需要大量疊代訓練，高斯過程另一項特色在於核函數(kernel function)的選擇，選定核函數的形式就如同選定模型會有怎麼樣的表現形式，核函數通常依據各個任務數據的特性來做選擇。
本次使用兩種化工數據，第一種是螺桿元件數據，分析不同元件排列組合的結果，第二部分使用多規格CSTR數據，藉由上述兩種化工製程的數據集來探討兩種在多規格製程上常見的問題，第一種是當某些任務有充足數據的情況下是否可以藉由多任務模型將充足的製程資訊遷移至其他少量數據的任務中，第二種則是當各個具有關聯的製程之間數據量偏少的情況下是否可以藉由多任務模型將任務之間數據的資訊區分出哪些是互相關聯而哪些又是不相關，共享相關聯的資訊以彌補單個任務資訊量偏少使模型沒辦法有效訓練參數的缺點來解決化工製程小量數據建模的問題。

關鍵字

高斯過程 ； 遷移學習 ； 多任務學習 ； 多品質製程

並列摘要

This thesis discusses how to use transfer learning to solve the problem of insufficient data in process modeling. The machine learning model we adopt herein is Gaussian process. By modifying the kernel matrix in the conventional Gaussian process, the model original only applicable to a single task can be extended to the processing of multiple tasks. The modified multi-task Gaussian process can automatically determine the amount of information to share between tasks by parameter training.
In chemical industry processes, we often need to keep process variables in control. How to predict variable trends accurately is an important issue . As the process becomes more complex, the number of operation units is larger, while the relation between units become difficult to handle. In this situation, the data-driven methods that use machine learning algorithms to extract the relation between huge data without deep understanding of the physical relationship between variables can assist the establishment of the prediction models.
In the chemical industry, the products have different applications according to the different quality standards. Taking polymers as an example, there are different grades in a polymer manufacturing process according to the degree of polymerization. As a result, multi-grade processes are common in industry, which often switch the settings of process variables during operating. Although the settings are often changed, the same operation unit is used, while the process variables to manipulated or recorded are also same.
The characteristic of multi-grade process is that the amount of data collected in some grades is often small and the process information is unbalanced between different grades. Therefore, we want to design a model which can model different grades at the same time by sharing information between them.
Gaussian process is a probabilistic model based on the Bayes’ theorem which has the advantage of establishing a probability distribution of the predicted target. Gaussain process models allow to choose the kernel functions,providing a significant flexibility in different modeling tasks.
The proposed method was illustrated with two different datasets. The first one is about a simulated twin-screw extruder and the second one contain the data of a multi-grade CSTR process. Based on the results of the case studies,the following two issues were discussed. The first issue to discuss is whether sufficient information of auxiliary task data can be migrated to assist the modeling of the target task which only contains a small amount of data. The second problem to answer is whether the multi-task model can identifyand quantify the similarity between tasks.

並列關鍵字

Gaussian process ； transfer learning ； multi-task learning ； multi-grade process

參考文獻

[1] Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)

[2] W. Yan, D. Tang, Y. Lin. A data-driven soft sensor modeling method based on deep learning and its application. IEEE Transactions on Industrial Electronics. DOI: 10.1109/TIE.2016.2622668. (2016)

[3] Huang, Z. & Pan, Z. & Lei, B. Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data. Remote Sens. (2017)

[4] E. Bonilla, K. M. Chai, & C. Williams. Multi-task gaussian process prediction. Proceedings of the 20th Annual Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, pp. 153–160. (2008)

[5] Evgeniou, T., Pontil, M. Regularized multi-task learning. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Washington, USA 109–117 (2004)