基於深層學習的生產系統動態控制

Dynamic Control of Manufacturing System – A Deep Learning Approach

周芳屹(Fang-Yi Zhou)

指導教授 ： 吳政鴻

國立臺灣大學/工學院/工業工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201701817

摘要

本研究通過利用結合深層學習與動態規劃的方法，發展一套用以預測近似最佳生產系統控制策略的預測模型。動態規劃由於受制于維度詛咒，在求解較大規模系統的最佳控制策略時往往會花費很長時間。然而，動態最佳控制策略著和系統內特征存在著一定規律性。若有一種方法可以從小規模系統的最佳控制策略中提取有用的規律，並且用來預測大規模系統的最佳控制策略，將可以克服因為利用動態規劃求解最佳策略亦或是重新建模等所花費的時間成本
在本研究中，我們考量一個考慮可靠度不確定性的三個工作站生產系統。目標是最小化所有等候線的等候成本。我們建構了由正交化的小規模系統的最佳控制策略集合與各工作站平行機台數的策略集合所組成的訓練樣本用來訓練深層神經網路。經由充分訓練過後的深層神經網路，可重複使用並且能高效的預測未來在系統參數，產能發生變化時的最佳動態控制策略。我們透過k-cv交叉驗證深層神經網路的學習效果，並且將其預測的近似最佳策略與動態規劃求解的最佳策略應用於離散事件模擬進行成本差異的驗證。結果表明，本研究所建構的動態策略預測模型可以針對新系統進行高準確率的預測且其控制策略所導致的與最佳控制策略的差異降低在一個極小的範圍內。

關鍵字

深層學習 ； 動態規劃 ； 馬可夫決策過程

並列摘要

This study presents a dynamic approach method for manufacturing systems by combing dynamic programming (DP) with deep learning. Due to the model complexity, dynamic programming cannot efficiently find optimal control policies for large systems. However, deep neuron network can now be used to predict control rules for a large scale of states. In this research, we consider a production system with reliability uncertainties and the objective is to minimize the average queue length. We construct an optimal policy space by combing an set of smaller scale systems. Then we apply the optimal policy space to train the deep neuron network as our policy predictor.
The accuracy of DNN is validated by the k-fold cross-validation (k-cv) test in a wide variety of manufacturing systems. Then, discrete simulation is used to verify the cost different between near-optimal policies from deep learning and optimal policies from dynamic programming. Our result shows the near-optimal police output by deep neuron network high degree of accuracy as optimal dynamic police and the difference in simulation results is minimal.

並列關鍵字

Deep Learning ； Dynamic Programming ； Markov decision process

參考文獻

[1] R. Ferrero, J. Rivera, and S. Shahidehpour, "A dynamic programming two-stage algorithm for long-term hydrothermal scheduling of multireservoir systems," IEEE Transactions on Power Systems, vol. 13, pp. 1534-1540, 1998.

[2] T. Volling, D. E. Akyol, K. Wittek, and T. S. Spengler, "A two-stage bid-price control for make-to-order revenue management," Computers & Operations Research, vol. 39, pp. 1021-1032, 2012.

[3] H.-S. Ahn, I. Duenyas, and R. Q. Zhang, "Optimal stochastic scheduling of a two-stage tandem queue with parallel servers," Advances in Applied Probability, vol. 31, pp. 1095-1117, 1999.

[4] C.-H. Wu, M. E. Lewis, and M. Veatch, "Dynamic allocation of reconfigurable resources ina two-stage Tandem queueing system with reliability considerations," IEEE Transactions on Automatic Control, vol. 51, pp. 309-314, 2006.

[6] S. P. Meyn, "Workload models for stochastic networks: Value functions and performance evaluation," IEEE Transactions on Automatic Control, vol. 50, pp. 1106-1122, 2005.