利用Q-learning訓練時間序列模型提高PM2.5污染預測的準確率

Improving the accuracy Rate of PM2.5 Pollution Forecast Using Q-learning training Time-series Model

常中嶺(Chung-Ling Chang)

指導教授 ： 廖世偉

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2018年)

https://doi.org/10.6342/NTU201804115

摘要

空氣污染是嚴重影響人類生命和健康的問題。儘管前人已改進了多種空氣污 染的預測模型，但準確預測空氣污染指數的能力仍然有限(limited)。時間序列預 測在許多領域發揮著重要作用，前人已經嘗試了需多許人工神經網絡，使用線 性差分整合移動平均自迴歸 (Autoregressive Integrated Moving Average , ARIMA) 模型搭配非線性神經網路 (Neural Network , NN) 模型，並假設時間序列數據在 資料時距較長且無噪音的情況下進行空氣污染時間序列預測。然而，對於真實時 間在較短且噪音較大的時間序列數據，這些方法不能保證神經網路模型的預測 誤差最小化。因此本研究從機器學習方法上提出新的改進，基於一種強化學習 (Reinforcement Learning , RL) 的學習方法來預測未來 PM2.5 的指數， 它使用強 化學習中的 Q-learning 算法，根據其狀態特徵在於神經網路模型上選擇輸入維度 和輸入之間的時間延遲，計算最佳或接近最佳策略，使用神經網絡模型來評估計 算複雜性和準確性，以達到預測誤差最小化。

關鍵字

差分整合移動平均自迴歸 ； 時間序列 ； 神經網路 ； 強化學習 ； Q- learning

並列摘要

Air pollution is a serious problem affecting human life and health. Although predecessors have improved a variety of predictive models of air pollution, the ability to accurately predict air pollution indices is still limited. Time series prediction plays an important role in many fields. Predecessors have tried more artificial neural networks, using linear autoregressive integrated moving average (ARIMA) models with nonlinear neural network (NN) models, and assuming that the time series data is predicted by the air pollution time series in the case where the data is long and no white noise. However, for real-time in short and white noisy time series data, these methods do not guarantee that the prediction error of the NN model is minimized. Therefore, this study proposes a new improvement to the machine learning method. Based on a Reinforcement Learning (RL) method to predict the future PM2.5 value, it uses the Q-learning algorithm in reinforcement learning, based on its state characteristics on the NN model. Select the input between the input dimension and the time delay, calculate the best or near optimal strategy, and use the neural network model to evaluate the computational complexity and accuracy to minimize the prediction error.

並列關鍵字

Autoregressive Integrated Moving Average ； time series ； Neural Network ； Reinforcement Learning ； Q-learning

參考文獻

Bibliography

[1] Y.-F. Xing, Y.-H. Xu, M.-H. Shi, and Y.-X. Lian, “The impact of pm2.5 on the human respiratory system,” in J. Thoracic Disease, 2016.

[2] Y. Zheng et al., “Forecasting fine-grained air quality based on big data,” in in Proc. 21st ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pp. 2267– 2276, 2015.

[3] a. B. D.J.Lary, T.Lary, “Using machine learning to estimate global pm2.5 for environmental health studies,” in Environ. Health Insights, vol. 9, pp. 41–52, 2015.

[4] S. Mahajan, L.-J. Chen, and T.-C. Tsai, “An empirical study of pm2.5 forecasting using neural network,” in Int. IEEE Conf. Ubiquitous Intell. Comput., Adv. Trusted Comput., Scalable Comput. Commun., Cloud Big Data Comput., Internet People Smart City Innov., pp. 327–333, 2017.