深度強化式學習在指數追蹤的應用

Deep Reinforcement Learning on Index-Tracking

鍾文豪(Wen-Hao Chung)

指導教授 ： 葉小蓁

共同指導教授 ： 韓傳祥

國立臺灣大學/管理學院/財務金融學研究所/碩士(2018年)

https://doi.org/10.6342/NTU201801196

摘要

指數追蹤是一種投資組合管理，藉由建構投資組合來追蹤特定指數的績效，同時極小化追蹤偏離度及追蹤誤差。如果我們知道指數的成分股及成分股的權重或者指數的編制規則，指數追蹤的問題就變得相當容易。如果上述的資訊全部是私有訊息呢？本文提出深度強化式學習方法，在不知道指數成分股、成分股的權重及指數編製規則的情況下，建立指數追蹤投資組合追蹤該指數。本文使用深度強化式學習中的策略梯度，來建構指數追蹤投資組合。策略梯度能夠將狀態訊息轉換成連續的動作，相較於深度Q-學習，更適合用來做投資組合管理。美國股票市場的所有
普通股將作為強化式學習模型的輸入，用來追蹤股票指數（S&P500、NASDAQ Composite）及主動式基金（FSCSX、FBSOX、NASDX）。追蹤偏離度的均方是我們主要衡量指數追蹤的依據。實驗結果顯示，我們提出的深度強化式學習方法所建構的指數追蹤投資組合可以良好的追蹤標的。在樣本外測試期間，追蹤偏離度的均方至少可以達到2.71E-05 的水準。

關鍵字

指數追蹤 ； 投資組合管理 ； 強化式學習 ； 策略梯度 ； 追蹤偏離度

並列摘要

An index-tracking problem is a kind of portfolio management, building a portfolio that tracks the performance of a certain index while minimizing the tracking difference and tracking error. If constituents of the index and portfolio weights of constituents are known or rules to build the index are public information, tracking index is trivial. What if the above information is private? In this paper, we propose a deep reinforcement learning method to build an index-tracking portfolio to track the index while not knowing the constituents of the index, portfolio weights of constituents or rules to build the index. Deep reinforcement learning (RL) with Policy Gradient is deployed to build the indextracking portfolio. Policy Gradient transform state information to continuous actions which is more suitable for portfolio management than the deep Q-learning. The whole U.S. equity will be put into the deep RL model to track the indexes (S&P 500 and NASDAQ Composite) or the active funds (NASDX, FSCSX and FBSOX). Mean square difference is used as our main measurement for index-tracking. The experiment result shows that the index-tracking portfolio build by the proposed RL method could excellently track the target. The mean square of tracking difference could at least achieve 2.71E-05 in the whole testing period.

並列關鍵字

Index-tracking ； Portfolio Management ； Reinforcement Learning ； Policy Gradient ； Tracking Difference

參考文獻

Chapter 6 Reference

[1] I. Kaastra and M. S. Boyd, "Forecasting Futures Trading Volume Using Neural Network," The Journal of Futures Markets, vol. 15, no. 8, pp. 953-970, December 1995.

[2] J. Bollen, H. Mao and X.-J. Zeng, "Twitter mood predicts the stock market," Journal of Computational Science, vol. 2, no. 1, pp. 1-8, March 2011.

[3] R. Choudhry and K. Garg, "A Hybrid Machine Learning System for Stock Market Forecasting," International Scholarly and Scientific Research & Innovation, vol. 2, no. 3, pp. 689-692, 2008.

[4] Y. Lin, H. Guo and J. Hu, "An SVM-based Approach for Stock Market Trend Prediction," in Neural Networks (IJCNN), The 2013 International Joint Conference on, Dallas, TX, USA, 2014.