利用深度強化式學習建構價差交易策略:以台指期與摩台期為例

The Construction of TX-TW Pair Trading Strategies based on Deep Reinforcement Learning

許晏寧(Hsu Yen-Ning)

指導教授 ： 江彌修

國立政治大學/商學院/金融學系/碩士(2019年)

https://doi.org/10.6814/NCCU201900104

摘要

本研究使用三種基於模型的深度強化式學習DQN、Double DQN和Dueling DQN來建構價差交易策略，本研究會選擇開發此類交易策略，主要是因為深度強化式學習的獎勵機制和建構交易策略有很好的對應性且價差交易策略能夠有效的減少市場風險。本研究採用2006/01/01至2018/11/16的台股期貨和摩台期貨進行回測，並設計隨機策略、固定策略當作基準策略，實證結果發現深度強化式學習均可以獲得比基準策略更好的表現，而整體上DQN表現勝過Double DQN和Dueling DQN。但細看可以發現，在不同的回測期間，三種深度強化式學習分別有其表現最好的時候，代表此三種模型分別學到不一樣的規則，此規則在不同的時期有不一樣的適用性。

關鍵字

價差交易 ； 強化式學習 ； 類神經網路 ； 台股期貨 ； 摩台期貨

並列摘要

In this paper, we implement three model-based reinforcement learning algorithms with deep learning, Deep Q-Learning Network (DQN), Double Deep Q-learning Network (Double DQN) and Dueling Deep Q-Learning Network (Dueling DQN) in pair trading strategy. In addition, deep reinforcement learning (DRL) has appealing theoretical properties which are hopefully potential since the reward mechanism in DRL with pair trading rules is able to significantly reduce the market risk. We conduct experiments in TX and TW historical data (2006/01/01-2018/11-16) and design the random strategy and fixed strategy to be the benchmark. The empirical results show that three DRL strategies can achieve better performance than the benchmark strategies overall and DQN is more desirable than Double DQN and Dueling DQN. However, during different back-testing period, we observe that they have the best performance respectively. It means that three models learn different rules separately and the rules have different applicability in different periods.

並列關鍵字

Pairs trading ； Reinforcement learning ； Neural network ； Taiwan Stock Index Futures ； MSCI Taiwan Index Futures

參考文獻

[1] Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003.

[2] Binh H. D. & Robert W. F. (2012). Are Pairs Trading Profits Robust to Trading Costs? The Journal of Financial Research, 35(2), 261-287.

[3] Chien Y. H. (2018). Financial Trading as a Game: A Deep Reinforcement Learning Approach. arXiv preprint arXiv:1807.02787

[4] Evan G. , William N. G., & K. G. R. (2006). Pairs Trading: Performance of a Relative-Value Arbitrage Rule. The Review of Financial Studies, 19(3), 797-827.

[5] Gold C. (2003), FX trading via recurrent Reinforcement Learning, Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, 363-370.