增強式學習建構臺灣股價指數期貨之交易策略

Reinforcement Learning to Construct TAIFX Trading Strategies

洪子軒(Hong Tzu-Hsuan)

指導教授 ： 林士貴 ； 蔡瑞煌

國立政治大學/商學院/金融學系/碩士(2018年)

https://doi.org/10.6814/THE.NCCU.MB.023.2018.F06

摘要

機器學習與人工智慧的技術能夠應於金融交易之決策，並獲得創新的交易策略，本研究則希望發掘增強式學習法應用於金融交易之決策領域之可能。增強式學習法利用建構學習代理人(RL-agent)與環境交流的方式，具有自主學習策略並優化的能力，其所擁有的環境探索(Exploitation)及延遲報酬(Delayed Reward)兩項特性，與應用於金融市場的交易策略建構之問題不謀而合，因此本研究採用增強式學習法來建立臺灣股價指數期貨的交易策略。在研究的設計上，我們嘗試了三種不同的實驗設計方式、採用 Q-learning、SARSA以及DQN 三種不同的演算法進行討論。我們將 2007 年 7 月 1 日至 2017 年 12 月 31 日之臺灣股價指數期貨歷史資料設定為研究之標的，並在此區間訓練模型並分析績效表現。透過實證結果發現，在合理的實驗設計下，學習代理人能通過增強式學習模型建構出得超越大盤並穩定獲利之交易策略。

關鍵字

演算法交易 ； 臺股期貨 ； 機器學習 ； 增強式學習法 ； SARSA ； Q-Learning ； DQN

並列摘要

Reinforcement Learning features the self-learning ability on strategy construction and optimization by forming the way in which RL-agent interact with environment. Two characteristics of reinforcement learning, interacting with environment and delayed reward, can be applied on decision control system, such as constructing trading strategy. Therefore, this research is to build the trading strategy on TWSE futures index by adopting reinforcement learning. In terms of system design, we examine three kinds of situation definition and algorithm, including Q-learning, SARSA and DQN. To test the availability, this article utilizes TWSE futures historical data (2007/7/1-2017/12/31) to conduct learning training and performance examination. Our findings illustrate that RL-agent would be able to construct the trading strategy which defeats the market and make profits steadily if environment is effectively defined. Moreover, the results conclude that machine learning and artificial intelligence are in favor of decisions on financial trading and pioneering trading strategy creation.

並列關鍵字

Algorithm trading ； Taiwan stock index future ； Machine learning ； SARSA ； Q-Learning ； DQN ； Reinforcement learning

參考文獻

[1] Bekiros S. D. (2010), Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach, Journal of Economic Dynamics & Control,34 (6), 1153-1170.

[2] Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003.

[3] Fama, E. F., (1970). Efficient Capital Markets: A Review of Theory and Empirical Work. The Journal of Finance, 25(2), 383-417.

[4] Gold, C. (2003). FX trading via recurrent Reinforcement Learning, Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, 363-370.

[5] Irwin, S. H. and Park, C. H., (2007). What Do We Know About the Profitability of Technical Analysis? Journal of Economic Surveys, 21(4), 786–826.