Black-Litterman 模型結合強化學習之投資組合配置

Black-Litterman Portfolios with Reinforcement Learning Derived View

李雍群(Li Yung-Chun)

指導教授 ： 廖四郎

國立政治大學/商學院/金融學系/碩士(2022年)

https://doi.org/10.6814/NCCU202200681

摘要

本研究嘗試將強化學習 (Reinforcement Learning) 應用於預測金融資產價格走勢，並結合Black-Litterman模型建構全球性多元投資組合。本研究使用近端策略優化演算法 (Proximal Policy Optimization, PPO)，以資產價量資料預測資產價格漲跌及漲跌幅度，並將預測結果作為Black-Litterman模型中的投資者觀點進行資產配置，比較投資組合在不同獎勵設定及不同更新次數下的績效表現。本研究以美國五檔不同資產類別ETF作為基礎資產，研究結果顯示強化學習在一定更新次數上具有預測力，本研究建立之投資組合績效在更新次數600000皆能贏過其餘基準模型。另外，對於強化學習而言，以不同獎勵設定訓練模型比起增加更新次數對績效有著較大的影響。

關鍵字

投資組合 ； 強化學習 ； Black-Litterman模型 ； 近端策略優化

並列摘要

In this thesis, we try to apply reinforcement learning to forecasting price trends of finance assets. We combine the forecasts with the Black-litterman model and construct various globally diversified portfolios. In our paper, we use the Proximal Policy Optimization algorithm to forecast assets’ price trends by historical price and volume. The prediction of results are used as the investor’s views in the Black-Litterman model for asset allocation. This study compares the performance of the portfolios under different reward setting and number of updates. The empirical results show that reinforcement learning has predictive power at a certain number of updates. The portfolios performance in this study outperform the benchmark portfolios at 600,000 updates. In addition, for the reinforcement learning, training the model with different reward setting has a greater impact on performance than increasing the number of updates.

並列關鍵字

Portfolio ； Reinforcement Learning ； Black-Litterman Model ； Proximal Policy Optimization

參考文獻

[1] Black, F., & Litterman, R. (1991), “Asset Allocation: Combining Investor Views with Market Equilibrium.” The Journal of Fixed Income, 1(2), 7-18.

[2] Black, F., & Litterman, R. (1992), “Global Portfolio Optimization.” Financial Analysts Journal, 48(5), 28-43.

[3] Black, F., Jensen, M. C., & Scholes, M. (1972), “The Capital Asset Pricing Model: Some Empirical Tests.” In M. Jenson, ed., Studies in the Theory of Capital Markets (Praeger, New York, NY).

[4] Donthireddy, P. (2018), “Black-Litterman Portfolios with Machine Learning derived Views.” ResearchGate. Retired April 12, 2022, from https://www.researchgate.net/publication/326489143_Black-Litterman_Portfolios_with_Machine_Learning_derived_Views

[5] He, G., & Litterman, R. (2002), “The intuition behind Black-Litterman model portfolios.” Available at SSRN 334304.