量子強化學習中的離散資料編碼與量子雜訊之優勢

在近幾年來，不論是在學術領域還是現實世界的問題解決上，機器學習已經發展得既廣且深。另一方面，為了有效地對某些問題做出歸納或預測，資料分析在各業界也愈來愈受到重視；也因為現在碰到的待解問題更加複雜等，這些以及其他因素使得我們需要更高的運算效能，在此趨勢下，量子計算受到相當大的關注。量子機器學習是結合機器學習以及量子計算兩觀念下的產物，其中，變分量子電路的方法受到高度討論。變分量子電路和古典神經網路類似，藉由調整電路中的可訓練參數來逼近未知或複雜的函數；但由於量子疊加與糾纏效應，和古典神經網路相比，變分量子電路往往只需要較少的可訓練參數。此外，也可在一個模型裡同時使用變分量子電路和古典神經網路，這樣的模型被稱為混合模型。混合模型被認為會在嘈雜量子機器的世代中被廣泛地使用；而事實上，現在已經出現了這樣的機器。在本論文中，我們首先針對量子機器學習裡一個重要的問題：資料的編碼，在深度強化學習的架構下進行研究。在這個部分，我們專注於離散資料上的編碼。此外，我們採用深度Q學習的演算法並將演算法中古典神經網路的部分替換成混合模型。我們發現運用量子隨機存取碼來編碼可以取得不錯的成效。接下來，我們將這個方法推廣到更複雜或具有隨機性的環境來進行實驗，並發現在此發法依然可行。順道一提，將混合模型運用在隨機環境下的深度強化學習在我們的認知中是一項嶄新的研究。最後，我們將噪聲模型中或是IBM量子設備中的量子雜訊嵌入至我們的模型並進行模擬，我們發現不論雜訊的來源如何，量子雜訊可以幫助深度強化學習中的代理人進行探索。

關鍵字

量子機器學習；深度強化學習；變分量子電路；量子雜訊；離散資料編碼

並列摘要

Machine Learning (ML) has been widely and deeply developed in recent years, in either academic domains or in real-world problem-solving. On the other hand, there is increasing emphasis on data analysis in many industries, in order to make effective summaries or predictions on certain issues; also, real-world problems needed to be solved have become more and more complex. These and other reasons lead to the demand for higher computing power; therefore, quantum computing draws considerable attention under this trend. The concept of quantum machine learning (QML) is the combination of ML and quantum computing. Under QML, the variational quantum circuit (VQC) architecture is highly discussed. VQCs are similar to classical neural networks (NNs), which are used as function approximators by tuning trainable parameters in the circuits, but they often need fewer parameters compared with classical NNs thanks to the quantum superposition and entanglement. Furthermore, both methods can also be combined in a model simultaneously, which is called a hybrid model. Hybrid models are considered to be popularly used in the era of noisy intermediate-scale quantum (NISQ) machines, and such devices are available now. In this thesis, we first investigate an important issue in QML, the data encoding, under the framework of deep reinforcement learning (DRL). In this part, we focus on the encoding of discrete data. Besides, we adopt the Deep Q-Learning (DQN) algorithm and replace classical NNs in DQN with hybrid models. We find that using quantum random access codes (QRACs) as the encoding methods brings effective results. Next, we generalize the architecture to environments with higher complexity, or with stochasticity, and the method is still feasible. Besides, to our knowledge dealing with stochastic environments is new in the hybrid-model DRL domain. Last, we import quantum noise from either noise models or IBM quantum devices in our simulations. We find that in either case, quantum noise can help the agent with exploration in DRL.

並列關鍵字

quantum machine learning ； deep reinforcement learning ； variational quantum circuits ； quantum noise ； discrete data encoding

參考文獻

[1] P. P. Shinde and S. Shah, “A review of machine learning and deep learning applications,” in 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), 2018, pp. 1–6.

Google Scholar

[2] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” 2013.

Google Scholar

[3] V. Mnih, K. Kavukcuoglu, and D. S. et al., “Human-level control through deep reinforcement learning,” in Nature, vol. 518, 2015, pp. 529–533.

Google Scholar

[4] S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 3389–3396.

Google Scholar

[5] D. Silver, A. Huang, and C. M. et al., “Mastering the game of go with deep neural networks and tree search,” in Nature, vol. 529, 2016, p. 484–489.

Google Scholar

國際替代計量

量子強化學習中的離散資料編碼與量子雜訊之優勢

未授權

主題瀏覽