基於強化學習和基底核-丘腦動態網路之帕金森氏症閉迴路深腦電刺激演算法

Closed-loop Deep Brain Stimulation Algorithm for Parkinson's Disease based on Reinforcement Learning and Basal Ganglia-Thalamus Network Dynamics

卓嘉虹(Chia-Hung Cho)

指導教授 ： 林啟萬

國立臺灣大學/工學院/醫學工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202200118

摘要

帕金森氏症 (Parkinson’s disease, PD) 是一種影響中樞神經系統的慢性神經 退化性疾病，目前影響全球約一千萬人 [1]。深腦電刺激 (deep brain stimulation, DBS) 的技術在運動障礙和神經系統疾病中的應用，包括 PD、震顫、肌張力障礙、癲癇、強迫症等，已被證明是一種有效的治療方式 [2]。然而，廣泛使用的 開迴路系統仍然存在一些尚待修正的缺點，例如它們的個體依賴性、能量消耗 程度、頻繁回診和試錯性調整的特徵 [3]。閉迴路的策略採用具有判別性的訊 號或生物標誌物，從而使系統能夠透過算法自動調整 DBS 參數 [3]。我們設計強化學習 (reinforcement learning, RL) 與 Gym 框架，模擬基底神經節-丘腦 (basal ganglia-thalamic, BGT) 大腦網路作為訓練環境，並為任何輸入狀態找到適當的刺激參數(頻率和振幅)。特徵提取模塊則作為 BGT 大腦網路(動作電位訊號)與來自真實大腦的胞外訊號之間的映射工具，進而允許未來的動物實驗和臨床試 驗的測試。結果顯示，基於RL的DBS控制策略在能秏上較開迴路系統節省了 68.81% 的平均功率，並修正丘腦(thalamus, TH)中的錯誤響應(平均錯誤響應在正常情況為 0.0; 在PD下修正回 0.0258)，同時為未來應用奠定了基礎。

關鍵字

基底核-丘腦網路 ； 閉迴路深腦電刺激 ； 帕金森氏症 ； 強化學習 ； 獎勵塑造

並列摘要

Parkinson’s disease (PD) is a chronic neurodegenerative disease affecting the central nervous system and currently influencing about 10 million people worldwide [1]. The usage of deep brain stimulation (DBS) technology in movement disorders and neurological diseases, including PD, tremor, dystonia, epilepsy, obsessive-compulsive disorder (OCD), etc., has proven to be an effective treatment modality [2]. However, general open-loop systems pose several shortcomings that have yet to be revised, such as their subject dependency, energy-consuming, frequent-clinic visiting, and trial-and-error adjusting features [3]. The closed-loop strategy employs discriminative signals/biomarkers to enable the system to tune parameters automatically through the designed algorithms [3]. We designed reinforcement learning (RL) with the Gym framework that models the basal ganglia-thalamic (BGT) brain network as a training environment and finds appropriate stimulation parameters (frequency and amplitude) for different input states. The feature extraction module was a mapping tool between the BGT brain network (AP signals) and extracellular signals from real brains, permitting future animal experiments and clinical trials. Results showed that the RL-based DBS control strategy significantly outperforms open-loop systems in energy efficiency, i.e., conserving 68.81% of average power dissipation, and revises error responses in the thalamus (i.e., an average EI of 0.0 in normal and 0.0258 in PD states) while establishing a foundation for future application.

並列關鍵字

basal ganglia-thalamic (BGT) network ； closed-loop deep brain stimulation (cl-DBS) ； Parkinson’s disease (PD) ； reinforcement learning (RL) ； reward shaping

參考文獻

[1] Parkinson’s disease foundation. Available at: https://www.parkinson.org/ Understanding-Parkinsons/Statistics. Accessed 2022-02-10.

[2] A Amon and F Alesch. Systems for deep brain stimulation: review of technical features. Journal of Neural Transmission, 124(9):1083–1091, 2017.

[3] Mahboubeh Parastarfeizabadi and Abbas Z Kouzani. Advances in closed-loop deep brain stimulation devices. Journal of neuroengineering and rehabilitation, 14(1):1– 20, 2017.

[4] Michael S Okun. Deep-brain stimulation for parkinson’s disease. New England Journal of Medicine, 367(16):1529–1538, 2012.

[5] Chia-Chi Hsieh and Ming-Dou Ker. Design of multi-channel monopolar biphasic stimulator for implantable biomedical applications. In 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS), pages 1–4. IEEE, 2018.