用於失智症預測的多模態注意力網路

Multi-Modal Attention Network for Dementia Prediction

歐鎧豪(Kai-Hao Ou)

指導教授 ： 周承復

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202202423

摘要

老年人口的長期照護在高齡社會中是一個至關重要、需要被深入探索的課題。人類的身體隨著年齡逐漸增長，時常會伴隨許多內科疾患或慢性病，嚴重者可能會導致認知功能衰退或生活自理上的困難；如此一來，將造成龐大的照顧負擔與醫療支出。其中，失智症（dementia）是造成認知功能障礙或喪失生活自理能力的重點疾患之一。
面對這樣棘手的問題，民眾會想要把風險轉嫁予保險公司；而因應這樣的需求，保險公司需要推出一些相應的商品。有鑒於此，本篇論文結合人工智慧與深度學習技術，提出了一個具備高可用性與高彈性的疾病預測模型架構；透過輸入一位客戶的個人資訊與其疾病歷史，來獲得一個代表該位客戶未來一段時間內罹患失智症之風險高低的數值，藉此來輔助保費的試算。
在機器學習的相關研究中，大量的資料是其中不可或缺的重要環節。本篇研究所使用的資料係由臺灣衛生福利部所提供之全民健康保險資料庫，該資料庫中包含了個人基本資料（如：出生年、性別）以及病患的歷史病程資訊，可以提供研究機構專案申請使用。
本論文主要探討的問題是利用患者的歷史就醫紀錄，來推測其未來可能會罹患失智症的機會高低。若從另一個角度來看待這個問題，相當類似於一般推薦系統中常見的點擊率（Click-Through Rate，CTR）預測問題。在本論文中其中一些模組設計係由點擊率預測相關研究獲得啟發，而我們也將在論文中的實驗部分應證點擊率預測模型應用於疾病預測的可行性。
在我們的論文內所提出的模型架構主要可以切分為兩大部分，分別是處理患者個人資訊的部分以及處理患者病史資訊的部分。在處理病患病程的部分，我們整合了許多當代著名的經典方法，像是word2vec embedding、Multi-Head Self-Attention等手法，讓疾病之間的高階交互特徵得以被有效地汲取。此外，在我們精巧的架構設計下，使得預測模型具有更高的可用性與彈性，並使遷移學習變得可行。
在實驗的部分中，我們進一步對模型架構中的各個模組進行測試與觀察，分別探討了像是模型輸出層的寬度與深度、word2vec embedding在資料編碼上的視覺化呈現……等。此外，在我們的實驗結果中證實了，將word2vec embedding技巧應用在ICD碼的轉換上，能夠大幅提升模型建置過程的穩定性，並且能夠有效地提升模型的預測性能表現。
接著，我們會對預測模型進行可解釋性的剖析，嘗試找出對於疾病預測模型的性能有較顯著影響的個人資料特徵，並利用注意力分數（Attention Score）來找出在患者就醫紀錄中，哪個時期或哪些疾病對於預測模型而言具有相對較為重要的影響能力。而前述所發現之重要特徵或疾病結果，皆與原始資料內的相關統計數值一致，並與相關醫學文獻的結論不謀而合。

關鍵字

ICD碼嵌入 ； 失智症 ； 疾病發生率預測 ； 注意力網路 ； 機器學習

並列摘要

Long-term care of the elderly population is an important issue that needs to be focused on in an aging society. As the human body ages, many internal or chronic diseases are often accompanied. In severe cases, it may lead to cognitive decline or difficulties in self-care. As a result, it will cause a considerable burden of care and medical expenses. Dementia is one of the critical diseases that cause cognitive dysfunction or loss of self-care ability.
Many would want to pass on the risk to the insurance company for such a severe problem. Insurance companies need to develop some related products. This paper will use machine learning techniques to build a flexible disease prediction model architecture. Feeding a consumer's personal information and disease history into the predictive model will output a decimal value representing the customer's future risk of dementia, which can be used to help calculate the insurance price.
A large amount of data in machine learning research is an essential element. The data used in this study were the National Health Insurance database provided by the Taiwan Ministry of Health and Welfare, which contains basic personal information (e.g., year of birth, gender) and medical records (represented by the International Classification of Diseases code). Research institutions can apply for permission to use the database after an academic ethics review.
The major problem of this paper is using the patient's personal information and medical records to evaluate the risk of dementia in the future. On the other hand, this problem is quite similar to the click-through rate (CTR) prediction problem in general recommender systems. In this paper, some of the module designs are inspired by CTR prediction research, and we will also demonstrate the feasibility of the CTR prediction model in disease prediction in the experimental part of the paper.
The model architecture proposed in our paper can be divided into two parts: the part that deals with patients' personal information and the part that deals with patients' medical history. In the part that deals with patients' medical history, we integrate many famous and classic methods, such as Word2vec embedding, multi-head self-attention, and other techniques, so that the high-order interactive features between diseases can be effectively extracted. In addition, according to the proposed model architecture, the prediction model has higher usability and makes transfer learning practical.
In the experiment, we conducted various tests and discussions on each module in the model architecture, such as the width and depth of the proposed model's output layer and the visual presentation of Word2vec embedding on data encoding. In addition, we further use the experimental results to show that applying the Word2vec embedding technique to the dimensionality reduction of ICD codes can significantly improve the model construction process's stability and prediction performance.
We will then conduct the explainable analyses of our proposed model. We will first attempt to identify personal data features that significantly affect disease prediction models' performance. We then used the attention score to determine which period or disease in the patient's history significantly impacted the prediction result. Finally, we found that those major features or diseases identified in the experimental results were consistent with the statistical values of the raw data and the conclusions of the relevant medical literature.

並列關鍵字

ICD Code Embedding ； Dementia ； Disease Incidence Prediction ； Attention Network ； Machine Learning

參考文獻

[1] 國家發展委員會, 中華民國人口推估（2020年至2070年）. 國家發展委員會 (in 繁體中文), 2020.

[2] E. B. Larson, "Evaluation of cognitive impairment and dementia," in UpToDate. Waltham, MA., 2019.

[3] M. S. Beeri et al., "Age, gender, and education norms on the CERAD neuropsychological battery in the oldest old," Neurology, vol. 67, no. 6, p. 1006, 2006, doi: 10.1212/01.wnl.0000237548.15734.cd.

[4] E. Smith and C. B. Wright, "Etiology, clinical manifestations, and diagnosis of vascular dementia," in UpToDate. Waltham, MA., 2020.

[5] "Global, regional, and national burden of Alzheimer's disease and other dementias, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016," (in eng), Lancet Neurol, vol. 18, no. 1, pp. 88-106, Jan 2019, doi: 10.1016/s1474-4422(18)30403-4.