應用多模態深度學習於急性胸痛之成人急診去向處置預測

Multimodal Deep Learning for Prediction of the Disposition of Adults with Acute Chest Pain in the Emergency Room

林博翔(Po-Hsiang Lin)

指導教授 ： 謝哲光

義守大學/智慧科技學院/電機工程學系/博士(2022年)

摘要

深度學習在臨床醫療決策中，不論是在醫療影像、病歷文字或是臨床實驗數據資料處理上面等都已有許多成功的例子，然而這些應用大多侷限在單一模態的資料訓練。多模態深度學習藉由不同的模態資料，並使用深度學習來加以訓練，達到比過去單一模態深度學習更好的效果。
在本研究中，我們模仿醫師的臨床思考流程，並加入了多模態深度學習的機制來學習不同模態之間的關聯，如：模態表現、模態排列、模態融合。急性胸痛在急診室是個非常重要的病症之一，諸如其疾病發生率、嚴重度以及疾病診斷的即時性等都是極為重要的課題。本研究利用多模態深度學習來建立模型，模仿學習臨床醫師的臨床思考流程，並透過病歷文字記載以及心電圖檢查，即時預測病人去向的機率。本論文首度利用孿生網路來建立多模態深度學習模型並運用於急性胸痛的風險預測上面。
本研究比較不同模態之間的預測效果。多模態深度學習模型的均方誤差(MSE)和平均絕對誤差(MAE)分別為0.44和0.52，其表現優於任何單一模態的模型。因此，多模態模型可以為急診醫師對於急性胸痛的成人去向處置提供更好的決策建議。

關鍵字

多模態深度學習 ； 急性胸痛 ； 心電圖 ； 處置 ； 孿生神經網路

並列摘要

Deep learning has made great success in many clinical decision problems for data from medical images, medical texts, or clinical laboratory. However, it is mostly limited to a single modality data training. Multimodal deep learning, by using data of different modalities and deep learning usually achieves better results than those by using a single modal.
In this study, we imitate the thinking process of clinical physicians and add the mechanism of multimodal deep learning among different modalities such as representation, alignment, and fusion. Acute chest pain is one of the critical issues in the emergency room, in terms of incidence, severity, and immediacy of diagnosis. We use multimodal deep learning to build a model that mimics the clinical decision-making process of clinicians to predict the probability of disposition in time through textual documentation of medical records and electrocardiograms (ECG). This is the first time that uses a siamese neural network to build a multimodal deep learning model for the risk prediction of the disposition of adults with acute chest pain in the emergency room.
The performance comparisons are also made among various modalities. The mean square error (MSE) and mean absolute error (MAE) of the multimodal deep learning model are 0.44 and 0.52 separately, and the performance is better than any model of single modality. Consequently, multimodal models can provide emergency physicians with better suggestion of medical decisions about the disposition of adults with acute chest pain.

並列關鍵字

Multimodal deep learning ； Acute chest pain ； Electrocardiograms ； Disposition ； Siamese neural network

參考文獻

[1] O. I. Abiodun, A. Jantan, A. E. Omolara, K. V. Dada, N. A. Mohamed, and H. Arshad, "State-of-theart in artificial neural network applications: A survey," Heliyon, vol.4, issue 11, 2018. https://doi.org/10.1016/j.heliyon.2018.e00938.

[2] J. G. Hsieh, J. H. Jeng, Y. L. Lin, and Y. S. Kuo, Pathyways to Machine Learning and Soft Computing, Kaohsiung: EGBBooks, 2018, pp. 164.

[3] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.

[4] K. Hornik, M. Stinchcombe, and H. White, "Multilayer feeforward networks are universal approximators," Neural Networks, vol. 2, issue 5, pp. 359-366, 1989.

[5] A. Esteva, A. Robicquet, B. Ramsundar, V. Kuleshov, M. DePristo, K. Chou, C. Cui, G. Corrado, S. Thrun, and J. Dean, "A guide to deep learning in healthcare," Nature Medicine, vol. 25, pp. 24-29, 2019.