利用注意力機制的輕量深度學習模型進行非接觸式生理徵象偵測

Non-contact vital sign monitoring with attention-based lightweight deep learning model

蘇柏燁(Po-Yeh Shu)

指導教授 ： 李明穗

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202204210

摘要

生理徵象(vital signs)是一組表示生命現象的重要訊號，包括呼吸、心跳、血壓等內容。在新冠肺炎(Covid-19)的疫情下，遠距醫療的需求正在增加。基於影片對患者的呼吸喊心跳進行生理徵象分析，將可為遠距醫療帶來幫助。過去的研究指出從人類影片分析其呼吸及心跳是可行的。最近幾年，深度學習技術更為這項任務帶來更好的表現，但目前的深度學習方法仍有成本上的限制。訓練深度學習模型需要錄製呼吸或心跳的波型，而支援此功能的設備通常較難以取得，大部分生理徵象偵測的設備僅支援紀錄每分鐘呼吸或心跳次數。另一方面，使用深度學習進行影片處理往往需要使用較大的神經網路模型以及龐大的運算資源進行訓練，這也使的這個任務所需要的運算成本較高。本研究提出了弱監督式學習方法，讓訓練過程不再需要錄製生理訊號的波型，只要有頻率即可進行訓練。另外，結合了傳統電腦視覺演算法與深度學習，大幅降低了模型的大小及訓練過程需要的運算資源，本研究中提出的基於注意力機制的方法，可自動選擇適合偵測的區域，提升了呼吸心跳偵測的準確度。本研究使用台大醫院錄製的資料集來訓練提出的模型，並在公開的資料集上進行測試，在該公開資料集上的表現超越了目前表現最好的研究。

關鍵字

生理徵象 ； 遠程光體積描計圖法 ； 呼吸偵測 ； 電腦視覺 ； 深度學習 ； 弱監督式學習 ； 注意力機制

並列摘要

Vital signs are a group of the most important medical signs, including respiration, heart rate, and blood pressure. During the Covid-19 pandemic, the demand for remote telemedicine has increased. It would be helpful if vital signs such as heart rate or respiration rate can be detected from the patients’ videos. Previous researches show that detecting heart rate and respiration rate is feasible. In recent years, deep learning approaches have improved the performance of this task. However, there are still some limitations to this task. To train such a deep learning model, recording vital signs signals is needed, while most devices do not support this function. Most devices such as oximeters can only record the average heart rate or respiration rate. Besides, training such a video processing model costs a lot of computational resources and makes the cost of computational resources high. This research tends to solve these limitations and improve performance. The proposed weakly-supervised training methods make training a vital signs detection network more easily. Only an average heart or respiration rate label for a video is sufficient. Also, the proposed work combines traditional computer vision algorithms and deep learning. It makes the deep learning model extremely lightweight compared with current end-to-end models. The proposed channel attention architecture can wisely select a proper signal for detecting vital signs. The models are trained on a private dataset recorded by National Taiwan University Hospital and tested on a public dataset. This research has reached a better performance than current state-of-the-art works on the public dataset.

並列關鍵字

Vital signs ； Remote photoplethysmography ； Respiration rate detection ； Computer vision ； Deep learning ； Weakly-supervised learning ； Attention

參考文獻

[1] J. Allen. Photoplethysmography and its application in clinical physiological mea-

surement. Physiological measurement, 28(3):R1, 2007.

[2] A. Bulat and G. Tzimiropoulos. How far are we from solving the 2d 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In 2017 IEEE International Conference on Computer Vision (ICCV), pages 1021–1030, 2017.

[3] A. V. Challoner and C. A. Ramsay. A photoelectric plethysmograph for the measurement of cutaneous blood flow. Phys Med Biol, 19(3):317–328, May 1974.

[4] M. Chen, Q. Zhu, H. Zhang, M. Wu, and Q. Wang. Respiratory rate estimation from face videos. In 2019 IEEE EMBS International Conference on Biomedical Health Informatics (BHI), pages 1–4, 2019.