結合人臉辨識系統與遞迴神經網路處理成像式光體積描記訊號

Combining face recognition system with recurrent neural network to process imaging photoplethysmography signals

吳鐘晏(Chung-Yen Wu)

指導教授 ： 李世光 ； 吳文中

國立臺灣大學/工學院/應用力學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202100023

摘要

在目前的醫療系統下，家庭醫療已逐漸成為趨勢，因此家用的醫療裝置希望能同時滿足舒適度和易操作，並同時保有一定的準確度，所以非接觸式的醫療設備已漸成為主流。然而在生理參數部分，又以心率和血壓尤為重要，尤其在血壓量測方面，目前常見且成熟的商用量測方式多以脈壓袖帶做量測，不但過程不舒服，更無法提供連續的血壓波形。
光體積描記圖(Photoplethysmography， PPG)為目前醫療生理訊號中重要的一環，但對於傳統的PPG量測為以夾具夾在手指做量測，不但不夠舒適，對血液循環不佳的 老年人更有測量上的困難，然而成像式光體積描記圖(Imaging Photoplethysmography， iPPG)則是對臉部進行非接觸式量測，解決了這個問題，但卻有測量條件限制、光雜訊過大，而造成特徵點不夠明顯、波形不夠完整的問題。
本實驗設計一通用的光學架構搭配人臉辨識系統、機器學習演算法，針對成像式光體積描記圖的訊號進行訊號處理，希望能完整臉部的iPPG訊號，然後藉由臉部的iPPG訊號去推算心臟疾病的相關參數、心率甚至是血壓模型。
本實驗搭配商用的脈壓袖帶式血壓計、心電圖和手指的 PPG 訊號量測器來做本實驗系統和演算法的驗證。為了符合家庭醫療的通用性，本實驗設計在一般環境光源下做iPPG訊號擷取，先使用人臉辨識系統去做有效區域的選擇，消除人臉晃動可能會產生的誤差和剔除非皮膚區域，經由傳統訊號的預處理過後，雖然已剔除非生理訊號的頻譜範圍，但iPPG訊號的波形仍有缺陷，因此再以遞迴神經網路架(Recurrent Neural Network， RNN)搭配長短期記憶模型(Long Short-Term Memory， LSTM)的 LSTM-RNN 架構，針對iPPG訊號去做機器學習，最後針對處理過後的iPPG訊號來提取心臟疾病的相關特徵時間點，如：波峰時間間隔(CT Calculation)、波峰波谷時間間隔(Delta T Calculation)，並搭配心電圖得到連續的脈衝傳遞時間(Pulse Transit Time， PTT)，以建立適當的血壓模型。本實驗發現訓練過後的iPPG波形不但能明顯看到長時間的完整波形，在心率、特徵時間間隔上有高度相關，且在血壓模型上，也有一定的相關性。
本實驗的結果發現，在傳統的訊號處理上，沒辦法完全的顯示iPPG訊號的特徵時間點和波形，在 LSTM-RNN 的架構下進行訊號處理之後，經由驗證，心率的平均誤差為 -0.294 bpm；波峰時間間隔的平均誤差為 -0.002 秒；波峰波谷時間間隔的平均誤差為 -0.0023 秒；搭配商用心電圖所得的脈衝傳遞時間推算出的收縮壓模型的相關係數為 0.5738，且滿足英國高血壓學會的等級 C，比起其他非接觸式量測上的迴歸程度上有明顯改善，且證明 LSTM-RNN 的訓練結果是有效的，並且可以不受特定光源限制和人臉晃動的影響。本研究證明，本光學架構和其演算法，可以適用在一般家用環境下，進行心率、血壓的非接觸式量測。

關鍵字

人臉辨識系統 ； 遞迴神經網路 ； 長短期記憶模型 ； 成像式光體積描記圖(iPPG) ； 家庭醫療 ； 脈衝傳導時間法

並列摘要

Under the current medical system, home medical treatment has gradually become a trend. Therefore, home medical devices need to be comfort and user-friendly when maintain the accuracy, so contactless medical equipment will be the priority. However, for the measurement of physiological parameters, blood pressure is particularly important and difficult to be measured. The current common commercial measurement method is mostly measured by blood pressure cuff, but not only it’s uncomfortable, but also we can’t get the continuous blood pressure waveform.
Photoplethysmography (PPG) is an important physiological signal in medical. It is difficult to measure for the elderly with poor blood circulation on the traditional PPG measurement which is performed by a device put on the finger. And It is also uncomfortable. However, Imaging Photoplethysmography (iPPG) is a contactless measurement method which solves the problem, it has limited measurement conditions and excessive optical noise which result in unclear features and incomplete waveforms. In this experiment, we designed a general optical structure with face recognition system and machine learning algorithm to process the iPPG signal. Then, after the iPPG signal is processed, we hope have a full iPPG signal of the face and use it to calculate related parameters of cardiovascular disease, heart rate and blood pressure.
This experiment setup and algorithm flow are validated with commercial systems which have blood pressure, finger PPG signal and ECG. According to the family medical treatment, this experiment is designed to capture iPPG signals under the normal ambient light. First, we use the face recognition system to select the Region of Interest to eliminate errors caused by motion artifact and remove non-skin areas. We remove the spectrum range of the non-physiological signal after the preprocessing of the traditional algorithms, but the iPPG signal waveform is still defective. Therefore, we combine Recurrent Neural Network (RNN) with the Long Short-Term Memory model (LSTM) to become LSTM-RNN model which is used to process the raw iPPG signal. Finally, we can extract feature points of cardiovascular disease based on the processed iPPG signal, such as：CT Calculation, delta T Calculation. We also obtain the heart rate from the processed iPPG signal while we get continuous pulse transit time with electrocardiogram to establish an appropriate blood pressure model. The experiment found that the LSTM-RNN model fulfills the complete iPPG waveform. Then, we found that there are a high correlation in the heart rate and characteristic time interval, and a certain correlation in the blood pressure model.
The results of this experiment found that traditional signal processing is insufficient to fully display the feature time points and waveform of the iPPG signal. After signal processing under the LSTM-RNN structure, it is verified that the mean difference of heart rate is -0.294 bpm. The mean difference of the CT Calculation is -0.002 seconds, the mean difference of the Delta T Calculation is -0.0023 seconds, and the reliability of the systolic blood pressure regression model is 0.5738 which is more accurate than other contactless experiments without machine learning. The predicted systolic blood pressure matches grade C of the British Hypertension Society. It is proved that the training results of LSTM-RNN structure are effective and we do not need the specific light. This study proves that the optical architecture and machine learning algorithms can be applied to contactless measurement of heart rate and blood pressure in a general environment.

並列關鍵字

face recognition system ； recurrent neural network ； long short-term memory ； imaging photoplethysmography ； iPPG ； home medicine ； pulse transit time method

參考文獻

[1] Who.int, 'The top 10 causes of death', 2018.

[2] Mohw.gov.tw, '民國107年主要死亡原因’, 2019.

[3] Zheng, Y. L., Ding, X. R., Poon, C. C. Y., Lo, B. P. L., Zhang, H., Zhou, X. L., ... Zhang, Y. T. (2014). Unobtrusive sensing and wearable devices for health informatics. IEEE Transactions on Biomedical Engineering, 61(5), 1538-1554.

[4] Zhang, Y. T., Poon, C. C. (2013). Health informatics: Unobtrusive physiological measurement technologies. IEEE journal of biomedical and health informatics, 17(5), 893-893.

[5] Allen,J.(2007). Photoplethysmography and its application in clinical physiological measurement. Physiological measurement, 28(3), R1.