睡眠事件之偵測與瀏覽

On Detection and Browsing of Sleep Events

陳昭伶(Chao-Ling Chen)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/博士(2016年)

https://doi.org/10.6342/NTU201601713

摘要

本研究的目的，在提供睡眠自我檢測的客觀量測工具，並提出應用於蓋被情境的睡姿辨識方法。醫學上的睡眠多項生理檢查，需要使用接觸式的設備，取得生理訊號資訊；然而接觸式的設備可能影響睡眠，且檢查的費用昂貴。本研究設計非接觸式的睡眠瀏覽系統，採用普及的深度攝影機設備，方便於居家使用。目前採用影像處理的睡姿辨識方法，在未蓋被情況下進行睡姿分類，無法應用於真實睡眠情境。使用深度影像的三維訊息，本研究實現在蓋被情況下進行睡姿辨識的方法。
本研究提出時間區間的方法用於偵測睡眠事件，基於深度資訊的睡姿辨識方法用於分類睡姿。睡眠瀏覽系統採用具有多重感測器的裝置，偵測床上的使用者及周遭環境中發生的睡眠事件。裝置包含紅外線深度感測器、彩色攝影機及麥克風陣列，分別偵測三種睡眠事件：動作事件、光照事件及聲音事件。由輸入的深度訊號流及彩色影像流，系統建立背景模型，偵測動作變化及光照變化，並同步量化三種訊號。當訊號分數超過各別的實驗門檻值，即觸發時間區間的方法，紀錄各別的睡眠事件，紀錄的內容包含深度影像、彩色影像及聲音檔案。系統提供睡眠歷程的瀏覽介面，呈現睡眠事件的分數曲線及整合後的影音檔案。睡姿偵測的方法分類四種睡姿類別：左側睡、右側睡、平躺及趴睡；其中左側睡及右側睡包含胎兒型、思念型及木頭型睡姿，平躺包含軍人型及海星型睡姿，趴睡包含木頭型和自由落體型睡姿。在實驗前測階段，為了計算床平面，將記錄空床的深度影像轉換為世界座標系。將每張記錄使用者的深度影像轉換為世界座標系，計算每個深度點至床平面的距離。將每張深度影像的距離陣列做為輸入，採用支持向量機的機器學習方法，實現睡姿分類。實驗模擬真實睡眠情境，在三種條件下進行，沒蓋被狀態、蓋薄被狀態及蓋厚被狀態。
本研究結果顯示：（1）睡眠瀏覽系統具有效率及可靠性，使用者透過本系統快速瀏覽睡眠事件紀錄，並經由觀看影片檢視睡眠事件的內容。（2）睡姿辨識方法在厚被的條件下，比未蓋被和薄毯的條件下有更好的辨識結果，因為厚被的厚度增強睡姿動作的特徵。本研究的發現可作為未來在居家情境中，進行睡眠事件偵測及睡姿辨識研究的指南。

關鍵字

深度影像 ； 影像序列分析 ； 睡眠瀏覽 ； 非接觸式睡眠事件偵測 ； 睡姿辨識

並列摘要

The purpose of the study was to provide an objective measurement tool for sleep self-examination, and to propose a method for sleep posture recognition of a subject under covering. Polysomnography (PSG) in clinical therapy requires attached devices to obtain bioinformation; however, the attached devices may result in uncomfortable sleeping, and the cost of the examination is expensive. In the study, using a common depth camera device, an unconstrained sleep browsing system has been developed for applying to home scenario. Current methods based on image processing technique in sleep posture recognition, classified sleep postures in the condition of a subject without covering, and were unable to apply to real sleep scenario. Using three-dimensional information of depth image, a method of sleep posture recognition was realized in the condition of a subject under covering.
In the study, epoch method was proposed for recording sleep events, and a method of sleep posture recognition based on depth image was proposed for classifying sleep postures. Using a device with multiple sensors, the sleep browsing system detected sleep events from a subject in bed and the surrounding environment. Based on the multiple sensors of the device, including an infrared depth camera, a color camera, and a four-microphone array, three types of sleep events were detected: motion event, lighting event and sound event. From the input of depth image stream and the input of color image stream, background modeling in the system was used to measure body movements and lighting changes, and the three types of signals were quantified simultaneously. When type of signal score was greater than each empirical threshold, the epoch method was triggered for recording independent sleep events, and the recording contained depth images, color images and audio files. The system provided a browsing interface with sleep diagram, presenting the score curves of sleep events and integrated videos. The method of sleep posture recognition classified sleep postures into four classes: left side, right side, supine and stomach, in which left side and right side contained fetus, yearner and log types of sleep postures, supine contained soldier and starfish types of sleep postures, and stomach contained log and freefaller types of sleep postures. In preliminary stage, the depth image capturing an empty bed was transformed into world coordinate for calculating the bed plane. Each depth image capturing a subject was transformed into world coordinate, and the vertical distance of each depth pixel to the bed plane was calculated. From the input distance array of each depth image, the Support Vector Machine (SVM) method was adopted for classifying sleep postures. Experiments for simulating real sleep scenario with three conditions were carried out: without covering condition, blanket covering condition and quilt covering condition.
The survey concluded: (1) The sleep browsing system had efficiency and reliability that users browsed the recording of sleep events efficiently, and examined the content of sleep events by watching the videos. (2) The method of sleep posture recognition had a better performance in quilt covering condition than without covering condition and blanket covering condition, because the layer of quilt enhances the features of sleep postures. The study findings may serve as a guide for future research on sleep event detection and sleep posture recognition in home scenario.

並列關鍵字

Depth image ； image sequence analysis ； sleep browsing ； unconstrained sleep event detection ； sleep posture recognition

參考文獻

[17] J. Du, C.E. Postma, H.C. Van Vugt and E. Zwartkruis, “Understanding the user needs for sleep enhancement,” Philips Internal Report, 2008.

[1] I. Szollosi, T. Roebuck, B. Thompson and M. T. Naughton, “Lateral sleeping position reduces severity of central sleep apnea / cheyne-stokes respiration,” Sleep: Journal of Sleep Research & Sleep Medicine, 29(8), 2006.

[2] N. A. Eiseman, M. B. Westover, J. M. Ellenbogen and M. T. Bianchi, “The impact of body posture and sleep stages on sleep apnea severity in adults,” Journal of Clinical Sleep Medicine, 8(6), 2012, pp. 655-666.

[3] R. D. Cartwright, “Effect of sleep position on sleep apnea severity,” Sleep: Journal of Sleep Research & Sleep Medicine, 7(2), 1984, pp. 110-114.

[5] R. S. T. Leung, M. E. Bowman, J. D. Parker, G. E. Newton and T. D. Bradley, “Avoidance of the left lateral decubitus position during sleep in patients with heart failure: relationship to cardiac size and function,” Journal of the American College of Cardiology, 4(2), 2003, pp. 227-230.