即時低成本額溫量測擴充系統-KiTemp Cam

Real-time, Low-cost Forehead Temperature Measurement Expansion System- KiTemp Cam

林佩穎(Angela Pei Ying Lim)

指導教授 ： 張時中

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202200937

摘要

在建築物入口處有效地測量人們的體溫已成為發燒檢測和預防COVID-19流行病傳播的基本做法。如何以較低的成本實現自動、無接觸和即時測量，仍然是業界所面對的挑戰。基於成本、即時性及精確度的考量，目前市面上現有的高成本、高即時以及高精確度的非接觸式額溫量測系統的售價約爲NT 70,000，測溫時間約為1s，精確度±0.2℃。低成本、低即時以及高精確度的額溫量測系統以手持免接觸額溫槍量測方式最爲普遍，市售額溫槍價格介於NT 2,000至3,000其精確度為±0.2℃。低成本、高即時且低精度的額溫量測方案以站立式自動感溫機最爲普遍，其成本介於NT2,000至3,000，即時性可達1s完成額溫量測，但精確度±0.3℃且易發生誤測人體溫度的狀況。
爲了搭配既有的終端銷售裝置新增額溫量測系統，本論文提出了一種創新的額頭溫度測量擴充系統設計，該系統取名為KiTemp Cam，具有高即時性、低成本以及人臉辨識之優勢。KiTemp Cam擴充系統成本約為NT3,000，額溫測量執行時間為100ms，精確度為±0.4℃。 KiTemp Cam相較其他現有的額溫量測系統的優點是含有人臉辨識，無法透過人手、物品等等溫度變化來觸發系統量測，避免量測非正規待測物溫度。本擴充系統之設計概念、系統架構以及功能需求規格皆由飛捷科技股份有限公司所提供。擴充系統架構的硬體由i）普通生物紅外線感測器（I.R.S.）以及ii）單個網路攝影機所組成。
針對即時低成本額溫量測擴充系統，我們的研究問題以及其相應挑戰為：
P1. 受測者額溫量測區域問題：在考量低成本網路攝影機對焦距離的情況下，如何讓受測者知道站在正確的量測區域？
C1. 在KiTemp Cam的使用介面上讓受測者知道站位與測溫處的距離是合適的量測距離必須達到高即時性。因此在估算受測者和攝影機距離必須即時計算是一大挑戰。
P2. 單眼視覺下受測者臉部特徵與辨識問題：若使用單眼視覺的條件下，僅透過偵測臉部特徵與臉部辨識如何估算人機距離？
C2. 挑戰難點為在單眼視覺僅能提供二維的影像資訊條件下，如何精準辨識人臉的特徵點來進行人臉檢測，以及選擇哪些特徵點來進行距離估測，於時間範圍內完成量測以滿足系統即時性。
P3. 臉部特徵點估算人機距離問題：如何透過受測者臉部影像中萃取出的三個特徵點，雙眼各中心以及嘴唇上緣的影像坐標來進行距離估算?
C3. 要找到特徵點與人機距離之間的關係，挑戰難點為選擇計算方法尋找特徵點座標分布與距離之間的關係，並考量不同性別之間特徵點的差異性，提升距離估測的準確性。
P4. 提升額溫量測精確度問題：本研究所使用的低成本生物紅外線感測器，其原廠規格書上標示溫度量測精度為±3℃，該精度標準並不適用於體溫量測用途。如何有效提升紅外線溫度感測器精度以符合體溫量測的使用條件？
C4. 透過實際量測發現環境溫度的改變會直接影響量測溫度與紅外線感測器的測得的原始溫度。挑戰難點則為如何透過環境溫度的條件來建立量測溫度模型，提升量測精度以符合人體體溫量測之標準。
針對以上問題，本論文提出並設計解決方案如下：
M1) 以即時距離量測搭配人機介面使受測者維持適當量測距離
爲了讓受測者可以即時確認站立在最適合量測的位置，在人機距離估算過程使用輕量化預訓練模型來達到即時性。透過人機界面引導受測者距離攝影機 40至60cm的時候， KiTemp Cam才開始進行量測動作。
M2) 單眼視覺臉部特徵選用評估
為了有效降低成本以及實現系統即時性，我們利用Google輕量的Mediapipe臉部特徵辨識並使用Dlib的68個人臉部特徵點偵測，避免量測非正規待測物溫度例如人手、發熱物品等的熱輻射來觸發系統量測，且預防漏檢測的情況發生。在考量不同個體之間的差異性，額外選用嘴唇上緣的特徵點來進行人機距離估算，以減弱對單一臉部特徵的依賴性。
M3) 新提出單眼視覺人機距離估算演算法
運用M2偵測出的受測者臉部特徵點座標，雙眼中心以及嘴唇上緣的影像坐標點所形成的虛擬倒三角形，其邊長利用點距離公式及海龍公式計算出虛擬面積，進行性別偵測之後在預先計算好的不同性別所對應虛擬倒三角形的面積關係式，估算受測者與攝影機之間的距離。
M4) 新設計依據環境溫度之額溫模型
透過實際量測並記錄出不同環境溫度下，量測溫度與實際溫度之間的線性關係，並經由線性擬合得出各環境溫度下的線性方程後，再透過曲線擬合的方法，得到環境溫度與各線性方程係數之多項式方程式。透過本研究提出的模型建立方法使該感測器之量測精度提升至±0.4℃。
本論文的研究發現和貢獻如下：
1. 本研究建立人機介面將原始量測數據以圖像化方式提供開發者判讀，並建立受測者人機互動顯示，減少受測者操作學習的時間成本。
2. 本研究在即時性和低成本的考量下，利用單眼視覺以及開源預訓練模型估算受測者與攝影機之間的距離且測距時間為5ms，同時避免溫度變化觸發系統量測造成誤測。
3. 新提出的單眼視覺人機距離估算演算法因其準確性及應用價值，提出台灣專利申請號碼NP33697【量測人機距離的方法與系統】以及美國專利申請號碼NPE-28018-AM【METHOD, SYSTEM AND COMPUTER-READABLE STORAGE MEDIUM FOR MEASURING DISTANCE】。
4. 提出普通生物紅外線感測器（I.R.S.）搭配單個網路攝影機的額溫模型，精確度由±3℃提升至±0.4℃且額溫量測執行時間為100ms。

關鍵字

量測區域 ； 人機界面 ； 單眼視覺 ； 人機距離 ； 額溫量測擴充系統 ； 網路攝 影機 ； 額溫模型 ； 環境溫度 ； 曲線擬合

並列摘要

Effective measurement of people's body temperature at building entrances has become a fundamental practice for fever detection and prevention of COVID-19 epidemic transmission. The challenge remains to achieve automatic, contactless, and real-time measurements at a low cost. Based on cost, real-time, and accuracy considerations, the existing high-cost, real-time, and high-accuracy non-contact additional temperature measurement systems on the market are priced at about NT 70,000, with a measurement time of about 1s and accuracy of ±0.2°C. Low cost, low real-time, and high accuracy of the additional temperature measurement system is the most common handheld contactless temperature gun measurement method, the price of the commercially available additional temperature gun between NT 2,000 to 3,000 and the accuracy of ± 0.2℃. The low cost, high real-time, and low accuracy of the additional temperature measurement solution is the most common standing automatic temperature sensor, the cost of which is between NT2,000 and 3,000, and the real-time can be 1s to complete the additional temperature measurement, but the accuracy is ± 0.3℃ and prone to misdetection of human body temperature.
In order to add a forehead temperature measurement system to the existing Point of Service (POS) device, this thesis proposes an innovative forehead temperature measurement extension system design named KiTemp Cam, which has the advantages of high real-time, low cost, and face recognition. The advantage of KiTemp Cam over other existing additional temperature measurement systems is that it contains face recognition, which cannot be triggered by temperature changes of human hands, objects, etc., and avoids measuring the temperature of non-conventional objects to be measured. The design concept, system architecture, and functional requirements of this expansion system are provided by Flytech Technology Co. The hardware of the expansion system architecture consists of i) a general bio-infrared sensor (I.R.S.) and ii) a single webcam.
Our research questions and their corresponding challenges for the real-time low-cost additional temperature measurement expansion system are
P1. The respondent's area of measurement: How to let the respondent know the correct area of measurement, considering the focus distance of the low-cost webcam?
C1. In the KiTemp Cam interface, the distance between the respondent and the temperature measurement area must be known to the respondent in order to achieve a high real-time measurement distance. Therefore, it is a challenge to estimate the distance between the respondent and the camera in real-time.
P2. The problem of respondent's facial features and recognition under monocular vision: How to estimate the distance between human and camera by only detecting facial features and facial recognition under the condition of monocular vision?
C2. The challenge is how to accurately identify the features of human faces for face detection, and which features to select for distance estimation, and to complete the measurement within the time frame to meet the system timeliness, given that monocular vision can only provide two-dimensional image information.
P3. Estimation of human-machine distance by facial feature points: How to estimate the distance by extracting three feature points from the face image, the center of each eye, and the upper edge of the lips?
C3. To find the relationship between feature points and distance, the challenge is to choose the calculation method to find the relationship between feature point coordinates and distance, and to consider the difference of feature points between different genders to improve the accuracy of distance estimation.
P4. Improving the accuracy of additional temperature measurement: The original specification of the low-cost bio-infrared sensor used in this study indicates that the temperature measurement accuracy is ±3°C. This accuracy standard does not apply to the body temperature measurement. How to effectively improve the accuracy of infrared temperature sensors to meet the conditions of use for a body temperature measurement?
C4. Through the actual measurement found that changes in the ambient temperature will directly affect the measured temperature and the infrared sensor measured the original temperature. The challenge is how to correct the measured temperature by the ambient temperature condition and improve the measurement accuracy to meet the standard of human body temperature measurement.
To address the above problems, the following solutions are proposed and designed in this paper.
M1) Maintain the appropriate distance for the respondent with real-time distance measurement and a human-machine interface.
In order to allow the respondent to immediately confirm that he or she is standing at the most suitable position for measurement, a lightweight pre-training model is used to achieve real-time distance estimation. The KiTemp Cam starts the measurement only when the respondent is guided 40 to 60 cm away from the camera through the UI.
M2) Monocular vision facial feature selection evaluation
In order to effectively reduce the cost and realize the real-time system, we utilize Google's lightweight Mediapipe facial feature recognition and use Dlib's 68 human facial feature points detection to avoid measuring the temperature of non-regular objects to be measured such as human hands, hot objects, etc. thermal radiation to trigger the system measurement and prevent the occurrence of missed detection. In consideration of the differences between individuals, an additional feature point on the upper lip is chosen for the estimation of the human-machine distance to reduce the dependence on a single facial feature.
M3) New proposed monocular visual human-machine distance estimation algorithm
The distance between the respondent and the camera is estimated by using the virtual inverted triangle formed by the coordinates of the facial features detected by M2, the center of both eyes and the image coordinates of the upper edge of the lips, and the edge length of the virtual area calculated by using the point distance formula and the Heron formula.
M4) New design of the ambient temperature-based IRS module accuracy improvement model
The linear relationship between the measured temperature and the actual temperature at different ambient temperatures is measured and recorded, and then the linear equation for each ambient temperature is obtained by linear fitting. The modeling method proposed in this study improves the measurement accuracy of the sensor to ±0.4°C.
The research findings and contributions of this paper are as follows：
1. This study establishes a human-machine interface to provide the raw measurement data to the developer for interpretation in a graphical way, and establishes a human-machine interaction display for the respondent to reduce the time cost for the respondent to operate and learn.
2. This study uses monocular vision and open source pre-training model to estimate the distance between the respondent and the camera with a measurement time of 5ms under the consideration of real-time and low cost and avoids temperature change to trigger the system measurement causing mismeasurement.
3. The proposed monocular vision human-machine distance estimation algorithm has been applied for Taiwan patent 【量測人機距離的方法與系統】and United States patent【METHOD, SYSTEM AND COMPUTER-READABLE STORAGE MEDIUM FOR MEASURING DISTANCE】due to its accuracy and application value. The application number of Taiwan patent is NP33697 while United States patent is NPE-28018-AM.
4. Proposed a general infrared sensor (I.R.S.) with a single camera for forehead temperature model, the accuracy is improved from ±3℃ to ±0.4℃ and the execution time of forehead temperature measurement is 100ms.

並列關鍵字

measurement area ； human-machine interface ； monocular vision ； humanmachine distance ； forehead temperature measurement expansion system ； single camera ； forehead temperature modeling ； ambient temperature ； curve fitting

參考文獻

References

[ASH18] D. F. Al-khalidi, A. Sadiq, and S. Hameed, "A survey of human face detection methods," vol. 10, 03/01 2018, doi: 10.29304/jqcm.2018.10.2.392.

[AtP79] M. Attfield and D. Peck, "Temperature self-regulation and relaxation with migraine patients and normals," Behaviour research and therapy, vol. 17, no. 6, pp. 591-595, 1979.

[Bea21] E. Beall, "Pixel luminance artifact measurement, calibration, and correction for accurate thermographic body temperature determination," in Infrared Sensors, Devices, and Applications XI, 2021, vol. 11831: International Society for Optics and Photonics, p. 118310N.

[BLC06] C. Byrne, J. K. W. Lee, S. A. N. Chew, C. L. Lim, and E. Y. M. Tan, "Continuous thermoregulatory responses to mass-participation distance running in heat," Medicine and science in sports and exercise, vol. 38, no. 5, p. 803, 2006.