即時視線感知晶片與其家電控制系統之應用

A Real-time Eyesight Awareness Chip for Home Appliance Control

宋柏翰(Po-Han Song)

指導教授 ： 陳朝烈

共同指導教授 ： 王文壽(Wein-So Wang)

崑山科技大學/工程學院/電子工程研究所/碩士(2009年)

https://doi.org/10.6828/KSU.2009.00050

摘要

我們提出一種適用於動態影像之單一使用者即時視線偵測演算法，並實現其矽智財(SIP) FPGA雛型，以朝向複合式人機介面單晶片的目標邁進。
本系統主要使用視線來對機器(例如：家電)做選擇控制，只要看著我們欲操作的家電設備，即可切換設備控制對象，對該家電進行控制。傳統的臉部辨視，大都使用複雜的演算法來計算其臉部特徵，進而找出眼睛。傳統的做法，在硬體實現上，既費時又佔空間。本系統最大的特點在於。利用人類在視線轉移時會眨眼的特性，來簡化演算法的複雜度。在硬體實現上，我們提出同步於CCD像素輸入的緩衝技術，因此不必使用任何外部記憶體。從CCD攝影機輸入的YCbCr影像，利用邊緣偵測演算法，及前後張影像差分運算，並利用人類眨眼時臉部上半部呈現瞳孔與膚色切換的原理，來判斷畫面中眼睛的位置。當選擇控制對象時，以直視前方時所抓取到的眼睛位置為參考點，以相對位置來換算視線角度，當視線範圍內有其他家電時，便可切換設備控制對象。
為了驗證本系統的功能與效能，我們結合手勢辨識系統進行家電控制之應用，使用者不需配戴或手持任何裝置，以CCD攝影機搭配人類最自然的動作--眨眼、揮手，來達到控制設備的目的。從整合後的系統效能來看，我們使用現場可程式化邏輯閘陣列(Field Programmable Gate Array; FPGA)成功實現了視線辨識系統的矽智財，並且可以和手勢辨識系統整合成複合式人機介面單晶片系統。

關鍵字

複合式人機介面 ； 視線偵測 ； 家電控制 ； 矽智財 ； 系統單晶片

並列摘要

This paper proposes a low-cost real-time eyesight recognition algorithm for single user and its Silicon Intellectual Property (SIP) prototype in FPGA for single-chip integration of multimodal man-machine interfaces. This system mainly uses human’s eyesight to switch among machines such as home appliances. The eyesight is defined as the ability to switch from seeing one target to another. The system recognizes change of line of sights and incooperate with controller to control the target appliance. Traditional face detection and recognition methods require complex algorithms to calculate facial features and then find the eyes. Therefore, it is not practical to implement these algorithms into hardware integrated circuits because they are time-consuming and requires too much chip area. We exploit the nature that human eye blinks when line of sight changes to simplify complexity of algorithms.
For hardware realization, we propose a buffering technique which is synchronous to pixel input from CCD. Therefore, we avoid external memory to save circuit area and power consumptions caused by large amount of memory accesses. Regarding Cb-Y-Cr pixel input information from the CCD camera, we use Sobel operator to detect the edge of the input image, use filtered image differences to find the moving area of the image, and detect eyes blinks by calculating chrominance change to find the location of the eyes.
In order to verify the system's function and performance, we combine a gesture recognition system for home appliance control as an application example. Users need neither any remote control nor any wearable device. With single CCD camara, a user use the most nature blinks and hand movement to control the home appliances. We successfully implement the eyesight detection system into an SIP prototype with the Field Programmable Gate Array (FPGA). Gesture recognition can be integrated becoming a multimodal man-machine interface of a single-chip.

並列關鍵字

multimodal man-machine interface ； eyesight detection ； home appliance remote control ； SIP prototyping ； SoC

參考文獻

[2] Liming Zhang, Patrick Lenders, “Knowledge-Based Eye Detection for Human Face Recognition,” Fourth International Conference on knowledge-Based Intelligent Engineering System & Allied Technologies, 30th Aug-1st Sept 2000, Brighton, UK

[3] Duy Nguyen, David Halupka, “Real-Time Face Detection and Lip Feature Extraction Using Field-Programmable Gate Arrays,” IEEE Transactions on System, Man, and Cybernetics—Part B: Cybernetics, Vol. 36, No. 4, August 2006

[5] Roberto Brunelli, Tomaso Poggio, “Face Recognition: Features versus Templates,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, No. 10, October 1993

[8] Zhiwei Zhu, Qiang Ji, “Robust real-time eye detection and tracking under variable lighting conditions and various face orientations,” Computer Vision and Image Understanding, Vol. 98, 2005, pp.124–154

[9] Arnon Amir, Lior Zimet, Alberto Sangiovanni-Vincentelli, Sean Kao, “An embedded system for an eye-detection sensor,” Computer Vision and Image Understanding, Vol. 98, 2005, pp. 104–123