結合沈浸式體驗之互動式故事書與手勢滑鼠系統之實現

The Design of Interactive Story Book and the Implementation of Mouse Control System by Using Gesture

張同昇(Tong-Sheng Jhang)

指導教授 ： 周建興

共同指導教授 ： 蔡奇謚(Chi-Yi Tsai)

淡江大學/工學院/電機工程學系碩士在職專班/碩士(2019年)

https://doi.org/10.6846/TKU.2019.00730

摘要

本論文設計一套結合互動式體驗之故事書，名為4D故事書。體驗之互動式是針對有故事臨場感，譬如當故事中出現光、熱、風及雨等劇情時，我們結合外部硬體裝置來模擬這些體驗。例如利用風扇、熱燈、熱風扇及水瓶喷水來模擬模擬，風雨、燒熱、火光故事情境等，讓小朋友在說故事時擁有「4D」的互動式體驗。此互動式故事書採用光敏電阻、繼電器模組來控制交流電電器設備此外我們也設計一個「電源模組」，此電源模組控制中介用配電盤交流電器裝置，並對故事內容可「回饋場景」，直接開啟電源或關閉電源。此4D故事書能給讀者生動、活潑和場景模擬，讓讀者在具娛樂性與興趣中閱讀。此外，在人機介面設計上，本論文也實現了手勢滑鼠控制系統，藉由RGB-D影像資訊進行手勢辨識及滑鼠游標控制。此功能未來可與4D故事書系統結合，達到更自然的互動體驗。

關鍵字

沈浸式體驗 ； 模擬場景 ； 互動式體驗 ； 故事書 ； 手勢辨識 ； 滑鼠游標控制

並列摘要

This thesis designs a set of storybooks that combine interactive experiences, Named 4D storybook. The interactive experience is for a sense of story, For example, when there are scenes such as light, heat, wind and rain in the story, We combine external hardware devices to simulate these experiences. For example, using a fan, a heat lamp, a hot fan, and a water bottle spray to simulate the simulation, Wind and rain, hot, fire story etc, Let children have a "4D" interactive experience when telling stories. This interactive storybook uses photoresistors and relay modules to control AC electrical equipment. we also design a "power module", The power module controls the intermediary switch board AC electrical device, and “return the scene” to the content of the story. Turn the power on or off directly. This 4D storybook can give readers vivid, lively and scene simulation, Let readers read in entertainment and interest.
In addition, in the human-machine interface design, This paper also implements the gesture mouse control system, Gesture recognition and mouse cursor control are performed by RGB-D image information. This feature can be combined with the 4D storybook system in the future, Achieve a more natural interactive experience.

並列關鍵字

Immersive experience ； Simulation scenario ； Interactive experience ； story book ； Gesture recognition ； Mouse cursor control

參考文獻

參考文獻

[1] H.S. Grif, C. C. Farcas, “Mouse Cursor Control System Based on Hand Gesture‚ ” Procedia Technology, Vol 22, 2016, pp 657-661

[2] H.S. Grif, T. Turc, “Human hand gesture based system for mouse cursor control‚ ” Procedia Technology, Vol 22, 2018, pp 1038-1042

[3] K. Pendke, P. Khuje, S. Narnaware, S. Thool, S. Nimje, “COMPUTER CURSOR CONTROL MECHANISM BY USING HAND GESTURE RECOGNITION, ” International Journal of Computer Science and Mobile Computing, IJCSMC, Vol 4, Issue.3, March 2015, pp.293 – 300

[4] S.M.T.Saleem, S.Fareed, F.Bibi, A.Khan, S.Gohar, H.H. Ashraf, “IMouse: Eyes Gesture Control System‚” Article Published in International Journal of Advanced Computer Science and Applications, IJACSA, Vol 9, Issue 9, 2018