- 期刊
沉浸式虚拟现实环境中认知投入的自动测评研究
Automatic Assessment of Cognitive Engagement in Immersive Virtual Reality Environments
摘要
沉浸式虚拟现实（Immersive Virtual Reality, IVR）环境在提升学生的行为、情感和认知投入方面具有显著优势。由于认知投入的内隐性，当前研究主要采用自我报告方法来测评学生在IVR环境中的认知投入，无法实现对认知投入的全面捕获和实时追踪。研究旨在探索采用计算机视觉技术实现对IVR环境中认知投入的实时追踪和自动测评方法。根据信息加工理论和具身认知理论，研究从视觉行为和身体行为两方面间接表征学生在IVR环境中的认知投入，并通过计算机视觉和深度学习技术自动识别和检测学生在IVR环境中的视觉行为和身体交互行为，检测准确率高达98%。相关性分析发现，学生在IVR学习中对学习内容的视觉覆盖度与其后续的知识保留和知识迁移成绩显著正相关；学生在IVR学习中使用手柄交互的累计时长与其后续的知识迁移成绩显著正相关。上述研究结论表明：基于计算机视觉技术自动识别学生在IVR学习中的认知投入是一种可行且有效的方法，该方法能够为IVR学习环境的设计和优化提供客观且丰富的过程性数据，有助于破解IVR教学实践中教师无法精准监测学生的学习投入并提供及时指导和反馈这一现实困境。
並列摘要
It is confirmed that students' behavioral, emotional, and cognitive engagement has been enhanced in immersive virtual reality (IVR) environment. Due to the implicit nature of cognitive engagement, current research has mainly adopted self-report methods to assess students' cognitive engagement in IVR environment, which does not allow for comprehensive capture and real-time tracking of cognitive engagement. The study aims to explore the method of real-time tracking and automatic measurement of cognitive engagement in IVR environment using computer vision technology. Students' visual behavior and physical behavior were selected as proxies of their cognitive engagement in IVR environment based on information processing theory and embodied cognition theory. Computer vision and deep learning technology were employed to automatically identify and detect students' visual behavior and physical interaction in IVR environment, with a detection accuracy of up to 98%. By the correlation analysis, it is found that students' visual coverage of learning content in IVR learning was significantly and positively correlated with their subsequent performance on knowledge retention and knowledge transfer. The cumulative duration of students' usage of VR controller in IVR learning is positively correlated with their subsequent performance on knowledge transfer. The above findings indicate that it is feasible and effective to automatically detect students' cognitive engagement in IVR environment by computer vision technology, which can provide rich process data for the design and optimization of IVR learning environment, and help to solve the dilemma that teachers cannot accurately monitor students' learning engagement in IVR environment and provide timely guidance and feedback.