Towards an Emotion-Aware Metaverse: A Human-Centric Shipboard Fire Drill Simulator

Abstract

[Abstract]: Traditional Extended Reality (XR) and Metaverse applications focus heavily on User Experience (UX) but often overlook the role of emotions in user interaction. This article addresses that gap by presenting an emotion-aware Metaverse application: a Virtual Reality (VR) fire drill simulator for shipboard emergency training. The simulator detects emotions in real time, assessing trainees’ responses under stress to improve learning outcomes. Its architecture incorporates eye-tracking and facial expression analysis via Meta Quest Pro headsets. Two experimental phases were conducted. The first revealed issues like poor navigation and lack of visual guidance. These insights led to an improved second version with a refined User Interface (UI), a real-time task tracker and clearer visual cues. The obtained results showed that the included design improvements can reduce task completion times between 14.18% and 32.72%. Emotional feedback varied, suggesting a need for more immersive elements. Overall, this article provides useful guidelines for creating the next generation of emotion-aware Metaverse applications.