CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° VR Videos

CEAP-360VR has 32 participants watching eight one-minute affective 360° videos in a Vive Pro Eye headset while annotating valence and arousal continuously. It records head and eye movement and pupil size from the headset, and Empatica E4 wrist signals (BVP, EDA, skin temperature, HR, IBI, ACC), plus momentary and discrete emotion ratings, motion sickness, presence and workload. Baseline Random Forest models on 2-second segments reach 66.8% (valence) and 64.3% (arousal) binary accuracy subject-independent, and 49.9% / 52.2% for 3-class; subject-dependent models reach 68.5% / 71.3% binary and 60.4% / 62.4% 3-class.

Watching 360^ videos using Virtual Reality (VR) head-mounted displays (HMDs) provides interactive and immersive experiences, where videos can evoke different emotions. Existing emotion self-report techniques within VR however are either retrospective or interrupt the immersive experience. To address this, we introduce the Continuous Physiological and Behavioral Emotion Annotation Dataset for 360^ Videos (CEAP-360VR). We conducted a controlled study (N=32) where participants used a Vive Pro Eye HMD to watch eight validated affective 360^ video clips, and annotated their valence and arousal (V-A) continuously. We collected (a) behavioral (head and eye movements; pupillometry) signals (b) physiological (heart rate, skin temperature, electrodermal activity) responses (c) momentary emotion self-reports (d) within-VR discrete emotion ratings (e) motion sickness, presence, and workload. We show the consistency of continuous annotation trajectories and verify their mean V-A annotations. We find high consistency between viewed 360^ video regions across subjects, with higher consistency for eye than head movements. We furthermore run baseline classification experiments, where Random Forest classifiers with 2s segments show good accuracies for subject-independent models: 66.80% (V) and 64.26% (A) for binary classification; 49.92% (V) and 52.20% (A) for 3-class classification. Our open dataset allows further experiments with continuous emotion self-reports collected in 360^ VR environments, which can enable automatic assessment of immersive Quality of Experience (QoE) andmomentary affective states.

Verbatim from Semantic Scholar (opens in a new tab) · Abstract via Semantic Scholar; © the authors/publisher

1 row is credited to this paper. Ranks apply within each leaderboard only, as reported by the source.