EgoHRV: Continuous Heart Rate Variability Estimation from Egocentric Systems for Autonomic Response and Skill Assessment

Estimates HR + HRV from inward-facing eye-tracking / gaze cameras already on egocentric headsets (no contact sensor at test time). Pipeline: 3D backbone + low–high decomposition → BVP from gaze video; cross-domain pretraining aligns frequency-domain contact vs camera signals for IBI; IMU for uncertainty; PhysFusion for downstream. Claims SOTA egocentric HR/HRV; integrating HRV + confidence into EgoExo4D proficiency estimator +17.8% accuracy (arXiv abstract).

• Form factor: camera_rppg

Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used noninvasive marker of autonomic regulation under stress. HRV reflects small timing differences between successive heartbeats and has so far been out of reach for egocentric platforms, where motion and noise in gaze video mask exactly this fine-grained timing. We propose EgoHRV, a method that estimates HRV as well as heart rate (HR) from the gaze cameras that are already integrated into egocentric headsets. Our pipeline combines a 3D backbone with a novel low–high decomposition module that extracts the blood volume pulse (BVP) signal from gaze video. Our cross-domain pretraining aligns the frequency-domain representations of contact-based and camera-derived signals. This alignment gives EgoHRV the temporal precision to recover HRV from the subtle fluctuations in gaze video. EgoHRV achieves state-of-the-art accuracy for HR and HRV estimation from egocentric video, and its uncertainty-aware design improves downstream behavioral modeling. Integrating our HRV estimates and confidence measures into EgoExo4D's proficiency estimator raises accuracy by 17.8%. Beyond skill, continuous HRV estimation also opens egocentric systems to stress- and arousal-aware estimation tasks. Code: https://github.com/eth-siplab/EgoHRV

Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)

No results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.