VLM-guided interference-aware temporal feature refinement for rPPG (arXiv 2026)
Activity counts measure attention, not model quality. Missing counts are unknown. Publication status follows the linked source; a listed venue alone does not confirm peer review.
A refinement stage that sits after feature learning. An rPPG backbone outputs temporal features and a coarse signal. Local temporal statistics of that signal provide physiological reliability evidence.
physiological backbone (spatial encoder + temporal features + coarse rPPG) → signal-derived reliability evidence; frozen VLM on sampled frames with an interference prompt → visual evidence head; temporal cross-attention context; shared correction unit + 4 interference-specific experts with adaptive routing
Remote photoplethysmography (rPPG) enables contactless physiological measurement from facial videos, yet its subtle pulse-related variations are easily affected by illumination variation, head motion, facial blur, and region-of-interest instability. Existing methods mainly suppress interference during feature learning, while whether the learned temporal features remain affected by interference and how to further suppress such interference before rPPG estimation are rarely examined. To address this limitation, we propose PhysVR, a vision-language model guided interference-aware temporal feature refinement framework for rPPG estimation. Specifically, a physiological backbone produces global temporal features and a coarse rPPG prediction, from which signal-derived physiological reliability evidence is constructed from local temporal characteristics. In parallel, a frozen vision-language model processes sampled facial frames under an interference-oriented prompt, and an evidence head extracts visual interference evidence from the VLM output. Temporal cross-attention integrates the physiological and visual evidence with the global temporal features to construct interference-aware temporal context. Guided by this context, a shared temporal correction unit performs general refinement, while four interference-specific experts selectively suppress different interference through adaptive routing. The refined temporal features are then used for final rPPG estimation. Extensive experiments on five public benchmarks demonstrate that PhysVR consistently outperforms representative methods under both intra-dataset and cross-dataset evaluation protocols.
Verbatim fromarXiv· arXiv metadata (abstracts are released under CC0)
Datasets linked in the hub; pretraining vs evaluation roles follow the paper.
| Task | Dataset | Metric | Value | Rank |
|---|---|---|---|---|
| Rppg hr | BUAA-MIHR intra | HR MAE ↓ | 0.42 | #1 / 4 |
| Rppg hr | mmpd intra | HR MAE ↓ | 3.92 | #3 / 6 |
| Rppg hr | mr-nirp NIRP-DRV intra | HR MAE ↓ | 2.56 | #1 / 1 |
| Rppg hr | mr-nirp NIRP-IND intra | HR MAE ↓ | 0.38 | #1 / 1 |
| Rppg hr | mr-nirp → vipl-hr | HR MAE ↓ | 3.92 | #1 / 1 |
| Rppg hr | vipl-hr intra | HR MAE ↓ | 3.59 | #1 / 2 |
| Rppg hr | vipl-hr → mr-nirp NIRP-DRV | HR MAE ↓ | 5.14 | #1 / 1 |
| Rppg hr | vipl-hr → mr-nirp NIRP-IND | HR MAE ↓ | 1.34 | #1 / 1 |
PhysVR. (2026). arXiv preprint. https://arxiv.org/abs/2608.29663
@misc{physvr2026physvr,
title = {{PhysVR}},
year = {2026},
howpublished = {arXiv preprint},
eprint = {2608.29663},
archivePrefix = {arXiv}
}Record updated