Activity counts measure attention, not model quality. Missing counts are unknown. Publication status follows the linked source; a listed venue alone does not confirm peer review.
The model learns which video tokens to trust before it estimates vitals. An evidence miner weights spatiotemporal tokens so that skin and chest-motion tokens count and background, lighting and pose tokens are suppressed. Masked video-token modelling makes the encoder robust to missing evidence.
video encoder → contextualised spatiotemporal tokens; masked video-token reconstruction regulariser; evidence miner assigns token reliability weights (softmax over tokens); signal encoder for in-batch video↔signal alignment (training only); latent physiological state queries with self- and cross-attention over reliability-weighted tokens; three heads (waveform, rates, log-variance uncertainty)
Contactless vital-sign estimation from video remains difficult because physiological cues are weak, spatially localized, and easily corrupted by illumination changes, motion artifacts, pose variation, and sensor noise. Existing methods typically learn deterministic mappings from video to physiological waveforms or rates, but they rarely model which visual evidence is reliable or when a prediction should be trusted. We propose a reliability-guided video token learning framework for uncertainty-aware physiological signal estimation. The framework learns to mine informative spatiotemporal tokens, suppress unreliable visual evidence, and infer structured physiological dynamics through masked video-token modeling, signal-guided alignment, and supervised vital-sign estimation. In addition to predicting physiological waveforms and rates, the model outputs an uncertainty score that reflects visual ambiguity and prediction reliability. We evaluate the framework across heterogeneous contactless sensing settings, including respiratory and rPPG benchmarks. In addition to standard accuracy metrics, we assess reliability using uncertainty separation, high-error detection, and risk–coverage analysis. Results show that explicit visual-evidence reliability modeling improves robustness across datasets and provides a practical confidence measure for contactless physiological monitoring.
Verbatim fromSemantic Scholar· Abstract via Semantic Scholar; © the authors/publisher
Datasets linked in the hub; pretraining vs evaluation roles follow the paper.
Reliability-Guided Video Token Learning for contactless HR + RR with uncertainty (CIKM 2026). (2026). CIKM 2026 , Rome, 7–11 Nov 2026. https://doi.org/10.1145/3799682.3840049
@article{rgvtlreliability2026reliabilityguided,
title = {{Reliability-Guided Video Token Learning for contactless HR + RR with uncertainty (CIKM 2026)}},
year = {2026},
journal = {CIKM 2026 , Rome, 7–11 Nov 2026},
doi = {10.1145/3799682.3840049}
}Record updated
