Cross-Dataset Transfer and Reliability of Explainable Artificial Intelligence for RhythmFormer Remote Photoplethysmography

Trains condition-specific RhythmFormer models on multi-condition NCKU-rPPG (illumination, speak, rotate, bike) vs UBFC-rPPG reproduction; evaluates XAI methods (raw attention, rollout, flow, Beyond Intuition) with skin coverage and Salience-guided Faithfulness Coefficient (SaCo). Key finding: attribution skin coverage / SaCo carry information complementary to MAE — looking at skin does not guarantee accurate HR. Beyond Intuition ranks highest on clean conditions; fails hard at 40 lux.

• Form factor: camera_rppg

Background. Remote photoplethysmography estimates the cardiovascular pulse from facial video, and its explanations have rested on inspecting heatmaps rather than on quantitative evidence about where a model reads it. We quantified the explanations and asked whether such explanations transfer between datasets and track model performance. Method. We trained eight condition-specific RhythmFormer models on NCKU-rPPG, recorded under three illumination levels, speaking, rotation, and cycling, estimated one heart rate per 5.12-second clip, and set them beside a UBFC-rPPG reproduction. Raw attention, rollout, attention flow, and Beyond Intuition were assessed by skin coverage and the Salience-guided Faithfulness Coefficient (SaCo). Results. Beyond Intuition ranked highest on both datasets, at median coverage 0.789 and SaCo 0.837 on Static level 3 against 0.826 and 0.917 on UBFC-rPPG; lower ranks differed. Within one participant of one condition, neither measure was related to a clip's heart-rate error, waveform correlation, or signal-to-noise ratio on either dataset: 186 of the 252 coefficients fell below ||=0.10 and 28 reached p<0.05 against the 13 expected by chance. Across the eight scenarios only Beyond Intuition's coverage followed the three performance measures, at =-0.43, +0.57, and +0.43, while the attention-only methods' SaCo ran opposite to each. It failed at 40 lux alone, its median coverage falling to 0.180 and its median SaCo to -0.178, whereas motion degraded the estimates far more without such a drop. Conclusions. Skin coverage and SaCo carry information complementary to the performance measures rather than a proxy for them: attributing to the skin does not guarantee an accurate estimate. What an attribution reveals about a condition is where the model looks rather than how faithfully its map is ordered.

Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)

No results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.

Wavelengths

RGB_camera