Activity counts measure attention, not model quality. Missing counts are unknown. Publication status follows the linked source; a listed venue alone does not confirm peer review.
A hybrid of a conventional rPPG CNN and a language model. A fixed PhysNet backbone produces a raw rPPG waveform, which is smoothed (Dual-Domain Stationary module). Text Prototype Guidance projects the signal and the multi-scale visual features into the LLM's token space.
frozen/fixed PhysNet backbone gives a raw rPPG waveform → Dual-Domain Stationary (DDS) smoothing; multi-scale visual features; Text Prototype Guidance (TPG) maps signal and visual features to LLM token space; Adaptive Prompt Learning fuses task, vision (LLaVA caption) and signal-statistics cues; LLM (DeepSeek best in ablation; GPT-2, LLaMA, BERT also supported) predicts the waveform; MSE loss
Remote photoplethysmography (rPPG) enables non-contact physiological measurement but remains highly susceptible to illumination changes, motion artifacts, and limited temporal modeling. Large Language Models (LLMs) excel at capturing long-range dependencies, offering a potential solution but struggle with the continuous, noise-sensitive nature of rPPG signals due to their text-centric design. To bridge this gap, we introduce the PhysLLM, a collaborative optimization framework that synergizes LLMs with domain-specific rPPG components. Specifically, the Text Prototype Guidance (TPG) strategy is proposed to establish cross-modal alignment by projecting hemodynamic features into LLM-interpretable semantic space, effectively bridging the representational gap between physiological signals and linguistic tokens. Besides, a novel Dual-Domain Stationary (DDS) Algorithm is proposed for resolving signal instability through adaptive time-frequency domain feature re-weighting. Finally, rPPG task-specific cues systematically inject physiological priors through physiological statistics, environmental contextual answering, and task description, leveraging cross-modal learning to integrate both visual and textual information, enabling dynamic adaptation to challenging scenarios like variable illumination and subject movements. Evaluation on four benchmark datasets, PhysLLM achieves state-of-the-art accuracy and robustness, demonstrating superior generalization across lighting variations and motion scenarios. The source code is available at https://github.com/Alex036225/PhysLLM.
Verbatim fromarXiv· arXiv metadata (abstracts are released under CC0)
Datasets linked in the hub; pretraining vs evaluation roles follow the paper.
| Task | Dataset | Metric | Value | Rank |
|---|---|---|---|---|
| Rppg hr | BUAA-MIHR intra | HR MAE ↓ | 6.48 | #4 / 4 |
| Rppg hr | PURE intra | HR MAE ↓ | 0.17 | #3 / 5 |
| Rppg hr | UBFC-rPPG intra | HR MAE ↓ | 0.21 | #5 / 5 |
| Rppg hr | mmpd intra | HR MAE ↓ | 4.36 | #5 / 6 |
| Rppg hr | others → buaa-mihr | HR MAE ↓ | 6.01 | #1 / 1 |
| Rppg hr | pure+ubfc-rppg → mmpd | HR MAE ↓ | 9.95 | #2 / 2 |
| Rppg hr | ubfc-rppg+pure+buaa-mihr → mmpd | HR MAE ↓ | 12.2 | #1 / 1 |
PhysLLM. (2025). ICLR 2026. https://arxiv.org/abs/2505.03621
@inproceedings{physllm2025physllm,
title = {{PhysLLM}},
year = {2025},
booktitle = {ICLR 2026},
eprint = {2505.03621},
archivePrefix = {arXiv}
}Record updated
