Model family 03 of 04
Sensor–language
Sensor–language models connect wearable recordings with text. Some align signals and captions for retrieval, others generate reports or answer questions about a recording, and a few are language agents that call signal-processing tools.
Signals ↔ language- Models
- 16
- Open weights
- 4
- Benchmarks
- 6
- Years
- 22–26
- Text · 10
- PPG · 7
- ECG · 7
- IMU · 6
- EEG · 4
- time-series · 2
16 sensor–language models
Newest first · one primary role per record
AnyMoFoundation modelIMU2026 · Open weights
BLPM (Brain Latent Predictive Model)Foundation modelEEGText2026
CAPFoundation modelPPGText2026- ECG-GPTFoundation modelECG2026
HALOFoundation modelIMU2026
HiPPTwrist-PPG encoder aligned to text at record, cross-recording and generative scopesFoundation modelPPGText2026
Lapras (latent-reasoning post-training for time-series language models)ECGIMUText2026 · Open weights
LuminaECGECGText2026
MS-ECG-FMmulti-source contrastive ECG foundation model (Apple)Foundation modelECG2026
OpenSLASensor-Language-Action model (UCLA, Yuzhe Yang lab)Foundation modelPPGECGBP2026 · Weights promised
PPG-LMFoundation modelPPGText2026
SleepLMFoundation modelSpO₂RespEEG2026 · Open weights
SLIP (Sensor Language-Informed Pretraining; any-channel sensor-language FM)Foundation modelPPGECGIMU2026 · Open weights
VitalAgenttool-augmented ECG/PPG monitoring agent (Univ. of Melbourne, Ting Dang group)PPGECGText2026
SensorLMFoundation modelPPGIMUSpO₂2025
IMU2CLIPFoundation modelIMU2022
Where these models are benchmarked
All leaderboards
Atrial fibrillation detection
Long-term ECG and wrist PPG with annotated AF episodes (AFPPGECG): balanced accuracy
Includes VitalAgent · 1 result

Atrial fibrillation detection
Icentia11k single-lead continuous raw ECG (PhysioNet): balanced accuracy
Includes VitalAgent · 1 result

Atrial fibrillation detection
ptb-xl-rhythm: AUROC
Includes MS-ECG-FM · 1 result

Heart rate estimation
PPG-DaLiA: MAE (BPM)
Includes HiPPT · 14 results

Human activity recognition
opentslm-har-cot: macro-F1
Includes Lapras (latent-reasoning post-training for time-series language models) · 1 result

Sleep staging
opentslm-sleep-cot: macro-F1
Includes Lapras (latent-reasoning post-training for time-series language models) · 1 result
Datasets these models use
All datasets
VitalDBUsed by CAP, OpenSLA, PPG-LMintraoperative multi-parameter vitals (PhysioNet mirror)6,388 subjects · PPG · ECG · BP
WESADUsed by CAP, SLIP (Sensor Language-Informed Pretraining; any-channel sensor-language FM), VitalAgentWearable Stress and Affect Detection15 subjects · PPG · ECG · IMU
OPPORTUNITYUsed by AnyMo, HALO4 subjects · IMU
PAMAP2Used by AnyMo, HALOPhysical Activity Monitoring (n=9, 3× IMU + HR)9 subjects · IMU · Temp · HR
PPG-DaLiAUsed by HiPPT, VitalAgentPPG dataset for motion compensation / HR15 subjects · PPG · ECG · IMU
RealWorld HARUsed by AnyMo, HALOmulti-position smartphone/smartwatch IMU (n=15)15 subjects · IMU · Audio · gps
WISDM AR v1.1Used by AnyMo, HALOcellphone accelerometer HAR (n=36)36 subjects · IMU
BIDMC PPG and Respiration (Pimentel 2016 / PhysioNet)Used by CAP53 subjects · PPG · ECG · SpO₂- Long-term ECG and wrist PPG with annotated AF episodes (AFPPGECG)Used by VitalAgent39 subjects · PPG · ECG · IMU
MIMIC-IV Waveform Database (PhysioNet preview)Used by OpenSLA198 subjects · PPG · ECG · BP
PulseDB v2.0Used by CAPcleaned MIMIC-III + VitalDB cuffless-BP benchmark (MIMIC part ODbL; VitalDB part CC BY-NC-SA)5,361 subjects · PPG · ECG · BP
Stanford wrist PPG AF (DeepBeat)Used by CAPTorres-Soto & Ashley 2020, Synapse syn21985690PPG
Families are editorial primary-role groupings reviewed against each model record, not an architecture taxonomy. A model appears in one family even when it could fit more than one. Diagrams are conceptual.
Other families
Browse model families
Flow previews are paused. Each diagram shows the complete process.
Conceptual diagrams, not model architectures. One primary role per record; families can overlap.
Other / unclassified 79