PaperarXiv:2610.33964
QECGBERT: a scalable ECG foundation model based on self-supervised language learning via VQ-VAE waveform tokenization
arXiv 2610.33964v1 (cs.LG, 25 Oct 2026)
Licence: arXiv.org perpetual non-exclusive licence (arXiv default; not an open-content licence)
TL;DR
QECGBERT tokenizes 12-lead ECG heartbeats with per-segment VQ-VAEs and pretrains a 15-million-parameter BERT with masked language modelling on MIMIC-IV-ECG (800,035 recordings). With LoRA fine-tuning, the abstract and paper tables report competitive macro-AUROC on PTB-XL and related diagnosis benchmarks at a total size of 21.4 million parameters. No code or weights are released.
- Form factor: other (12-lead clinical ECG)
Abstract
AI foundation models play a pivotal role in advancing heart disease diagnosis from electrocardiogram (ECG) data, particularly for rare diseases where the lack of sufficient labeled data hinders supervised learning. However, most self-supervised learning methods treat ECGs as generic sequence or image data without exploiting underlying physiological structure. To address this limitation, we propose QECGBERT, a self-supervised ECG language model that explicitly encodes physiological semantics by tokenizing the P, QRS, T, and isoelectric background waveforms. Vector-quantized variational autoencoder (VQ-VAE) tokenization is applied to each 12-lead ECG, with tokens enriched by continuous waveform features to preserve morphology lost during quantization without increasing sequence length. This design enables compact BERT masked language modeling with short sequences, improving scalability and reducing computational cost relative to long-sequence approaches. QECGBERT applies BERT pre-training to the MIMIC-IV dataset containing over 800,000 unlabeled ECG recordings, and fine-tuning on labeled PTB-XL, CPSC2018, and Chapman–Shaoxing–Ningbo datasets for evaluation. The results demonstrate that QECGBERT achieves comparable or superior performance to published benchmarks, offering a scalable, physiologically aware approach to ECG foundation modeling for cardiac diagnosis.
Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)
Results on ChatPPG leaderboards
All leaderboardsNo results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.
Tasks & signals
Signals
Keywords
Datasets
How it works
Cite this paper
APA
Mohan, A., Ozek, B., Zilbershot, O., & Afghah., F. (2026). QECGBERT: a scalable ECG foundation model based on self-supervised language learning via VQ-VAE waveform tokenization. arXiv 2610.33964v1. https://arxiv.org/abs/2610.33964
BibTeX
@misc{mohan2026qecgbert,
title = {{QECGBERT: a scalable ECG foundation model based on self-supervised language learning via VQ-VAE waveform tokenization}},
author = {Mohan, Aruna and Ozek, Burcu and Zilbershot, Or and Afghah., Fatemeh},
year = {2026},
howpublished = {arXiv 2610.33964v1},
eprint = {2610.33964},
archivePrefix = {arXiv}
}Built from the hub record. Check it against the publisher before submitting.
Added to the ChatPPG research hub
