PaperarXiv:2610.30995
EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties
arXiv 2610.30995v1, 2026-10-22 (cs.LG)
Open codeOpen weights
TL;DR
Builds an EEG-language dataset from TUEG v2.0.2: 160,781 segments, each with one feature-grounded caption and 25 candidate relational QA pairs across five property families (dominant band, strongest region, peak period, temporal pattern, lateral dominance). Answers are computed from the signal, so they can be scored objectively. EEG-Captioner: encoder tokens are projected into the Qwen2.5-Omni-7B Thinker with a trainable projector and LoRA. It's trained on QA first, then three caption stages with rising caption fractions (0.25, 0.75, 0.90).
- Form factor: other
Abstract
Electroencephalography (EEG)-to-language models offer a promising approach for translating complex neural signals into human-readable descriptions. However, generating plausible language does not necessarily demonstrate that a model has learned to interpret the underlying EEG signal. A direct way to assess such interpretation is to probe objectively measurable properties of the signal itself. We therefore construct an EEG-language dataset of 160,781 segments annotated with signal-level question answering (QA) pairs, where each question targets one of five property families and requires a quantitative comparison involving spectral, spatial, temporal, or lateral signal properties. Each segment is also paired with a feature-grounded EEG caption. We further propose a Dual-Domain EEG Encoder that jointly models temporal waveforms and spectral-spatial representations, and introduce EEG-Captioner, a unified EEG-to-language framework trained with both signal-level QA and EEG caption supervision. Experiments show that the proposed encoder achieves the highest overall macro-averaged balanced accuracy (Macro-BA) among the compared EEG encoders, while EEG-Captioner achieves a Macro-BA of 88.81% on signal-level QA and improves caption factual consistency from 82.06% to 83.46% over caption-only language adaptation, demonstrating the complementary benefits of signal-level QA and caption supervision.
Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)
Results on ChatPPG leaderboards
All leaderboardsNo results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.
Tasks & signals
Signals
Keywords
Models
How it works
Cite this paper
APA
Jin, L., Mei, J., Yuan, Y., Li, S., Wu, M., & Ding, Y. (2026). EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties. arXiv 2610.30995v1, 2026-10-22. https://arxiv.org/abs/2610.30995
BibTeX
@misc{jin2026eegcaptioner,
title = {{EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties}},
author = {Jin, Leyuan and Mei, Jiahao and Yuan, Yixuan and Li, Shimeng and Wu, Mengyue and Ding, Yue},
year = {2026},
howpublished = {arXiv 2610.30995v1, 2026-10-22},
eprint = {2610.30995},
archivePrefix = {arXiv}
}Built from the hub record. Check it against the publisher before submitting.
Added to the ChatPPG research hub
