EEG-Captioner (SJTU X-LANCE; Dual-Domain EEG encoder + Qwen2.5-Omni LoRA)
- Year
- 2026
- Form factor
- Other
Activity counts measure attention, not model quality. Missing counts are unknown. Publication status follows the linked source; a listed venue alone does not confirm peer review.
Joint QA + caption training: Macro-BA 88.81% on signal-level QA, caption factual consistency 83.46% (vs 82.06% caption-only), ROUGE-L 0.4865, BLEU-4 0.3302. The encoder beats LaBraM and CBraMod on QA (89.87 vs 76.06 / 50.45 at the classifier stage). The paper's QA checkpoint was selected on a val+test diagnostic set, so treat the numbers as optimistic.
- Signals
- EEG, Text
- Form factor
- Other
- Year
- 2026
- Architecture
Dual-Domain EEG Encoder (temporal patch branch + spectral-spatial branch, aligned and fused, global Transformer → compact EEG tokens) + MLP projector into Qwen2.5-Omni-7B Thinker with LoRA; trained on signal-level relational QA, then caption stages
- Pretraining data
No self-supervised pretraining; supervised on 128,625 TUEG v2.0.2 training segments (QA + captions)
- License
Code MIT (repo LICENSE); HF bundle Apache-2.0 (encoder, projector, LoRA). Base Qwen2.5-Omni-7B is downloaded separately under its own terms.
- Type
- FM adaptation
- Form factor
- Other
- Parameters
- Not reported
- Architecture
- Dual-Domain EEG Encoder (temporal patch branch + spectral-spatial branch, aligned and fused, global Transformer → compact EEG tokens) + MLP projector into Qwen2.5-Omni-7B Thinker with LoRA; trained on signal-level relational QA, then caption stages
- Venue
- arXiv 2610.30995v1 2026-10-22
- Year
- 2026
- DOI
- Not reported
- Weights
- Open (opens in a new tab)
- Licence
- Code MIT (repo LICENSE); HF bundle Apache-2.0 (encoder, projector, LoRA). Base Qwen2.5-Omni-7B is downloaded separately under its own terms.
- Caveats
- Not reported
- Datasets
- Not reported
- Added
- Oct 23, 2026
- Updated
- Oct 26, 2026
Abstract
Electroencephalography (EEG)-to-language models offer a promising approach for translating complex neural signals into human-readable descriptions. However, generating plausible language does not necessarily demonstrate that a model has learned to interpret the underlying EEG signal. A direct way to assess such interpretation is to probe objectively measurable properties of the signal itself. We therefore construct an EEG-language dataset of 160,781 segments annotated with signal-level question answering (QA) pairs, where each question targets one of five property families and requires a quantitative comparison involving spectral, spatial, temporal, or lateral signal properties. Each segment is also paired with a feature-grounded EEG caption. We further propose a Dual-Domain EEG Encoder that jointly models temporal waveforms and spectral-spatial representations, and introduce EEG-Captioner, a unified EEG-to-language framework trained with both signal-level QA and EEG caption supervision. Experiments show that the proposed encoder achieves the highest overall macro-averaged balanced accuracy (Macro-BA) among the compared EEG encoders, while EEG-Captioner achieves a Macro-BA of 88.81% on signal-level QA and improves caption factual consistency from 82.06% to 83.46% over caption-only language adaptation, demonstrating the complementary benefits of signal-level QA and caption supervision.
Verbatim fromarXiv· arXiv metadata (abstracts are released under CC0)
How it works
- Pretraining data
- No self-supervised pretraining; supervised on 128,625 TUEG v2.0.2 training segments (QA + captions)
- Architecture
- Dual-Domain EEG Encoder (temporal patch branch + spectral-spatial branch, aligned and fused, global Transformer → compact EEG tokens) + MLP projector into Qwen2.5-Omni-7B Thinker with LoRA; trained on signal-level relational QA, then caption stages
- Inputs
- EEG, Text
- Approach
- EEG · EEG to language · LLM · LoRA · sensor language · open weights
Datasets
Papers· 1
Cite
APA
Jin, L., Mei, J., Yuan, Y., Li, S., Wu, M., & Ding, Y. (2026). EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties. arXiv 2610.30995v1 2026-10-22. https://arxiv.org/abs/2610.30995
BibTeX
@misc{jin2026eegcaptioner,
title = {{EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties}},
author = {Jin, Leyuan and Mei, Jiahao and Yuan, Yixuan and Li, Shimeng and Wu, Mengyue and Ding, Yue},
year = {2026},
howpublished = {arXiv 2610.30995v1 2026-10-22},
eprint = {2610.30995},
archivePrefix = {arXiv}
}Built from the hub record. Check it against the publisher before submitting.
History
Record updated
Added to the ChatPPG research hub
