ChatPPG.
Join free

PaperarXiv:2406.14498

LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data

Venue listedDated Oct 2025Added Nov 9, 2026

  • SI
  • MK
  • SB
  • BL

Authors: Sheikh Asif Imran, Mohammad Nur Hossain Khan, Subrata Biswas, Bashima Islam (WPI ECE / BASH Lab)

Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp Companion ’25)

PDF (opens in a new tab) arXiv2406.14498 (opens in a new tab) Publisher (opens in a new tab) CodeBASHLab/LLaSA (opens in a new tab) Cite
CC BY

TL;DR

WPI BASH Lab builds SensorCaps (IMU→caption/narration via GPT-4o-mini) and OpenSQA (sensor-grounded QA), then fine-tunes Vicuna-7B/13B with a frozen LIMU-BERT encoder + MLP projector (LLaSA) for open-ended reasoning over raw IMU. Reports stronger scientific/narrative/reliability scores than GPT-3.5 and Vicuna-13B on Tune-OpenSQA; competitive with GPT-4o-mini; strong zero-shot HAR F1 vs text-only LLMs.

Abstract

Wearable systems can recognize activities from IMU data but often fail to explain their underlying causes or contextual significance. To address this limitation, we introduce two large-scale resources: SensorCap, comprising 35,960 IMU–caption pairs, and OpenSQA, with 199,701 question–answer pairs designed for causal and explanatory reasoning. OpenSQA includes a curated tuning split (Tune-OpenSQA) optimized for scientific accuracy, narrative clarity, and diagnostic insight. Leveraging these datasets, we develop LLaSA (Large Language and Sensor Assistant), a family of compact sensor-aware language models (7B and 13B) that generate interpretable, context-rich responses to open-ended questions grounded in raw IMU data. LLaSA outperforms commercial LLMs, including GPT-3.5 and GPT-4o-mini, on benchmark and real-world tasks, demonstrating the effectiveness of domain supervision and model alignment for sensor reasoning. Our code repository and datasets can be found at https://github.com/BASHLab/LLaSA.

Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)

Results on ChatPPG leaderboards

All leaderboards

No results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.

Tasks & signals

Tasks

Activity recognitionSignal quality

Signals

IMU

Keywords

HARLLMSensor LanguageLLaSASensorCapsOpenSQACaptioningWPI

Datasets used

How it works

How Method works: Setup → Method → TargetSetup: Signals (accelerometer, gyroscope, IMU). Method: Method (Language-aligned). Target: Activity recognition; Signal quality.SetupSignals: accelerometer, gyroscope, IMUxyzSignalsaccelerometer, gyroscope, IMUMethodMethodLanguage-alignedTargetActivity recognitionActivity recognitionSignal qualitySignal quality

Cite this paper

Cite this paper with the fields recorded here. Check it against the publisher before submitting.

Imran, S. A., Khan, M. N. H., Biswas, S., & Islam, B. (2025). LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data. Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing. https://doi.org/10.1145/3714394.3756187
Imran, Sheikh Asif, Mohammad Nur Hossain Khan, Subrata Biswas, and Bashima Islam. 2025. "LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data." Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing. https://doi.org/10.1145/3714394.3756187.
Imran, S. A., Khan, M. N. H., Biswas, S. and Islam, B. (2025) 'LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data', Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing. doi: 10.1145/3714394.3756187.
Imran, Sheikh Asif, et al. "LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data." Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2025. https://doi.org/10.1145/3714394.3756187.
S. A. Imran, M. N. H. Khan, S. Biswas, and B. Islam, "LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data," Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2025, doi: 10.1145/3714394.3756187.
@inproceedings{imran2025llasa,
  title         = {{LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data}},
  author        = {Imran, Sheikh Asif and Khan, Mohammad Nur Hossain and Biswas, Subrata and Islam, Bashima},
  year          = {2025},
  booktitle     = {Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing},
  doi           = {10.1145/3714394.3756187},
  eprint        = {2406.14498},
  archivePrefix = {arXiv}
}

Added to the ChatPPG research hub

Same tasks, data and authors

Papers

Models

Datasets