PaperarXiv:2507.13403
A multimodal drowsiness dataset using video, biometric, and behavioral data
Sci Data (published 24 Feb 2026); arXiv 2507.13403
TL;DR
Dataset paper for ul-dd: 19 subjects (16 with alert + drowsy sessions), 40-min truck-simulator sessions, KSS every 4 min (Cronbach α 0.99; expert vs self κw 0.967). Empatica E4 (HR 1 Hz, BVP, IBI, ACC 32 Hz, EDA/TEMP 4 Hz), Checkme O2 Max (SpO2 0.5 Hz, PR, motion), grip, telemetry, IR/depth/pose video. Validation: pulse rate, EDA, TEMP, SpO2 differ across KSS bins (mixed models); BVP not significant.
- Form factor: wrist (Empatica E4) + finger/wrist oximeter + cameras
Abstract
In this study, we present a comprehensive public dataset for driver drowsiness detection, integrating multimodal signals of facial, behavioral, and biometric indicators. Our dataset includes 3D facial video using a depth camera, IR camera footage, posterior videos, and biometric signals such as heart rate, electrodermal activity, blood oxygen saturation, skin temperature, and accelerometer data. This data set provides grip sensor data from the steering wheel and telemetry data from the American truck simulator game to provide more information about drivers' behavior while they are alert and drowsy. Drowsiness levels were self-reported every four minutes using the Karolinska Sleepiness Scale (KSS). The simulation environment consists of three monitor setups, and the driving condition is completely like a car. Data were collected from 19 subjects (15 M, 4 F) in two conditions: when they were fully alert and when they exhibited signs of sleepiness. Unlike other datasets, our multimodal dataset has a continuous duration of 40 minutes for each data collection session per subject, contributing to a total length of 1,400 minutes, and we recorded gradual changes in the driver state rather than discrete alert/drowsy labels. This study aims to create a comprehensive multimodal dataset of driver drowsiness that captures a wider range of physiological, behavioral, and driving-related signals. The dataset will be available upon request to the corresponding author.
Verbatim from arXiv (opens in a new tab) · arXiv metadata (abstracts are released under CC0)
Results on ChatPPG leaderboards
All leaderboardsNo results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.
Tasks & signals
Tasks
Signals
Keywords
How it works
Cite this paper
Bodaghi, M., Hosseini, M., Gottumukkala, R., Bhupatiraju, R. T., Ahmad, I., & Gabbouj, M. (2026). A multimodal drowsiness dataset using video, biometric, and behavioral data. Sci Data. https://doi.org/10.1038/s41597-025-06540-1
Bodaghi, Morteza, Majid Hosseini, Raju Gottumukkala, Ravi Teja Bhupatiraju, Iftikhar Ahmad, and Moncef Gabbouj. 2026. "A multimodal drowsiness dataset using video, biometric, and behavioral data." Sci Data. https://doi.org/10.1038/s41597-025-06540-1.
Bodaghi, M., Hosseini, M., Gottumukkala, R., Bhupatiraju, R. T., Ahmad, I. and Gabbouj, M. (2026) 'A multimodal drowsiness dataset using video, biometric, and behavioral data', Sci Data. doi: 10.1038/s41597-025-06540-1.
Bodaghi, Morteza, et al. "A multimodal drowsiness dataset using video, biometric, and behavioral data." Sci Data, 2026. https://doi.org/10.1038/s41597-025-06540-1.
M. Bodaghi, M. Hosseini, R. Gottumukkala, R. T. Bhupatiraju, I. Ahmad, and M. Gabbouj, "A multimodal drowsiness dataset using video, biometric, and behavioral data," Sci Data, 2026, doi: 10.1038/s41597-025-06540-1.
@article{bodaghi2026multimodal,
title = {{A multimodal drowsiness dataset using video, biometric, and behavioral data}},
author = {Bodaghi, Morteza and Hosseini, Majid and Gottumukkala, Raju and Bhupatiraju, Ravi Teja and Ahmad, Iftikhar and Gabbouj, Moncef},
year = {2026},
journal = {Sci Data},
doi = {10.1038/s41597-025-06540-1},
eprint = {2507.13403},
archivePrefix = {arXiv}
}Added to the ChatPPG research hub
