Benchmark on a large cohort for sleep-wake classification with machine learning techniques

Large open MESA Sleep benchmark for wrist actigraphy count sleep/wake vs PSG. Uses n=1817 subjects with synchronized overnight PSG + Philips Actiwatch Spectrum (of 2237 MESA Sleep). Defines Task Night (PSG window) and Task Night&Day (+ up to 8 h before/after with expert rest annotations).

• Form factor: wrist

Accurately measuring sleep and its quality with polysomnography (PSG) is an expensive task. Actigraphy, an alternative, has been proven cheap and relatively accurate. However, the largest experiments conducted to date, have had only hundreds of participants. In this work, we processed the data of the recently published Multi-Ethnic Study of Atherosclerosis (MESA) Sleep study to have both PSG and actigraphy data synchronized. We propose the adoption of this publicly available large dataset, which is at least one order of magnitude larger than any other dataset, to systematically compare existing methods for the detection of sleep-wake stages, thus fostering the creation of new algorithms. We also implemented and compared state-of-the-art methods to score sleep-wake stages, which range from the widely used traditional algorithms to recent machine learning approaches. We identified among the traditional algorithms, two approaches that perform better than the algorithm implemented by the actigraphy device used in the MESA Sleep experiments. The performance, in regards to accuracy and F1 score of the machine learning algorithms, was also superior to the device’s native algorithm and comparable to human annotation. Future research in developing new sleep-wake scoring algorithms, in particular, machine learning approaches, will be highly facilitated by the cohort used here. We exemplify this potential by showing that two particular deep-learning architectures, CNN and LSTM, among the many recently created, can achieve accuracy scores significantly higher than other methods for the same tasks.

Verbatim from the publisher's DOI record (opens in a new tab) · Publisher licence: https://creativecommons.org/licenses/by/4.0

1 row is credited to this paper. Ranks apply within each leaderboard only, as reported by the source.

• MESA SleepNSRR polysomnography + wrist actigraphy (Multi-Ethnic Study of Atherosclerosis)TrainingEvaluation2,237 subjects · PPG · ECG · IMU