Activity counts measure attention, not model quality. Missing counts are unknown. Publication status follows the linked source; a listed venue alone does not confirm peer review.
GaitFM continues masked-autoencoder pretraining of an ImageNet-pretrained ViT-Tiny on about 1.23 M IMU gait windows from ~4,000 participants. A sensor-to-image embedder maps 5 s, 50 Hz windows into the vision backbone, and gyroscope channels are masked at random so the model also runs on accelerometer only. It is evaluated on 18 gait-related tasks, including gait-parameter estimation, disease diagnosis, FoG detection, fall risk and post-stroke recovery.
MAE with ViT-Tiny backbone initialised from ImageNet MAE-Lite (mae_tiny_400e), sensor-to-image embedder; gyro channels randomly masked (p=0.5) so accel-only input works
Pretraining data
Continued MAE pretraining (mask ratio 0.75) on ~4,000 participants / 1.23 M 5 s gait windows from 7 datasets (GSTRIDE, FoGContest, COMPWALK-ACL, mPower, WearGait-PD, Mobilise-D, ClinicalGait)
License
Paper CC BY 4.0 (arXiv); repo has NO licence file (all rights reserved by default); no checkpoint in repo despite paper statement
MAE with ViT-Tiny backbone initialised from ImageNet MAE-Lite (mae_tiny_400e), sensor-to-image embedder; gyro channels randomly masked (p=0.5) so accel-only input works
Gait assessment supports the diagnosis, monitoring, and prognosis of diverse movement-related conditions, such as neurological and musculoskeletal disorders. Recent advances in deep learning using everyday wearable inertial measurement units (IMUs) offer an accessible and low-cost approach to automated gait assessment. However, IMU measurements vary substantially across individuals, body locations, and device configurations, while existing deep learning models are typically developed using small, homogeneous datasets for specific clinical tasks, limiting their robustness and generalizability in diverse real-world contexts. Here we present GaitFM, a cross-modal foundation model for gait assessment using IMU sensors. GaitFM improves the generalizability of gait assessment by transferring knowledge from large vision models and pretraining on unlabeled IMU sensor data. We pretrain GaitFM on a large curated dataset comprising about 4,000 participants and 1.23 million gait windows across diverse clinical and daily conditions, sensor modalities, and body locations. GaitFM demonstrates strong performance across 18 gait-related tasks, including 3 gait parameter estimation tasks, 14 disease and gait abnormality detection tasks, and 1 clinical outcome prediction task. Furthermore, GaitFM generalizes across different sensor configurations and body locations, including those commonly used in clinical settings and everyday wearable devices. These findings demonstrate the effectiveness of transferring visual knowledge in improving sensor-based gait assessment performance, offering a new promising pathway for addressing data scarcity in the sensor and medical domains.
Verbatim fromarXiv· arXiv metadata (abstracts are released under CC0)
How it works
Datasets linked in the hub; pretraining vs evaluation roles follow the paper.
Pretraining data
Continued MAE pretraining (mask ratio 0.75) on ~4,000 participants / 1.23 M 5 s gait windows from 7 datasets (GSTRIDE, FoGContest, COMPWALK-ACL, mPower, WearGait-PD, Mobilise-D, ClinicalGait)
Architecture
MAE with ViT-Tiny backbone initialised from ImageNet MAE-Lite (mae_tiny_400e), sensor-to-image embedder; gyro channels randomly masked (p=0.5) so accel-only input works
Cite the publication recorded for this model. Check it against the publisher before submitting.
Cai, Y., & Hong, Z. (2026). GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai & Hong; arXiv 2611.01684). arXiv 2611.01684v1. https://arxiv.org/abs/2611.01684
Cai, Yize and Zhiqing Hong. 2026. "GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai & Hong; arXiv 2611.01684)." arXiv 2611.01684v1. https://arxiv.org/abs/2611.01684.
Cai, Y. and Hong, Z. (2026) 'GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai & Hong; arXiv 2611.01684)', arXiv 2611.01684v1. Available at: https://arxiv.org/abs/2611.01684
Cai, Yize, and Zhiqing Hong "GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai & Hong; arXiv 2611.01684)." arXiv 2611.01684v1, 2026. https://arxiv.org/abs/2611.01684.
Y. Cai and Z. Hong, "GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai & Hong; arXiv 2611.01684)," arXiv 2611.01684v1, 2026, [Online]. Available: https://arxiv.org/abs/2611.01684.
@misc{cai2026gaitfm,
title = {{GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment (Cai \& Hong; arXiv 2611.01684)}},
author = {Cai, Yize and Hong, Zhiqing},
year = {2026},
howpublished = {arXiv 2611.01684v1},
eprint = {2611.01684},
archivePrefix = {arXiv}
}
History
Added to the ChatPPG research hub
Newsletter
Get new PPG datasets, papers and models
Get alerts when GaitFM is benchmarked on new datasets, plus the weekly digest.
New datasets, papers and models
Practical guides from the hub
Unsubscribe anytime
The waveform is an illustration of a PPG pulse, not a recording.
