wHuBERT: Layer Selection for Iterative Masked-Prediction Pre-training on Low-Resolution Wearable Biosignals

Anonymous (ICLR 2027 double-blind submission)

Summary

Conformer encoder pretrained with HuBERT-style iterative masked prediction on 25 Hz PPG + accelerometry. Shows the choice of clustering layer for iteration 2 matters and is expensive to validate; proposes per-layer probes at checkpoints; label-free representation statistics do not reliably pick the layer. On two public sleep corpora the teacher and a distilled student both exceed a consumer-device sleep-staging baseline.