ChatPPG.

Free research tool

Wearable validation toolkit

Paste paired device and reference readings to get the agreement statistics a validation paper needs: Bland–Altman bias and limits of agreement with confidence intervals, a proportional-bias check, ICC, MAE, MAPE and RMSE. Categorical outputs such as sleep stages get Cohen's kappa, and the sample-size tab plans a study with the method of Lu et al. (2016). Everything runs in your browser.

Updated . Free, no sign-up. Calculations run in your browser and your data is never uploaded.

1. Your paired data

One row per paired measurement: the device value and the reference value taken at the same time. Pre-filled with the peak expiratory flow data from Bland & Altman (1986), Table 1. Your data stays in this browser tab; nothing is uploaded.

2. Bland–Altman agreement

n = 17
Bias (mean difference)
−2.1 l/min
95% CI −22.0 to 17.8
Lower limit (95%)
−78.1 l/min
95% CI −112.9 to −43.3
Upper limit (95%)
73.9 l/min
95% CI 39.1 to 108.6
SD of differences
38.8 l/min
Proportional bias slope
0.029
95% CI −0.159 to 0.217; p = 0.749
Proportional bias?
Not detected
Regression of differences on means
300400500600−100−50050100Mean of wright and mini_wright (l/min)wright − mini_wright (l/min)+1.96 SD73.9Mean−2.1−1.96 SD−78.1

3. Error metrics and ICC

MAE
28.9 l/min
MAPE
7.5%
vs reference
RMSE
37.7 l/min
Pearson r
0.943
Association, not agreement
Intraclass correlation coefficients
FormModelICC95% CIF (df)p
ICC(1,1)One-way random effects, absolute agreement, single measurement0.9460.861 to 0.98036.05 (16, 17)< 0.001
ICC(2,1)Two-way random effects, absolute agreement, single measurement0.9460.857 to 0.98034.03 (16, 16)< 0.001
ICC(3,1)Two-way mixed effects, consistency, single measurement0.9430.850 to 0.97934.03 (16, 16)< 0.001

ICC(2,1) = 0.946. Koo and Li (2016) suggest judging reliability on the confidence interval: here it runs from good to excellent (<0.5 poor, 0.5–0.75 moderate, 0.75–0.9 good, >0.9 excellent). An ICC describes reliability relative to the spread between subjects, so it can be high even when the bias is clinically large; report it with the Bland–Altman limits.

4. Methods paragraph

Agreement between wright and mini_wright (n = 17 paired measurements) was assessed with Bland–Altman analysis (Bland & Altman, 1986). The mean difference (wright minus mini_wright) was −2.1 l/min (95% CI −22.0 to 17.8), and the 95% limits of agreement were −78.1 l/min (95% CI −112.9 to −43.3) to 73.9 l/min (95% CI 39.1 to 108.6); confidence intervals for the limits used the approximate standard error of Bland and Altman (1999). Proportional bias was assessed by regressing the differences on the pair means (slope 0.029, 95% CI −0.159 to 0.217, p = 0.749). Mean absolute error was 28.9 l/min, root-mean-square error 37.7 l/min and mean absolute percentage error 7.5% relative to mini_wright. ICC(2,1) (two-way random effects, absolute agreement, single measurement) was 0.946 (95% CI 0.857 to 0.980) (Shrout & Fleiss, 1979; McGraw & Wong, 1996). Analyses were run in the ChatPPG Wearable Validation Toolkit (version 1.0; https://chatppg.com/tools/wearable-validation-toolkit).

Check the wording against your study design before you use it, and report how pairs were time-aligned.

Cite this analysis

Cite the methods papers for the statistics you report, and the tool if it helped. ChatPPG. Wearable Validation Toolkit (version 1.0). 2026. https://chatppg.com/tools/wearable-validation-toolkit. Accessed 2026-10-18.

Preview BibTeX
@software{chatppg_wearable_validation_toolkit_2026,
  author = {{ChatPPG}},
  title = {{Wearable Validation Toolkit}},
  year = {2026},
  version = {1.0},
  publisher = {ChatPPG},
  url = {https://chatppg.com/tools/wearable-validation-toolkit},
  urldate = {2026-10-18}
}

@article{bland1986,
  author = {Bland, J. Martin and Altman, Douglas G.},
  title = {{Statistical methods for assessing agreement between two methods of clinical measurement}},
  journal = {The Lancet},
  year = {1986},
  volume = {327},
  number = {8476},
  pages = {307--310},
  doi = {10.1016/S0140-6736(86)90837-8}
}

@article{bland1999,
  author = {Bland, J. Martin and Altman, Douglas G.},
  title = {{Measuring agreement in method comparison studies}},
  journal = {Statistical Methods in Medical Research},
  year = {1999},
  volume = {8},
  number = {2},
  pages = {135--160},
  doi = {10.1177/096228029900800204}
}

@article{shrout1979,
  author = {Shrout, Patrick E. and Fleiss, Joseph L.},
  title = {{Intraclass correlations: uses in assessing rater reliability}},
  journal = {Psychological Bulletin},
  year = {1979},
  volume = {86},
  number = {2},
  pages = {420--428},
  doi = {10.1037/0033-2909.86.2.420}
}

@article{mcgraw1996,
  author = {McGraw, Kenneth O. and Wong, S. P.},
  title = {{Forming inferences about some intraclass correlation coefficients}},
  journal = {Psychological Methods},
  year = {1996},
  volume = {1},
  number = {1},
  pages = {30--46},
  doi = {10.1037/1082-989X.1.1.30}
}

@article{koo2016,
  author = {Koo, Terry K. and Li, Mae Y.},
  title = {{A guideline of selecting and reporting intraclass correlation coefficients for reliability research}},
  journal = {Journal of Chiropractic Medicine},
  year = {2016},
  volume = {15},
  number = {2},
  pages = {155--163},
  doi = {10.1016/j.jcm.2016.02.012}
}

What each statistic answers

Bland–Altman analysis answers "how far apart are the device and the reference, and in which direction?". The bias is the mean difference, and the 95% limits of agreement (bias ± 1.96 SD of the differences) are the range expected to hold about 95% of individual differences (Bland & Altman, 1986). The confidence intervals around the limits show how precisely your sample pins them down (Bland & Altman, 1999).

The intraclass correlation coefficient (ICC) describes reliability: how much of the total variation comes from real differences between participants rather than disagreement between methods (Shrout & Fleiss, 1979). MAE, MAPE and RMSE summarise the typical size of the error in plain units, which is how most consumer wearable accuracy studies report results.

For categories (sleep stages, rhythm labels), Cohen's kappa measures agreement beyond what chance alone would produce (Cohen, 1960).

Why correlation is not enough

Two methods can be perfectly correlated and still disagree, for example if the device always reads 10 bpm high. Bland and Altman (1986) made this point in the paper that introduced limits of agreement, and it is why this toolkit reports Pearson's r only as a secondary measure of association.

How this toolkit was validated

Every formula is unit-tested against reference implementations: R packages irr, psych and SimplyAgree, and Python's SciPy, pingouin and scikit-learn. It reproduces the published worked examples in Bland & Altman (1986), Shrout & Fleiss (1979) and Lu et al. (2016). The test suite and the scripts that generate the reference values are in the ChatPPG repository.

Before you report

This version treats each row as an independent pair. If you collected several pairs per participant, the limits need a method that accounts for within-participant correlation (Bland & Altman, 1999). Time-align device and reference samples first, and say in the paper how you did it. Then compare your result with published validations of the same device in the wearable accuracy tables and the research hub.

Frequently asked questions

Which statistics should a wearable validation study report?

For continuous measures such as heart rate, report the Bland–Altman bias and 95% limits of agreement with their confidence intervals, plus an error summary such as MAE or MAPE. Add an ICC if reliability across participants matters. For categorical outputs such as sleep stages, report overall agreement, Cohen's kappa with its confidence interval, and the confusion matrix.

What is the difference between bias and limits of agreement?

Bias is the average difference between device and reference. Limits of agreement (bias ± 1.96 SD of the differences) show the range expected to contain about 95% of individual differences, so they describe how far a single reading can be off.

How many participants do I need?

Use the sample-size tab. It implements Lu et al. (2016): given the expected mean and SD of the differences and a pre-specified maximum allowed difference, it finds the smallest number of pairs that gives the chosen power for the confidence intervals of both limits to fall inside that range.

Is my data uploaded anywhere?

No. The calculations run in your browser with JavaScript. Pasted or uploaded files are read locally and never sent to ChatPPG or anyone else, so you can use it with unpublished study data.

How do I cite the toolkit?

Use the BibTeX or RIS export under each analysis. It includes the toolkit and the methods papers for the statistics you report, such as Bland & Altman (1986) for limits of agreement and Shrout & Fleiss (1979) for the ICC.

References

  1. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet. 1986;327(8476):307-310. doi:10.1016/S0140-6736(86)90837-8
  2. Bland JM, Altman DG. Measuring agreement in method comparison studies. Statistical Methods in Medical Research. 1999;8(2):135-160. doi:10.1177/096228029900800204
  3. Lu MJ, Zhong WH, Liu YX, et al.. Sample size for assessing agreement between two methods of measurement by Bland-Altman method. The International Journal of Biostatistics. 2016;12(2):20150039. doi:10.1515/ijb-2015-0039
  4. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420-428. doi:10.1037/0033-2909.86.2.420
  5. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996;1(1):30-46. doi:10.1037/1082-989X.1.1.30
  6. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012
  7. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20(1):37-46. doi:10.1177/001316446002000104
  8. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin. 1968;70(4):213-220. doi:10.1037/h0026256
  9. Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin. 1969;72(5):323-327. doi:10.1037/h0028106
  10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174. doi:10.2307/2529310

Related

More free tools

Bland–Altman calculator and plotICC calculatorCohen's kappa calculatorBland–Altman sample size calculatorSpO₂ at altitude checkerAll tools