Paperdoi:10.1038/s41746-026-03251-8

Patient journey evaluation for consumer AI health assistants

Venue listedDated Sep 22, 2026Added Nov 19, 2026

  • ZY
  • HY

Authors: Zonghai Yao, Hong Yu

npj Digital Medicine (Perspective, 2026-09-22)

TL;DR

A Perspective arguing that consumer AI health assistants that link records, patient reports and wearable data should be evaluated over whole patient journeys, not single prompts. It proposes a 'personal health workspace' where the assistant is tested as a stateful system: what evidence it used, what it did, what it carried forward, whether corrections persist, how it handles missing or conflicting data, and when it escalates. No experiments.

  • Form factor: consumer LLM health assistant

Abstract

Consumer AI health assistants increasingly connect medical records, patient reports, and wearable data across repeated interactions, but evaluation still focuses on isolated prompts and answers. This Perspective has two objectives: to define a journey-level evaluation and governance framework for record-linked patient education, and to specify how longitudinal reliability should be tested before deployment and monitored afterward. We introduce the personal health workspace (PHW), a user-controlled setting for evaluating an assistant as a stateful system across a patient journey. The main outcomes include a clinical journey map and an auditable interaction loop that show what evidence the assistant uses, what actions it takes, and what it carries forward across sessions. Multi-source and long-horizon stress tests use this structure to assess whether claims remain tied to the right evidence, corrections persist, missing or conflicting data are handled safely, and escalation occurs when needed. The framework also specifies deployment requirements for interoperability, privacy, equity, and human oversight. Together, these outcomes support local validation, release decisions, post-deployment monitoring, and prospective studies of clinical effectiveness.

Verbatim from the publisher's DOI record (opens in a new tab) · Publisher licence: https://creativecommons.org/licenses/by/4.0

Results on ChatPPG leaderboards

All leaderboards

No results from this paper on ChatPPG leaderboards yet. Browse the leaderboards.

Tasks & signals

Keywords

LLMHealth AssistantAgent EvaluationLongitudinal EvaluationGovernancePersonal Health WorkspacePerspective

How it works

How Method works: Setup → Method → TargetSetup: Signals (medical records, patient reports, wearable data (as inputs to an LLM assistant)); Form factor (consumer LLM health assistant). Method: Method (Language-aligned).SetupSignals: medical records, patient reports, wearable data (as inputs to an LLM assistant)Signalsmedical records, patient reports…Form factor: consumer LLM health assistantForm factorconsumer LLM health assistantMethodMethodLanguage-aligned

Cite this paper

Cite this paper with the fields recorded here. Check it against the publisher before submitting.

Yao, Z., & Yu, H. (2026). Patient journey evaluation for consumer AI health assistants. npj Digital Medicine. https://doi.org/10.1038/s41746-026-03251-8
Yao, Zonghai and Hong Yu. 2026. "Patient journey evaluation for consumer AI health assistants." npj Digital Medicine. https://doi.org/10.1038/s41746-026-03251-8.
Yao, Z. and Yu, H. (2026) 'Patient journey evaluation for consumer AI health assistants', npj Digital Medicine. doi: 10.1038/s41746-026-03251-8.
Yao, Zonghai, and Hong Yu "Patient journey evaluation for consumer AI health assistants." npj Digital Medicine, 2026. https://doi.org/10.1038/s41746-026-03251-8.
Z. Yao and H. Yu, "Patient journey evaluation for consumer AI health assistants," npj Digital Medicine, 2026, doi: 10.1038/s41746-026-03251-8.
@article{yao2026patient,
  title         = {{Patient journey evaluation for consumer AI health assistants}},
  author        = {Yao, Zonghai and Yu, Hong},
  year          = {2026},
  journal       = {npj Digital Medicine},
  doi           = {10.1038/s41746-026-03251-8}
}

Added to the ChatPPG research hub