Evaluation of Health Navigator Diagnosis Symptom Checker Using 45 Standardized Clinical Vignettes

 

Authors:

  • Hannah L Semigran 1

1 Harvard Medical School, Boston MA

Objective: To determine the diagnostic accuracy of an online symptom checker (tool that uses computer algorithms to help patients with self diagnosis and/or self triage).

Design: Audit study.

Setting: A privately available symptom checker.

Participants: A symptom checker that was in English and provided advice across a range of conditions. 45 standardized patient (BMJ. 2015 Jul 8;351:h3480.) vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).

Main outcome measures: Our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses.

Results: The symptom checker provided the correct diagnosis first in 55.6% (95% confidence interval 40.5% to 70.7%) of SP evaluations and listed the correct diagnosis within the top 20 diagnoses given in 95.6% (95% CI 89.3% to 100.0%) of SP evaluations (P=0.1). Performance on diagnostic accuracy did not vary significantly across the three acuity levels of the SP evaluations. The symptom checker provided the correct diagnosis within the top 20 given for all of the emergent care required and self-care reasonable SP evaluations and for 86.7% (95% CI 67.2%-100.0%) of the non-emergent care reasonable SP evaluations (P=0.1).

Conclusions: The symptom checker had minimal deficits in diagnosis.

The chart compares the results for Health Navigator (blue bars; Semigran) from the current study to results for all other symptom checkers (grey bars; Semigran, et.al. BMJ 2015).

Citation:  Semigran HL. Evaluation of Health Navigator Diagnosis Symptom Checker Using 45 Standardized Clinical Vignettes.  March 2017. Health Navigator, LLC.

Related References

  • Semigran HL, Thompson DA. Evaluation of Health Navigator Triage Symptom Checker Using 45 Standardized Clinical Vignettes.  March 2017. Abstract and Chart.
  • Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study.  BMJ. 2015 Jul 8;351:h3480. PubMed Abstract. Full Text. Download the 45 clinical vignettes (PDF) used in this research study.

Related Links

View Research and References.