Evaluation of Health Navigator Triage Symptom Checker Using 45 Standardized Clinical Vignettes

 

Authors:

  • Hannah L Semigran 1
  • David A Thompson 2

1 Harvard Medical School, Boston MA

2 Northwestern University, Northwestern University Feinberg School of Medicine

Objective: To determine the triage accuracy of an online symptom checker (tool that uses computer algorithms to help patients with self diagnosis and/or self triage).

Design: Audit study.

Setting: A privately available symptom checker.

Methods: We used 45 standardized patient vignettes (BMJ. 2015 Jul 8;351:h3480; “BMJ Authors”) to test the Health Navigator triage symptom checker. These vignettes had already been categorized by the BMJ Authors into three levels of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection). A single reviewer (Semigran) tested the Health Navigator symptom checker against the patient vignettes, collected the data, and analyzed the results.

Main outcome measures: We compared the Health Navigator triage disposition (recommended level of care) to the triage dispositions recommended by the BMJ authors. Four outcomes were possible: exact agreement, over-referral (Health Navigator recommended a higher, more conservative, level of care), under-referral (Health Navigator recommended a lower level of care), and Not available (Health Navigator was unable to provide a triage recommendation). We also compared the results for Health Navigator triage testing to the results from the prior BMJ study’s listing of the seven best performing triage symptom checkers.

Results: Comparison results between Health Navigator and the BMJ Authors showed 73% exact agreement, 22% over-referral, and 4% under-referral. Health Navigator was able to provide a triage recommendation for all vignettes (0% not available). In comparison to other triage symptom checkers, Health Navigator had the highest exact agreement with the BMJ authors. Six of the seven other triage symptom checkers had 1 to 5 vignettes for which they were unable to provide responses.

Conclusions: The Health Navigator achieved a high level of agreement on triage testing against 45 standardized patient vignettes. Triage performance was higher than previously studied triage symptom checkers.

The chart compares the results for Health Navigator (left-most stacked bar; Semigran) from the current study to results for all other triage symptom checkers (Semigran, et.al. BMJ 2015). The stacked bars are sorted from left (highest triage performance) to right (lowest performance).

Citation:  Semigran HL, Thompson DA. Evaluation of Health Navigator Triage Symptom Checker Using 45 Standardized Clinical Vignettes.  March 2017. Health Navigator, LLC.

Related References

  • Semigran HL. Evaluation of Health Navigator Diagnosis Symptom Checker Using 45 Standardized Clinical Vignettes.  March 2017. Abstract and Chart.
  • Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study.  BMJ. 2015 Jul 8;351:h3480. PubMed Abstract. Full Text. Download the 45 clinical vignettes (PDF) used in this research study.

Related Links

View Research and References.