This is a plain English summary of an original research article. The views expressed are those of the author(s) and reviewer(s) at the time of publication.
Relatively little robust evidence exists on the impact of online or app-based health assessments for people seeking urgent care. The available evidence suggests that these services are not as good at making diagnoses as health professionals; though it is not clear whether this is a useful benchmark.
The ‘NHS 111 Online’ service for symptom checking and triage is available in England. It aims to reduce pressure on the equivalent telephone-based service. This NIHR-funded systematic review looked at the international evidence on this and similar digital services.
The review found inconsistent evidence on digital services’ ability to direct patients to appropriate care and to reduce service use and costs. However, this was based on relatively small and short-term studies. Those who use these digital health assessments, often younger and more highly educated individuals, are usually satisfied with them.
This review highlights the need for further assessment of these rapidly developing services.
Why was this study needed?
NHS 111 is a heavily used service providing telephone-based assessment and triage for health problems that are urgent but not considered an emergency. In November 2019 the service fielded nearly 1.6 million calls, an average of almost 53,000 per day. To reduce demand for the telephone-based service a digital version was recently launched, accessible via a website or smartphone app.
Achieving a high level of diagnostic or triage accuracy is crucial to the success of the digital NHS 111 service. However, some studies have shown variations in the accuracy of diagnoses and triage advice provided by NHS 111 and limited success in reducing emergency 999 calls. This has the potential to be counterproductive by increasing pressure on existing services.
This review aimed to identify the effect of digital symptom assessment and signposting for urgent health problems has on important clinical and health service outcomes.
What did this study do?
This NIHR-funded systematic review included 27 studies that evaluated online services designed to assess symptoms, provide health advice and direct patients to relevant care. Most studies were conducted in the USA or UK.
Only two studies were randomised; most of the remainder were observational or based on simulations using clinical vignettes or past patient records. The studies mostly compared digital services with health professionals.
Seventeen of the studies assessed stand-alone symptom checkers, five assessed them as part of an intervention where follow-up with a health professional could be arranged if needed. Five studies assessed patient attitudes to the services.
The overall quality of evidence was relatively weak as it was largely derived from observational studies and included some non-peer reviewed literature (seven studies). Variation between the studies meant that pooling of their results could not be carried out.
What did it find?
- No evidence was identified to suggest digital symptom checkers are unsafe. However, this was based on only six studies and most were relatively small and short-term so there is weak confidence in this outcome.
- Seven out of eight studies found that online symptom checkers are less accurate at diagnosis than health professionals. For example, one study compared 23 different symptom checkers versus doctors using 43 patient vignettes. Symptom checkers only selected the correct diagnosis in 34% of cases, compared with 72% accuracy from the physicians. There was wide variation in accuracy across the different symptom checkers and different conditions in this study.
- The six studies that assessed signposting to appropriate services had mixed results. Most found that symptom checkers tended not to be as good as health professionals at triage. The symptom checkers were more cautious in the advice they gave. While a study of one online service found a high level of accurate triage using patient vignettes (88% compared with 76% for doctors and 74% for nurses), this study was carried out by the company that developed the system and it was not published in a peer-reviewed journal.
- The six studies that assessed the impact on service use and diversion also had mixed findings so firm conclusions could not be drawn. There was also insufficient evidence about the clinical effectiveness of the symptom checkers (two studies identified), patient compliance with the advice given (two studies), or costs and cost-effectiveness (three studies identified).
- Younger and more highly educated people were the most common users of digital symptom checking services. Patients who have used the services generally have high levels of user satisfaction.
What does current guidance say on this issue?
There are currently no specific national guidelines relating to the use of online symptom checking services. The online NHS 111 service is available in England for people aged five years old and above. It has been reported that this service has been used over a million times since first being available in 2017.
What are the implications?
People in England with urgent but non-emergency care needs already have the option to access NHS 111 either online or on the telephone.
This review suggests that further evaluation is needed to assess the impact of this and other online triage services, to ensure that they are having the desired effects.
Citation and Funding
Chambers D, Cantrell AJ, Johnson M et al. Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review. BMJ Open. 2019;9(8):e027743.
This project was funded by the NIHR Health Services and Delivery Research Programme (project number 16/47/17).
Bibliography
Chambers D, Cantrell A, Johnson M et al. Digital and online symptom checkers and assessment services for urgent care to inform a new digital platform: a systematic review. Health Serv Deliv Res. 2019:7(29).
NHS England. NHS 111 Minimum Data Set 2019-20. Leeds: NHS England; 2019.
Semigran HL, Linder JA, Gidengil C and Mehrotra A. Evaluation of symptom checkers for self-diagnosis and triage: audit study. BMJ. 2015;351:h3480.
Turner J, O’Cathain A, Knowles E and Nicholl J. Impact of urgent care telephone service NHS 111 pilot sites: a controlled before and after study. BMJ Open. 2013;3(11):e003451.
Produced by the University of Southampton and Bazian on behalf of NIHR through the NIHR Dissemination Centre
NIHR Evidence is covered by the creative commons, CC-BY licence. Written content and infographics may be freely reproduced provided that suitable acknowledgement is made. Note, this licence excludes comments and images made by third parties, audiovisual content, and linked content on other websites.
