Evidence
Alert

Artificial intelligence tool rules out COVID-19 within an hour in emergency departments

Researchers have developed an artificial intelligence (AI) tool for rapidly detecting COVID-19 in people arriving at a hospital’s emergency department. The tool can accurately rule out infection within an hour of a patient arriving at hospital, significantly faster than the PCR (polymerase chain reaction) test that has a turnaround time of typically 24 hours. 

Widespread use of this tool could reduce the spread of COVID-19 in hospitals and avoid delays to treatment for people who are not infected with the virus. 

At the time of its publication, the research included the largest hospital dataset of any AI study on COVID-19. The models developed by the research team are based on data from more than 115,000 people attending hospital, and five million measurements.

What’s the issue?

At the peak of the second wave of the COVID-19 pandemic, in a single day (January 12, 2021), 4,578 people were admitted to UK hospitals with COVID-19. The illness shares symptoms such as fever and coughing with a number of other respiratory illnesses. This makes it difficult for clinicians to identify and isolate people are are infected. 

The current gold standard test for the virus is a PCR test using a swab of the nose and throat. But PCR tests do not detect all infections, typically take around 24 hours to process, and require specialist laboratory tools and expertise. More recently, lateral flow testing has been widely adopted in hospitals, but there are concerns that these tests may miss some infections.  

A faster and more accurate way of detecting the illness would mean that infected people arriving at hospital could be kept separate from non-infected people. This would reduce the spread of COVID-19 and allow non-infected people to be treated more quickly.

Researchers set out to develop a tool which can rapidly detect COVID-19 in people arriving at emergency departments, and among a subset who were subsequently admitted to hospital. They designed a tool based on artificial intelligence (AI) which uses only the data that is already routinely collected within a patients’ first hour in hospital.

What’s new?

The study, at two Oxford University Hospitals, included health data on more than 115,000 people. This is the data that is routinely collected when people are admitted to hospital: body temperature, blood pressure, heart rate, and blood tests. The research team also gathered PCR test results for the COVID-19 virus, SARS-CoV-2.

People in the study had either arrived at hospital before the pandemic began and therefore were certain to be COVID-19-negative; or were test-confirmed to have the illness in the early months of the pandemic and were certain to be COVID-19-positive. The researchers used their data to train machine learning algorithms. They looked at how reliably AI models could tell whether a person has COVID-19 at different stages of a pandemic with different levels of COVID-19 in the population. 

The AI test was then evaluated by applying it to all patients arriving at the emergency departments of the two hospitals during a two-week period.  

For 3,326 patients arriving at hospital, and 1,715 patients admitted, the AI models effectively identified people with COVID-19 using data typically available within an hour of admission to hospital. They reliably ruled out COVID-19 in both groups. 

During the two-week test period, the AI models:

    • agreed with the PCR result in more than nine in ten cases overall (92% accuracy)
    • produced negative results that were almost always (98%) correct.

The PCR test is not always accurate. When this was taken into account, the models were even more accurate. 

The models performed similarly across ethnic groups (White British versus Black, Asian and minority ethnic groups), genders, and ages (adults under 60 years versus those over 60).

Why is this important?

These AI models could be used to screen for COVID-19 in hospitals to help guide care and stream patients. The vast majority of people arriving at hospital who are COVID-19-negative according to the models could then be treated while awaiting confirmation from the hospital laboratory. When the models suggest someone does have COVID-19, they could await test results and undergo treatment away from other patients.

This approach uses data that is already routinely taken by hospital clinicians during the first hour of a patient’s arrival, meaning additional costs are minimal. The tool fits into existing laboratory testing infrastructure and approach to care in hospitals in high-and middle-income countries. The models can also be adjusted to work effectively at the different stages of a pandemic.

The study provides proof that AI-based screening tools could be useful for screening people in future pandemics.

What’s next?

Since this study, the research team have started investigating whether AI models to predict COVID-19 status could be used alongside a blood analyser that requires two drops of blood from a finger and does not require blood samples to be processed in the hospital lab. This optimised version of the AI screening tool promises faster results. The clinical team are using it at the John Radcliffe Hospital in Oxford to triage patients at the front door and help the flow of patients into various designated areas. Results are expected later this year.

The research team is also collaborating with University Hospitals Birmingham, Portsmouth University Hospitals, and Bedfordshire Hospitals, to test out the tool’s performance in real world hospital conditions. 

You may be interested to read

The full paper: Soltan AAS, and others. Rapid triage for COVID-19 using routine clinical data for patients attending hospital: development and prospective validation of an artificial intelligence screening test. Lancet 2021;3 

An industry insight article about the team’s ongoing research from Technology Networks: Combining AI and Point-of-Care Diagnostics for Rapid COVID-19 Screening  

 

Funding: This research was supported by the NIHR Oxford Biomedical Research Centre and NIHR Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at the University of Oxford. 

Conflicts of Interest: Two authors have received personal fees from pharmaceutical and digital health companies.

Disclaimer: NIHR Alerts are not a substitute for professional medical advice. They provide information about research which is funded or supported by the NIHR. Please note that views expressed in NIHR Alerts are those of the author(s) and reviewer(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Commentaries

Study author

We’ve just finished validating our model across emergency departments in three other NHS trusts. We are getting more confident at being able to reliably and quickly pick out which patients do not have COVID-19. We’ve also been working on our model over the past year to make a version that works without needing a hospital laboratory – that uses just clinical data that can be collected at the patient’s bedside. 

What that means is you save time. Unlike a PCR test, you don’t have to take and label an extra sample, transport it to the lab, and wait for results from lab processing.

Following on from a successful vaccination programme, our new data makes a strong case for using our tool to reduce the number of PCR swabs needed and improve flow in hospitals. As COVID-19 becomes less common, perhaps we will be able to save money by selecting out a small population of patients at higher risk of testing positive for lab PCR testing, rather than needing to test everybody being admitted.

In the future, we will also look at repurposing the AI to detect flu and other respiratory viruses. Different pandemics will behave in different ways, but this is proof of concept that you can very quickly use data to develop a screening model and improve flow in hospitals. The biggest challenges are the logistical aspects, such as getting digital infrastructure into hospitals, making your model compatible with hospital systems, and ensuring the governance is done properly.

Andrew Soltan, NIHR Academic Clinical Fellow, University of Oxford 

Member of the public

I am a member of a committee that is creating living guidelines for the diagnosis and management of COVID-19. This seems to be one of the most robust and extensive clinical AI studies in this context. 

Although the study indicates that the AI screening tool showed faster, more accurate results compared to gold standard PCR tests, further research is needed to see the true impact. The test needs to be scrutinised as part of existing clinical pathways in a greater number of hospitals within the UK. The test’s real-world performance should also be assessed in more ethnically diverse populations.

It is important to establish how much confidence clinical staff have in the tool and their willingness to use it and accept its results. Commissioners need to explore the cost of integrating it into existing clinical pathways and evaluate the overall financial impact.

Stella O’Brien, Public contributor, Essex

Clinician 

This research may be useful for allocating resources and triaging patients in future pandemics.

I would like to see the model trialled in different settings and hospitals with different ethnic populations, in patients without previous admissions, and in people with defined comorbidities (post-transplant, for example). It will be important to establish that critically ill patients, with or without COVID, do not suffer harm from having their symptoms assessed by an AI model and not a human.

The tool should be tested during the winter to explore whether other respiratory viruses with symptoms that may mimic COVID-19, have an impact on the model’s accuracy.

Jaisi Sinha, Consultant  Microbiologist and Virologist, Public Health Wales Microbiology Cardiff, University Hospital of Wales