SARS: finding a deadly needle in the haystack

Commentary

CJEM 2003;5(6):392-393

Fast-tracked article. Early release, published at www.caep.ca/cjem on Oct. 22, 2003.

See related article: EM Advances

SARS (severe acute respiratory syndrome) hit Toronto just as we were dealing with a local flu outbreak and preparing for a million summer tourists from around the world. We knew that if SARS arrived, our typically overcrowded Canadian emergency departments (EDs) would be in the line of fire, but nothing in our training or daily work prepared us for such a deadly and contagious agent.

Like everyone else around the world, we started retrofitting for SARS and instituted a screening tool at triage. The only one available was adapted by Health Canada from the World Health Organization (WHO) case definition.¹ We soon developed a fuzzy sense of security, believing that, if SARS appeared in our ED, the screening tool would help us. We also did our best to make Health Canada's recommendations fit work life in our ED.² In the end, SARS did not arrive. Like many other EDs around the world, we were lucky: we survived. But some of our health care colleagues in Ontario lost their lives. The complexity of the problem overwhelmed everybody.

In this issue, Wong Wing Nam and colleagues (see page 384) present a retrospective study in which they compared their ability to diagnose SARS with the WHO criteria for SARS case definitions.³ Retrospective audits of charted data are not as reliable as prospective data collection, but the world was not prepared to study an outbreak like SARS; therefore, this study is among the best evidence available to determine whether the WHO criteria would have worked.

Their results are chilling. In a short time, 323 of Hong Kong's Amoy Garden residents contracted SARS and 37 died. The United Christian Hospital (UCH) managed 821 people with SARS-like symptoms. In the UCH ED, physician judgement performed significantly better than the WHO criteria. Their data show that patients with SARS often presented without fever or respiratory symptoms, that 1:8 had a normal chest x-ray, and that nearly 1:10 had no clear contact risk. Even though gastrointestinal symptoms were uncommon, they still occurred in 1:30 to 1:50 cases, and it only takes 1 patient to start a secondary outbreak. Had the virus come to our ED, and had we relied on WHO criteria alone (like EDs everywhere), our screening process could have failed, we could have missed many early diagnoses, and we could have failed to identify SARS victims before our ED staff, patients and visitors became infected.

Health Canada data show that most Canadian victims caught the disease in a health care setting, and, so far, all the Ontario cases trace back to a single admitted patient who had been forced to wait 24 hours in a Toronto ED for an inpatient bed.^4,5 This could have happened in any ED, anywhere. In the end, the screening tools that WHO and our governments threw together on short notice during those early weeks may not have served us well.

ED staff who make up the front-line defence against potentially lethal infectious diseases like SARS may become health care's cannon fodder if we are sent into battle with inadequate tools. We need to know, with a high level of certainty, how to recognize patients with the disease when they appear at the triage desk -- not after staff, visitors and other patients have become infected. If the right tools have not arrived, we need to understand the limitations of the tools we are told to use. Based on the Hong Kong group's findings, we did not have this understanding during the SARS outbreaks, and we will be equally unprepared for the next outbreak.

One solution is to prepare, on an international level, to launch prospective ED-based data collection and research within days of the next outbreak. This will involve hiring and deploying trained research staff to gather the vital information. International jet travel makes outbreaks like SARS a global concern as soon as they begin anywhere on the planet, so the cost of that research should be shared internationally.

Retrofitting Canadian EDs and preparing for the next big outbreak is an expensive ongoing challenge, but SARS and agents like it will strike again. We can redesign triage areas and equip them with negative pressure rooms, but this is not enough. We cannot afford to send doctors and nurses into dangerous situations without the tools necessary to recognize the disease. This study proved that the WHO criteria are an inadequate screening tool and that human judgement is critical, but human judgement is also imperfect. SARS slipped past many nurses and doctors unrecognized because it looked like any of a dozen more benign illnesses. Finding one patient with SARS among many with nonspecific viral illnesses is like looking for a deadly needle in a haystack; therefore, it is critical that we develop rapid and accurate diagnostic tools to supplement clinical judgement.

A key problem in evaluating the utility of the WHO criteria is the prevalence bias that affects the performance of screening tools and diagnostic tests. In low prevalence situations (e.g., when a few patients with SARS presented among thousands of people without SARS, as was the case in British Columbia), then even a coin flip -- although it is a useless diagnostic test having 50% sensitivity and 50% specificity -- will appear to have excellent negative predictive value (NPV). Virtually every time the coin comes up tails ("negative") it will be correct because almost no one in the population being screened actually has the disease. Conversely, in this low prevalence (1:1000) situation, its positive predictive value (PPV) is dismal, and only 0.1% of the positives are true positives.⁶

A test that is 90% sensitive and 95% specific (e.g., the UCH physicians' clinical judgement) has virtually perfect NPV when disease prevalence is only 1:1000, but in this low prevalence group its PPV is still poor, with only 1.8% of "positives" being true positives. And although 90% sensitivity is considered good, 1 of every 10 SARS patients -- an unacceptable number -- would still be missed.

In the Hong Kong group's study, the WHO criteria were only 42% sensitive and 86% specific. Based on 42% sensitivity, they would miss 58% of patients with SARS, yet they would still have excellent NPV (99.9%) when applied in a population with a disease prevalence of 1:1000. Again, this is because only 1 in 1000 people actually has the disease in question, so any test would look like an excellent negative predictor. This is why physicians who apply weak diagnostic tests to very low prevalence populations often develop the incorrect belief that the tests are useful in ruling out disease. Despite reasonable (86%) specificity, if the WHO criteria are used in a similarly low prevalence (1:1000) population, PPV is only 0.3%, meaning that 99.7% of patients who meet criteria do not, in fact, have the disease.

We must devise the right tools and procedures to identify every SARS patient before they infect others. This will be difficult and, until we do, we must acknowledge that our current screening system is flawed. We breathed a sigh of relief when the outbreaks ended in Ontario and Asia, but we cannot afford to be complacent.

References

1. Population and Public Health Branch, Health Canada. Severe acute respiratory syndrome (SARS) case definitions. Available: www.hc-sc.gc.ca/pphb-dgspsp/sars-sras/sarscasedef_e.html (accessed 2003 Oct 10).
2. CAEP (Canadian Association of Emergency Physicians) emergency department SARS guidelines. (accessed 2003 Oct 15).
3. Wong WN, Sek ACH, Lau RFL, Li KM, Leung JKS, Tse ML, et al. Accuracy of clinical diagnosis versus the World Health Organization case definition in the Amoy Garden SARS cohort. Can J Emerg Med 2003;5(6):384-91.
4. Ovens H, Thompson J, Lyver M, Murray MJ, Innes G, on behalf of the Canadian Association of Emergency Physicians (CAEP). Implications of the SARS outbreak for Canadian emergency departments [position statement]. Can J Emerg Med 2003;5(5):343-7.
5. Epidemic curve of a SARS outbreak in Canada, February 23 to 2 July, 2003 [figure]. Available: www.hc-sc.gc.ca/pphb-dgspsp/sars-sras/pdf-ec/ec_20030808.pdf (accessed 2003 Oct 10).
6. Medical College of Wisconsin, Division of General Internal Medicine. Online clinical calculator: Bayesian Analysis Model. Available: www.intmed.mcw.edu/clincalc/bayes.html (last updated 3/9/99; accessed 2003 Sept 20).