A brief note about likelihood ratios

Pedagogical Tools and Methods

CJEM 2008;10(5):441-442

In this issue of the journal, Azzam and colleagues1 review the validation study of a clinical prediction rule for termination of resuscitation in patients suffering an out-of-hospital cardiac arrest.2 The clinical prediction rule is similar to a diagnostic test in that it predicts whether or not the patient will suffer the disorder of interest, in this case death, before hospital discharge. Given this, the results can be expressed using summary measures that are typically used for diagnostic efficacy. The most familiar of these are sensitivity and specificity, terms that have been adopted into clinical medicine from laboratory research on assays for the detection of substances.3 Application of the sensitivity to the clinical prediction rule refers to the percentage of patients that die before discharge who met the clinical prediction rule criteria. Clinically, this is counterintuitive since what we really want to know is what percentage of those who met the criteria will die before being discharged. This is the positive predictive value. The greatest problem with predictive values is that they are dependent on the prevalence of the disorder in the population in which the measurements are performed; therefore, they have limited generalizability, that is, limited clinical applicability outside of the study population.4

The solution to both of these problems, and perhaps the best single measure of a diagnostic test's strength, is the likelihood ratio (LR). As its name implies, the LR is a ratio of probabilities: the probability of a given test result (all clinical prediction rule criteria, met or not) among people who died divided by the probability of that test result among people who survived. The LR employs a mathematical relationship known as Bayes Theorem to indicate how much a given diagnostic test result will raise or lower the pretest probability of the patient having the disorder in question by multiplying the pretest odds by the LR.5,6 This is typically calculated using a nomogram or Web-based program (http://araw.mede.uic.edu/cgi-bin/testcalc.pl) but can easily be done by simply understanding the diagnostic strength of different LR values.7,8 Because the LR is a ratio, a value of 1 indicates that the numerator is the same as the denominator and multiplying the pretest probability by 1 yields a posttest probability of the same value, hence, an LR of 1 has no diagnostic value.9 As a ratio, the diagnostic test (the clinical prediction rule) has greater predictive value as the LR moves away from 1 toward ∞ or toward 0 such that an LR of less than 2 and an LR of greater than 0.5 are each of little clinical value but an LR of greater than 10 and an LR less than 0.1 are both considered conclusive.3,6 It should be noted that positive test results will often be expressed as a positive LR (LR+) and a negative test result as a negative LR (LR-).10 This is based on the assumption that the test result is dichotomous and overlooks one of the advantages of the LR: it can be used for test results with multiple categories.10,11 On the other hand, sensitivity and specificity assume that a diagnostic result is dichotomous. In fact, the interval LR is the slope between 2 points on a receiver operating characteristic curve.11,12

Azzam and colleagues have used the LR to express the results of the termination of resuscitation rule with different pretest probabilities of death and, as we saw, there is little impact on the posttest probability in part due to the inconclusive LR. However, the LR is of greatest diagnostic value when the pretest probability is in the range of 30%-70%.10 In the face of a diagnostic dilemma with a pretest probability of 50%, the same LR of 6.6 reported by Azzam and coauthors will significantly increase the posttest probability to more than 80% and might be considered conclusive. A perceived disadvantage of the LR is the need to determine a pretest probability from which to launch one's Bayesian analysis.13,14 Although pretest probability estimates may be evidence-based, when confronted with the life-or-death decision of not applying cardiopulmonary resuscitation, emergency physicians must ensure that the baseline cardiac arrest survival rates within their populations approximate those in the derivation study before they can confidently apply the yet-to-be-validated termination of resuscitation clinical prediction rule.15 If baseline survival rates are higher within an individual community, emergency medicine clinicians and prehospital health care providers would fail to attain equally low posttest probabilities within their settings.

Perhaps, as Azzam and colleagues have indicated, the measure that emergency medical services directors and other stakeholders are likely most interested in is not the sensitivity, specificity, predictive value or even the LR of the clinical prediction rule, but simply the percentage or proportion of incorrect termination of resuscitation.

References

1. Azzam MH, Lang E, Segal E. Terminating resuscitation for out-of-hospital cardiac arrest? CJEM 2008;10:474-7.
2. Morrison LJ, Visentin LM, Kiss A, et al. Validation of a rule for termination of resuscitation in out-of-hospital cardiac arrest. N Engl J Med 2006;355:478-87.
3. Hayden SR, Brown MD. Likelihood ratio: a powerful tool for incorporating the results of a diagnostic test into clinical decision making. Ann Emerg Med 1999;33:575-80.
4. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ 1994;309:102.
5. Sackett DL, Straus S, Richardson WS, et al. Evidence-based medicine. How to practise and teach EBM. 2nd ed. Edinburgh: Churchill Livingstone; 2000: 67-93.
6. Jaeschke R, Guyatt GH, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994;271:703-7.
7. Fagan TJ. Nomogram for Bayes Theorem. N Engl J Med 1975;293:257.
8. Sotos JG. Simplified calculations using likelihood ratios. ACP J Club 2007;146:A10.
9. Perera R. Making sense of diagnostic tests with likelihood ratios. ACP J Club 2007;146:A8-9.
10. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ 2004;329:168-9.
11. Brown MD, Reeves MJ. Interval likelihood ratio: another advantage for the evidence-based diagnostician. Ann Emerg Med 2003;42:292-7.
12. Gallagher EJ. The problem with sensitivity and specificity. Ann Emerg Med 2003;42:298-303.
13. Phelps MA, Levitt A. Pretest probability estimates: A pitfall to the clinical utility of evidence-based medicine? Acad Emerg Med 2004;11:692-4.
14. Scott IA. Estimating disease likelihood: a case of rubbery figures. Med J Aust 2004;180:438-9.
15. Richardson WS, Polashenski WA, Robbins BW. Could our pretest probabilities become evidence based? J Gen Intern Med 2003;18:203-8.