Diagnostic accuracy of clinical prediction rules to exclude acute coronary syndrome in the emergency department setting: a systematic review
Erik P. Hess, MD;* Venkatesh Thiruganasambandamoorthy, MD;* George A. Wells, PhD;† Patricia Erwin, MLS;‡ Allan S. Jaffe, MD;§ Judd E. Hollander, MD;** Victor M. Montori, MD, MSc;¶ Ian G. Stiell, MD, MSc*†
From the Departments of *Emergency Medicine and †Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ont., the ‡Mayo Medical Libraries, the §Division of Cardiology, Department of Internal Medicine, and the ¶Knowledge and Encounter Research Unit, Department of Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minn., and the **Department of Emergency Medicine, Hospital of the University of Pennsylvania, Philadelphia, Penn.
Objective: We sought to determine the diagnostic accuracy of clinical prediction rules to exclude acute coronary syndrome (ACS) in the emergency department (ED) setting.
Methods: We searched MEDLINE, EMBASE, Web of Science and the Cochrane Database of Systematic Reviews. We contacted content experts to identify additional articles for review. Reference lists of included studies were hand searched. We selected articles for review based on the following criteria: 1) enrolled consecutive ED patients; 2) incorporated variables from the history or physical examination, electrocardiogram and cardiac biomarkers; 3) did not incorporate cardiac stress testing or coronary angiography into prediction rule; 4) based on original research; 5) prospectively derived or validated; 6) did not require use of a computer; and 7) reported sufficient data to construct a 2 × 2 contingency table. We assessed study quality and extracted data independently and in duplicate using a standardized data extraction form.
Results: Eight studies met inclusion criteria, encompassing 7937 patients. None of the studies verified the prediction rule with a reference standard on all or a random sample of patients. Six studies did not report blinding prediction rule assessors to reference standard results, and vice versa. Three prediction rules were prospectively validated. Sensitivities and specificities ranged from 94% to 100% and 13% to 57%, and positive and negative likelihood ratios from 1.1 to 2.2 and 0.01 to 0.17, respectively.
Conclusion: Current prediction rules for ACS have substantial methodological limitations and have not been successfully implemented in the clinical setting. Future methodologically sound studies are needed to guide clinical practice.
Objectif : Nous avons cherché à déterminer l'exactitude diagnostique des règles de prévision clinique visant à exclure le syndrome coronarien aigu (SCA) dans les salles d'urgence.
Méthodes : Nous avons interrogé les bases de données MEDLINE, EMBASE, Web of Science et les revues systématiques Cochrane. Nous avons communiqué avec des experts du contenu afin d'inclure des articles additionnels dans notre étude. Les listes bibliographiques des articles répertoriés ont été analysées manuellement. Nous avons appliqué les critères suivants pour la sélection des articles de notre étude : 1) recrutement de patients consécutifs à l'urgence; 2) inclusion de variables sur les antécédents ou l'examen physique, l'électrocardiogramme et les biomarqueurs cardiaques; 3) exclusion des études incluant les électrocardiographies d'effort ou les coronographies dans les règles de prévision; 4) étude fondée sur les recherches originales; 5) règles de prévision dérivées ou validées prospectivement; 6) utilisation d'un ordinateur exclue; 7) collecte de données suffisantes pour construire un tableau de contingence 2 × 2. Nous avons évalué la qualité des études et extrait des données indépendamment et en double à l'aide d'un formulaire normalisé d'extraction de données.
Résultats : Huit études, regroupant 7937 patients, répondaient aux critères d'inclusion. Aucune des études ne vérifiait la règle de prévision avec un étalon de référence pour tous les patients ou un échantillon aléatoire de patients. Six études ne mentionnaient pas si les résultats obtenus avec l'étalon de référence étaient analysés en aveugle ou non. Trois règles de prévision ont été validées prospectivement. La sensibilité et la spécificité variaient de 94 à 100 % et de 13 % à 57 %, et les rapports de vraisemblance positif et négatif variaient respectivement de 1,1 à 2,2 et de 0,01 à 0,17.
Conclusion : Les règles actuelles de prévision du SCA ont des limites méthodologiques considérables et n'ont pas été mises en application avec succès en clinique. Il faut réaliser d'autres études méthodologiques solides pour orienter la pratique clinique.
Chest pain is a diagnostic dilemma for the emergency physician. Data from the United States suggest that 2.1% of patients with acute myocardial infarction and 2.3% of patients with unstable angina are misdiagnosed,1 with slightly higher rates reported in a recent Canadian study (4.6% and 6.4%, respectively).2 Information obtained from the history, the initial 12-lead electrocardiogram (ECG), and a single set of cardiac markers does not have sufficient sensitivity to identify those patients who are safe for early discharge.3,4
To assist the American College of Cardiology and American Heart association (ACC/AHA) in developing guidelines for the diagnosis and treatment of unstable angina, the Agency for Healthcare Research and Quality (AHRQ) produced an evidence report that summarized the literature on the prediction of risk for patients with unstable angina.5 The evidence report consisted of 3 systematic reviews: the first evaluated the diagnostic value of the clinical history, physical exam and ECG; the second evaluated cardiac troponin; and the third assessed chest pain units and emergency department (ED) protocols. Despite these 3 systematic reviews, neither the 2007 ACC/AHA Guidelines for the Management of Patients with Unstable Angina and Non- ST-Segment Elevation Myocardial Infarction,6 nor the practical implementation of the 2002 AHA guidelines for the ED7 identify a subset of patients at very low risk for acute coronary syndrome (ACS) who can be safely discharged from the ED without cardiac stress testing. As a result, many patients at low risk for ACS may undergo prolonged ED observation or extensive outpatient investigation, resulting in a greater likelihood of false positive cardiac stress testing and significant cost to the health care system.
Clinical prediction rules are clinical tools that are designed to be used at the bedside to assist physician decision-making.8 They are derived from original research and incorporate variables from the history, physical examination and basic laboratory tests.9 When evaluating patients with chest pain in the ED setting, clinicians rely on readily available information obtained from the history and physical examination, ECG and cardiac biomarkers. We conducted a systematic review to summarize the diagnostic accuracy of clinical prediction rules that incorporate these variables in decision-making. This review was designed to answer the question: In patients who present to the ED with chest pain, what is the diagnostic accuracy of clinical prediction rules to exclude ACS?
A protocol (available upon request) was written with input from both content experts (A.S.J. and I.G.S.) and 1 systematic review methodology expert (V.M.M.). This report adheres to the Quality of Reporting of Meta-analyses (QUORUM) guidelines as applicable to diagnostic accuracy reviews.10
The search strategy was designed and conducted by an expert reference librarian (P.E.) with input from the clinical lead. Given that clinical prediction guides are variably indexed in the literature as clinical prediction rules, clinical decision rules, risk scores, algorithms and risk stratification tools, we incorporated medical subject heading keywords and a highly sensitive validated search filter for retrieving methodologically sound clinical prediction guides in the search strategy (Appendix 111). A second information specialist reviewed the search strategy and Boolean logic of the combination for errors. The electronic search included the following databases: MEDLINE (1950 to May 2007), EMBASE (1988 to 2007 week 05), Web of Science (1993 to February 2007) and Cochrane Database of Systematic Reviews (fourth quarter, 2006). The Ovid interface was used for searching MEDLINE and EMBASE. No language restrictions were applied. Adjustments were made to the search strategy to account for differences in indexing between databases.
Conference proceedings from the Canadian Association of Emergency Physicians and the Society for Academic Emergency Medicine from 2005 to 2007 were hand searched to identify abstracts that had not yet been indexed in electronic bibliographic databases. We consulted content experts (A.S.J. and J.E.H.) to identify additional published or other unpublished reports. Reference lists of included studies were searched to identify other relevant studies.
The purpose of this review was to identify prediction rules of sufficient methodological rigour that would warrant consideration for use in clinical practice. To safeguard against intrusion of bias, inclusion criteria were chosen according to published methodological standards for clinical prediction rules, and are listed in Box 1.8,9
We included studies that incorporated information from commonly available tests (ECG and cardiac biomarkers) since clinicians rely on these basic investigations in conjunction with their history and physical examination in bedside decision-making. Moreover, the limitations of variables from the history, physical examination and ECG for detecting ACS are well-established.12 Studies incorporating cardiac stress testing or coronary angiography as part of the prediction rule were excluded to reduce the risk of incorporation bias. Prediction rules that were not prospectively derived or validated were excluded.9 Prediction rules requiring use of a computer to generate diagnostic probabilities were excluded because an in-depth review of artificial neural networks was beyond the scope of this review.
Study retrieval was conducted in duplicate by 2 independent reviewers in 2 phases. In phase I, we screened titles and abstracts to identify potentially relevant articles. In phase II, we obtained the full articles and assessed them for eligibility. We determined the degree of interobserver agreement for each phase and reported κ values based on the following: for phase I "potentially eligible yes/no" agreement; for phase II, "eligible yes/no." After making independent assessments, we resolved disagreements by consensus.
Assessment of methodological quality
Although methodological criteria for clinical decision rules have been published, they have not been validated for use in systematic reviews. As such, QUADAS (Quality Assessment of Diagnostic Accuracy Studies), a validated quality assessment tool, was used.13 Two reviewers (E.P.H. and V.T.), working independently, assessed the quality of included studies. We calculated κ values based on the total number of QUADAS criteria met and subsequently resolved all disagreements by consensus. We dichotomized answers as "yes" and "no/unclear" for κ calculations.
When assessing the methodological quality of clinical prediction rules, it is useful to classify studies based on the hierarchy of evidence for clinical prediction rules.8 In this classification, prediction rules that have been derived but not validated constitute the lowest level of evidence (level 4), followed by studies that have been validated in only 1 narrow prospective sample (level 3), studies that have been validated broadly in multiple settings (level 2) and studies that have undergone impact analysis (level 1).
Working independently and in duplicate using a standardized data extraction form, we recorded the following descriptive data from every study: year and journal of publication, patient population (e.g., age, sex, cardiac risk factors and medical history), percentage of patients in whom ECGs, cardiac biomarkers and stress tests or coronary angiograms were obtained, clinical prediction rule characteristics, duration of follow-up and outcome definitions. We extracted the necessary data to construct 2 × 2 contingency tables. If data from the primary report were unclear, we contacted the corresponding author for clarification. If the corresponding author did not respond, we entered consensus data into each cell of the contingency table. Data were entered into a Microsoft Excel database (Microsoft Corp.).
Sensitivity and specificity were calculated from 2 × 2 contingency tables. Forest plots of sensitivity and specificity, and positive and negative likelihood ratios were constructed using Meta-DiSc software (Unit of Clinical Biostatistics of the Ramón y Cajal Hospital).14
Figure 1 describes the flow of candidate and eligible articles. The initial search strategy produced 720 records. After removing 46 duplicates, 674 records remained for screening in phase I. Eighty potentially eligible records were identified for further review based on review of the title and abstracts (κ = 0.95). Review of the full articles in phase II identified 8 that met inclusion criteria (κ = 0.85).15-21 Two additional articles were selected for further review from the bibliographies of selected articles.22,23 On full text review these were excluded because one retrospectively analyzed data that was originally collected from 2 large clinical trials22 and the other required the use of a computer to generate diagnostic probabilities.23 One potentially relevant article was identified through consultation with content experts. The article did not incorporate variables from the history or physical examination into the prediction tool and was therefore excluded.24 Data were available to construct 2 × 2 contingency tables for all 8 articles that met inclusion criteria. The 8 articles included in the review represent 5 different clinical prediction rules.
Fig. 1. Flow diagram of study selection process. ECG = electrocardiogram; ED = emergency department.
Characteristics of included studies
The studies included in the review were published from 2003 to 2007 and included 7937 patients in total (Table 115-21,25). There were 3 studies conducted in the United States, 2 in the United Kingdom, 1 in Canada, 1 in Spain and 1 in Brazil. With the exception of 1 study that enrolled adults between 24 and 39 years of age,19 and 2 studies that did not report cardiac risk factors,18,20 the remaining studies reported approximately comparable baseline characteristics and cardiac risk factors.
|Study (and no. of patients enrolled)||Clinical prediction guide||Variables included in prediction guide||Duration of follow-up||Outcomes|
|Bassan et al.15(1003)||Classificationdiagnostic tree||chest pain type, history of CAD, history of diabetes, CK-MB level, ST depression or T-wave inversion on ECG||Not reported||• AMI• Unstable angina|
|Chase et al.17 (1481)||Risk score||age ³ 65 yr, known CAD, ³ 3 cardiac risk factors, ST-segment deviation, ³ 2 anginal events in past 24 hr, aspirin use in past 7 d, elevated cardiac biomarkers||30 d||• All-cause mortality• AMI• Revascularization|
|Christenson et al.16 (819)||Clinical prediction rule||Age < 40 yr, normal or T-wave flattening on initial ECG, initial CK-MB < 3.0 ng/mL, or initial CK-MB > 3.0 ng/mL with no change in ECG, rise in CK-MB, or rise in troponin I 0-12 hr after arrival||30 d||• AMI• Unstable angina|
|Conway Morris et al.18 (1000)||Risk score||age ³ 65 yr, known CAD, ³ 3 cardiac risk factors, ST-segment deviation, ³ 2 anginal events in past 24 hr, aspirin use in past 7 d, elevated cardiac marker levels||30 d||• All-cause mortality• ST-segment elevation myocardial infarction• AMI• Troponin-positive acute coronary syndrome• Revascularization|
|Fernandez Portales et al.21 (321)||Risk score||6 hr troponin T > 0.04 ng/mL, age > 70 yr, cardiac history, prolonged chest pain in past 15 d, chest pain on presentation without ST-segment depression, chest pain on presentation with ST-segment depression, 6 hr troponin T > 0.04 ng/mL without ST-segment depression, 6 hr troponin T > 0.04 ng/mL with ST-segment depression||15 d||• Recurrent ischemia • Death • Heart failure|
|Lyon et al.20 (1000)||Risk score||age, pulse rate at presentation, systolic blood pressure at presentation, serum creatinine level at presentation, Killip score, ST-segment depression on presenting ECG, elevated initial serum cardiac biomarker level, cardiac arrest on admission||30 d||• All-cause mortality • ST-segment elevation myocardial infarction• AMI• Troponin-positive acute coronary syndrome• Revascularization|
|Marsan et al.19(1077)||Clinical prediction rule||Absence of cardiac history , no cardiac risk factors, normal ECG, CK-MB < 5 ng/mL, troponin I < 0.3 ng/mL||30 d||• AMI• Unstable Angina|
|Tong et al.25 (1236)||Risk score||age ³ 65 yr, known CAD, ³ 3 cardiac risk factors, ST-segment deviation, ³ 2 anginal events in past 24 hr, aspirin use in past 7 d, elevated cardiac marker level||30 d||• All-cause mortality• AMI• Revascularization • Unstable Angina|
|AMI = acute myocardial infarction; CAD = coronary artery disease; CK-MB = creatine kinase; ECG = electrocardiogram.|
Assessment of quality based on QUADAS criteria count had an interobserver agreement of 89.3% (κ = 0.79) (Table 215-21,25). None of the studies verified the results of their index test (prediction rule) against a reference standard of diagnosis (cardiac stress testing or coronary angiography) on either the whole study population or random sample thereof. A proxy reference standard of close patient follow-up for the development of adverse events was used in 7 studies.16-21,25 The reference standard was not reported in 1 study.15 Of the 8 studies, 6 did not clearly specify that the index test was interpreted without knowledge of results of the reference standard,15,17,18,,20,21,25 nor did it specify that the reference standard was interpreted without knowledge of the results of the index test. Table 2 lists quality assessment scores, diagnostic testing and prevalence of disease for each study.
|Study; stage of development||No. of QUADAS criteria met||Test; no. (and %) of patients||Probability of disease; no. (and %) of patients||Comments|
|ECG||Cardiac markers||Stress testing or coronary angiography||Pretest probability (prevalence)||Post-positive test||Post-negative test|
|Bassan et al.;15 derivation||10||556 (100)||443 (78.3)||—||269/566 (47.5)||260/388 (67)||9/578 (5.1)||117 (20.7%) discharged based on clinical and ECG assessment alone|
|Chase et al.;17validation||10||1458 (100)||1293 (85)||unclear||136/1458 (9.3)||128/983 (13)||8/475 (1.7)||TIMI Risk Score|
|Christenson et al.;16 derivation||10||769 (100)||769 (100)||—||165/769 (21.5)||163/571 (28.5)||2/198 (1)||Vancouver Chest Pain Rule|
|Conway Morris et al.;18 validation||9||—||—||—||137/954 (14.4)||137/723 (18.9)||0/231 (0)||TIMI Risk Score|
|Fernandez Portales et al.;21derivation||8||321 (100)||321 (100)||Coronary angiography 60 (18.7)||81/321 (25.2)||79/210 (37.6)||2/111 (1.8)||Cardiology resident enrolled patients with suspected acute coronary syndrome|
|Lyon et al.;20 validation||9||—||—||—||123/734 (16.8)||123/657 (18.7)||0/77 (0)||Data to calculate a GRACE Score not available in 240 (24%) patients|
|Marsan et al.;19 validation||11||1023 (100)||—||—||57/1023 (5.6)||56/724 (7.7)||1/299 (0.3%)||Study restricted to patients 24-39 years old|
|Tong et al.;25 validation||9||957 (100)||957 (100)||unclear||138/957 (14.4)||135/802 (16.8)||3/155 (1.9)||TIMI Risk Score|
|ECG = electrocardiograms; GRACE = Global Registry of Acute Coronary Events; QUADAS = Quality Assessment of Diagnostic Accuracy Studies; TIMI = Thrombolysis in Myocardial Infarction; — = not reported. *Percentages were calculated based on the total number of patients analyzed in each study.|
Three of the prediction rules were derived but not validated (level 4 in hierarchy of evidence).15,16,21 Bassan and colleagues15 classified the type of chest pain according to the likelihood of ACS and based subsequent diagnostic testing on this assessment. This increased the risk of verification bias (i.e., failure to use the same gold standard on all patients), potentially overestimating diagnostic performance. In addition, it is unclear whether outcomes were assessed without knowledge of the chest pain classification and vice versa (i.e., the lack of blinding). Fernandez Portales and colleagues21 used cardiology residents to select patients for enrolment based on a presumed diagnosis of ACS thus reducing the generalizability of their results. The clinical decision rule derived by Christenson and colleagues,16 on the other hand, adhered to established methodology for clinical prediction rules.9 All patients with a primary complaint of chest pain were potentially eligible, reducing the risk of selection bias. Study assistants assessed potential predictor variables without knowledge of the outcome and vice versa (adequate blinding).
Two prediction rules have been prospectively validated in 1 narrow sample (level 3).19,20 Marsan and colleagues19 enrolled only adults from 24 to 39 years of age, limiting the study's generalizability. Lyon and colleagues20 enrolled all patients with chest pain, decreasing the likelihood of selection bias. However, outcome was determined from routinely available data rather than close prospective follow-up, increasing the risk of misclassification for some patients.
The Thrombolysis in Myocardial Infarction(TIMI) risk score has been prospectively validated by 3 studies (level 2).17,18,25 Chase and colleagues17 enrolled all patients with nontraumatic chest pain in whom an ECG was obtained, decreasing the likelihood of selection bias. However, they did not report whether the score was assessed while blinded to the outcome diagnosis and vice versa. Conway Morris and colleagues18 validated the TIMI score in patients with chest pain deemed to be potentially cardiac in origin by the treating physician, potentially introducing selection bias. Also, the outcome relied on discharge diagnosis rather than prospective follow-up. Finally, Tong and colleagues25 enrolled all patients with presumed cardiac chest pain that persisted longer than 30 minutes, but 237 (19%) patients were either lost to follow-up or had uninterpretable reference standard test results, potentially biasing the estimates of diagnostic performance.
We observed substantial heterogeneity between studies with regard to the variables incorporated in each clinical prediction tool, the cardiac biomarker assays used, the pretest probability of disease and the outcome measures. With the exception of 3 studies that validated the TIMI risk score,17,18,25 each study incorporated different variables into the prediction model. The study by Bassan and colleagues15 used CK-MB, 5 studies used cardiac troponin alone,17,18,20,21,25 and 2 studies used both CK-MB and cardiac troponin.16,19 Only 2 of the studies16,19 reported the actual assay used to measure the biomarker as recommended by reporting guidelines.26 The pretest probability of disease varied between studies, potentially contributing to variability in diagnostic performance. The outcome measures in each study, as well as their definitions, also varied. For these reasons, the study data were not pooled.
Figure 215-21,25 shows a forest plot of sensitivity. Two studies reported sensitivities of 100%, with the remainder ranging from 94% to 99%. The forest plot for specificity is displayed in Figure 3.15-21,25 Substantial variation between specificity estimates is evident, with most estimates falling well below the 50% range. Figure 415-21,25 and Figure 515-21,,25 display forest plots of positive and negative likelihood ratios. Given that clinical prediction rules prioritize sensitivity over specificity, the positive likelihood ratios are near 1 (little diagnostic information), while the negative likelihood ratios are small. As a result, patients who are prediction-rule negative have a low posttest probability of disease.
Fig. 2. Forest plot of sensitivity. Point estimates of sensitivity along with 95% confidence intervals (CIs) (horizontal bars) are plotted for each study population.
Fig. 3. Forest plot of specificity. Point estimates of sensitivity along with 95% confidence intervals (CIs) (horizontal bars) are plotted for each study population.
Fig. 4. Forest plot of positive likelihood ratios. Horizontal bars represent the 95% confidence interval (CI).
Fig. 5. Forest plot of negative likelihood ratios. Horizontal bars represent the 95% confidence interval (CI).
We conducted a systematic review of prospective clinical decision rules that exclude ACS in ED patients with chest pain. Of the 8 risk prediction rules, 5 have been prospectively validated and thus could be considered for use in clinical practice.8,9 Although the diagnostic performance of the TIMI18 and Global Registry of Acute Coronary Events(GRACE)20 scores was sufficient for incorporation into clinical practice, the performance of the TIMI risk score varied between studies and the GRACE score required data that was not available in all ED patients. We did not identify any prediction tool that was appropriate for incorporation into clinical practice.
All 8 studies had methodological weaknesses that could have led to the introduction of bias. Deficiencies in reference standard testing, follow-up and blinding could have led to overestimation of diagnostic performance.15,17,18,20,21,25
We observed considerable heterogeneity between the studies with regard to the variables incorporated in each clinical prediction tool, the cardiac biomarkers and cutoff values selected, the prevalence of disease, the outcome studied and the diagnostic performance.
Likelihood ratios (LRs), unlike positive and negative predictive values, are independent of disease prevalence, and thus useful for comparing diagnostic tests between populations. A negative LR less than 0.1 is generally considered clinically useful.27 The upper bound of the 95% confidence interval for the negative LR in each study included 0.1, suggesting that none of the prediction rules exclude ACS with enough certainty to be clinically useful.
Numerous risk stratification tools for ED patients with chest pain have been published. In the 1980s, Goldman and colleagues28,29 derived and validated a computer protocol to predict myocardial infarction. However, much like other computer-derived prediction tools in emergency medicine,4,23,30-41 it has not been broadly adopted by emergency physicians. The Acute Cardiac Ischemia Time-Insensitive Predictive Instrument (ACI-TIPI) designed by Selker and colleagues39 is another prediction tool that has been commonly used. A multicentre controlled clinical trial found that its use reduced hospitalization among ED patients with chest pain. The ACI-TIPI instrument, however, defines low risk as less than a 10% risk for ACS, which is not low enough to forego cardiac stress testing. Also, it does not incorporate the results of cardiac biomarkers.
There is limited evidence to suggest what miss rate clinicians would consider acceptable. It is conceivable that this may differ between Canadian and American emergency physicians. In Canada, few ED-based observation units exist, and patients are often managed on an individualized basis.2 Given the miss rate of 5.3%,2 and the cost implications of broadly adopting ED observation units, a clinical prediction rule that decreases the rate of misdiagnosis in a fiscally efficient manner is likely to be acceptable to clinicians. A survey of Canadian emergency physicians indicated that most (94%) would use a clinical prediction rule for ACS, provided that it did not increase the miss rate above 2%.42 In the United States, however, physicians' triage decisions may be influenced more by perceived medical and legal risk, and it is not known what miss rate would be acceptable to most US physicians.
In the course of the review we did not identify a prediction tool to recommend for use in clinical practice. Future studies in this area are needed to guide clinical practice and should be carefully designed to prevent intrusion of bias,8,9 to take into account current guidelines by the European Society of Cardiology and by the American College of Cardiology43 and standardized reporting guidelines for ED studies,26 and to use the most sensitive cardiac troponin assays available.44,45
Our review is primarily limited by the methodological quality of the primary studies and the relatively small number of studies meeting inclusion criteria. None of the studies verified the results of the prediction rule by using a reference standard of diagnosis on the whole sample or random selection of the sample, increasing the risk of verification bias. Although only 8 studies met inclusion criteria, we conducted a thorough literature search that was unrestricted by language. In addition, we hand searched recent conference abstracts and consulted content experts thus minimizing the potential for publication bias.
Chest pain is a diagnostic dilemma for the emergency physician. Current prediction rules are heterogeneous, have substantial methodological limitations and have not been successfully implemented in the clinical setting. Future methodologically sound studies are needed to guide clinical practice.
- Pope JH, Aufderheide TP, Ruthazer R, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med 2000;342:1163-70.
- Christenson J, Innes G, McKnight D, et al. Safety and efficiency of emergency department assessment of chest discomfort. CMAJ 2004;170:1803-7.
- Swap CJ, Nagurney JT. Value and limitations of chest pain history in the evaluation of patients with suspected acute coronary syndromes. JAMA 2005;294:2623-9.
- Limkakeng A Jr, Gibler WB, Pollack C, et al. Combination of Goldman risk and initial cardiac troponin I for emergency department chest pain patient risk stratification. Acad Emerg Med 2001;2001:696-702.
- Heidenreich PA, Go A, Melsop KA, et al. Prediction of risk for patients with unstable angina. Evid Rep Technol Assess (Summ) 2000; (31):1-3.
- Anderson JL, Adams CD, Antman EM, et al. ACC/AHA 2007 guidelines for the management of patients with unstable angina/non ST-elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non ST-Elevation Myocardial Infarction): developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons: endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. Circulation 2007;116:e148-304.
- Gibler WB, Cannon CP, Blomkalns AL, et al. Practical implementation of the guidelines for unstable angina/non-ST-segment elevation myocardial infarction in the emergency department: a scientific statement from the American Heart Association Council on Clinical Cardiology (Subcommittee on Acute Cardiac Care), Council on Cardiovascular Nursing, and Quality of Care and Outcomes Research Interdisciplinary Working Group, in Collaboration With the Society of Chest Pain Centers. Circulation 2005;111: 2699-710.
- McGinn TG, Guyatt GH, Wyer PC, et al. Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA 2000; 284:79-84.
- Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437-47.
- Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999; 27;354:1896-1900.
- Wong SS, Wilczynski NL, Haynes RB, et al. Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE. AMIA Annu Symp Proc 2003;728-32.
- Swap CJ, Nagurney JT. Value and limitations of chest pain history in the evaluation of patients with suspected acute coronary syndromes. JAMA 2005;294:2623-9.
- Whiting P, Rutjes AW, Reitsma JB, et al. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2006;6:6-9.
- Zamora J, Abraira V, Muriel A, et al. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol. 2006; 12;6:31.
- Bassan R, Pimenta L, Scofano M, et al. Accuracy of a neural diagnostic tree for the identification of acute coronary syndrome in patients with chest pain and no ST-segment elevation. Crit Pathw Cardiol 2004;3:72-8.
- Christenson J, Innes G, McKnight D, et al. A clinical prediction rule for early discharge of patients with chest pain. Ann Emerg Med 2006;47:1-10.
- Chase M, Robey JL, Zogby KE, et al. Prospective validation of the Thrombolysis in Myocardial Infarction Risk Score in the emergency department chest pain population. Ann Emerg Med 2006; 48:252-9.
- Conway Morris A, Caesar D, Gray S, et al. TIMI risk score accurately risk stratifies patients with undifferentiated chest pain presenting to an emergency department. Heart 2006;92:1333-4.
- Marsan RJ Jr, Shaver KJ, Sease KL, et al. Evaluation of a clinical decision rule for young adult patients with chest pain. Acad Emerg Med 2005;12:26-31.
- Lyon R, Morris AC, Caesar D, et al. Chest pain presenting to the emergency department — to stratify risk with GRACE or TIMI? Resuscitation 2007;74:90-3.
- Fernandez Portales J, Perez Reyes F, Garcia Robles JA, et al. Risk stratification using combined ECG, clinical, and biochemical assessment in patients with chest pain without ST-segment elevation. How long should we wait? Rev Esp Cardiol 2003; 56: 338-45.
- Boersma E, Pieper KS, Steyerberg EW, et al. Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation. Results from an international trial of 9461 patients. The PURSUIT Investigators. Circulation 2000;101:2557-67.
- Pozen MW, D'Agostino RB, Selker HP, et al. A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease. A prospective multicenter clinical trial. N Engl J Med 1984;310:1273-8.
- Henrikson CA, Howell EE, Bush DE, et al. Prognostic usefulness of marginal troponin T elevation. Am J Cardiol 2004;93: 275-9.
- Tong KL, Kaul S, Wang XQ, et al. Myocardial contrast echocardiography versus Thrombolysis In Myocardial Infarction score in patients presenting to the emergency department with chest pain and a nondiagnostic electrocardiogram. J Am Coll Cardiol 2005;46:920-7.
- Hollander JE, Blomkalns AL, Brogan GX, et al. Standardized reporting guidelines for studies evaluating risk stratification of emergency department patients with potential acute coronary syndromes. Ann Emerg Med 2004;44:589-98.
- Jaeschke R, Guyatt G, Lijmer J. Diagnostic tests. In: Guyatt G, Rennie D, editors. Users' guides to the medical literature. Chicago (IL): AMA Press, 2002: 121-40.
- Goldman L, Weinberg M, Weisberg M, et al. A computer- derived protocol to aid in the diagnosis of emergency room patients with acute chest pain. N Engl J Med 1982;307:588-96.
- Goldman L, Cook EF, Brand DA, et al. A computer protocol to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988;318:797-803.
- Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med 1991;115:843-8.
- Baxt WG. A neural network trained to identify the presence of myocardial infarction bases some decisions on clinical associations that differ from accepted clinical teaching. Med Decis Making 1994;14:217-22.
- Baxt WG, Shofer FS, Sites FD, et al. A neural network aid for the early diagnosis of cardiac ischemia in patients presenting to the emergency department with chest pain. Ann Emerg Med 2002; 40:575-83.
- Baxt WG, Shofer FS, Sites FD, et al. A neural computational aid to the diagnosis of acute myocardial infarction. Ann Emerg Med 2002;39:366-73.
- Baxt WG, Skora J. Prospective validation of artificial neural network trained to identify acute myocardial infarction. Lancet 1996;347:12-5.
- Lorenzoni R, Ebert AG, Lattanzi F, et al. A computer protocol to evaluate subjects with chest pain in the emergency department: a multicenter study. J Cardiovasc Med 2006;7:203-9.
- Poretsky L, Leibowitz IH, Friedman SA. The diagnosis of myocardial infarction by computer-derived protocol in a municipal hospital. Angiology 1985;36:165-70.
- Harrison RF, Kennedy RL. Artificial neural network models for prediction of acute coronary syndromes using clinical data from the time of presentation. Ann Emerg Med 2005;46:431-9.
- Mitchell AM, Garvey JL, Chandra A. Prospective multicenter study of quantitative pretest probability assessment to exclude acute coronary syndrome for patients evaluated in emergency department chest pain units. Ann Emerg Med 2006;47:447.
- Selker HP, Beshansky JR, Griffith JL, et al. Use of the acute cardiac ischemia time-insensitive predictive instrument (ACI-TIPI) to assist with triage of patients with chest pain or other symptoms suggestive of acute cardiac ischemia. A multicenter, controlled clinical trial. Ann Intern Med 1998;129:845-55.
- Green M, Bjork J, Forberg J, et al. Comparison between neural networks and multiple logistic regression to predict acute coronary syndrome in the emergency room. Artif Intell Med 2006; 38: 305-18.
- Qamar A, McPherson C, Babb J, et al. The Goldman algorithm revisited: prospective evaluation of a computer-derived algorithm versus unaided physician judgment in suspected acute myocardial infarction. Am Heart J 1999;138:705-9.
- Macgougan CK, Christenson JM, Innes GD, et al. Emergency physicians' attitudes toward a clinical prediction rule for the identification and early discharge of low risk patients with chest discomfort. CJEM 2001;3:89-94.
- Alpert JS, Thygesen K, Antman E, et al. Myocardial infarction redefined-a consensus document of The Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol 2000; 36:959-69.
- Jaffe AS. Chasing troponin: how low can you go if you can see the rise? J Am Coll Cardiol 2006;48:1763-4.
- Apple FS, Wu AH, Jaffe AS. European Society of Cardiology and American College of Cardiology guidelines for redefinition of myocardial infarction: how to use existing assays clinically and for clinical trials. Am Heart J 2002;144:981-6.
Dr. Erik P. Hess, Clinical Epidemiology Unit, Office F657, Ottawa Health Research Institute, The Ottawa Hospital, Civic Campus, 1053 Carling Ave., Ottawa ON K1Y 4E9; firstname.lastname@example.org