Emergency physicians' attitudes toward a clinical prediction rule for the identification and early discharge of low risk patients with chest discomfort

EM Advances

CJEM 2001;3(2):89-94

Abstract

Résumé

Introduction

The decision to admit or discharge patients with chest discomfort is often difficult. Despite recent diagnostic advances, there is no single test that identifies patients who are "safe" to discharge after a brief assessment. In the first few hours, the history, physical examination and ECG remain the key diagnostic tools used to determine patient disposition.

Multicentre studies demonstrate that approximately 2% of patients with acute myocardial infarction (AMI) or unstable angina are inadvertently discharged from the ED.^1,2 Although often referred to as "missed AMI," these patients are more accurately termed "missed ACS" (missed acute coronary syndromes). In response, many centres have developed chest pain units (CPU), which apply intensive 6- to 12-hour diagnostic algorithms to low risk patients with chest discomfort.^3-9 This approach leads to the investigation of many patients without disease, and reported AMI rates in CPU patients are as low as 0.2% to 6%. It may be possible to identify, on clinical grounds, a subset of "very low risk" patients who do not require intensive investigation and can be discharged after a brief emergency department (ED) assessment.

Clinical prediction rules help physicians make diagnostic or therapeutic decisions under conditions of uncertainty. They are developed by identifying clinical variables (e.g., historical, physical, ECG findings or serum markers) that are associated with the outcome of interest (e.g., death or AMI), then combining the strongest predictor variables into a decision tool that can be applied like a diagnostic test.¹⁰ Several groups have devised algorithms and clinical tools to risk stratify patients with chest discomfort;^11-17 however, none of these have been widely adopted by emergency physicians. The lack of support for existing decision tools may reflect the fact that they require longer observation periods and define "low risk" patients as those who have a 3% to 5% probability of AMI, which is higher than the current "missed AMI" threshold.

Our hypothesis was that emergency physicians would support a clinical prediction rule to identify chest discomfort patients who are safe for discharge after brief ED evaluation, as long as that rule does not increase the risk of discharging patients with ACS. Our objectives were to determine Canadian emergency physicians' 1) estimates regarding the safety and efficiency of chest discomfort management in their ED and 2) attitudes toward and perception of the need for a chest discomfort clinical prediction rule that identifies very low risk patients who are safe to discharge after a brief ED assessment.

Methods

Study design and population

This was a cross-sectional mail survey. To identify eligible emergency physicians, we acquired the CAEP (Canadian Association of Emergency Physicians) membership database. Using S-PLUS (Mathsoft, Seattle, Wash.), we randomly identified 300 CAEP members from urban, rural, community and academic centres. The study sample represented approximately 25% of the total membership at that time. Subjects not actively practising EM and those residing outside North America were excluded. The St. Paul's Ethics Committee for Human Experimentation considered this study exempt from formal review.

Survey instrument and process

The 20-question, self-administered survey (available from the investigators on request) was modeled after a previous survey that examined emergency physician attitudes toward a clinical prediction rule for diagnostic radiography.¹⁸ The draft survey was reviewed by a clinical epidemiologist (J.R.), then piloted on a convenience sample of 11 emergency physicians and modified to ensure clarity.

The survey elicited information about physicians, hospital setting and disposition practices. In addition, it asked respondents to provide their opinions regarding the value of a clinical prediction rule that would identify patients with chest discomfort who are safe to discharge after a brief (~2 hour) assessment. Table 1 summarizes survey data elicited.

The survey was initiated in June 1999 using Dillman's Total Design Method for mail surveys.¹⁹ Respondents were sent a survey package containing an introductory letter, a questionnaire, self-addressed stamped envelope and pre-stamped reply postcard. To ensure anonymity, the questionnaire and return envelope contained no identifiers. One week after the initial mailing, all potential respondents were sent a reminder postcard. Subjects who returned the reply post card to our research office were removed from the second mailing list. Two weeks after the initial mailing, non-respondents were sent a second survey package, and 5 weeks after the initial mailing, non-respondents were sent a final survey package encouraging them to respond.

Data analysis

Data was manually entered and analyzed using SPSS for MacIntosh version 6.1 (SPSS, Chicago, Ill.). Standard descriptive statistics with means and standard deviations and medians and interquartile ranges were calculated.

Results

Of the 300 subjects selected, 8 were excluded because they were not in active practice and 4 because they resided outside North America, leaving 288 in the study sample. Of these, 43% responded to the first mailing, 26% to the second, and 13% to the third mailing -- an overall 82% (235/288) response rate. Table 2 summarizes demographic, professional and practice characteristics for the study sample, showing that most respondents were males with formal emergency medicine certification. At the time of the study, two-thirds provided full-time clinical care and half worked in university-affiliated hospitals.

Eleven (5%) of 235 respondents reported that their ED has a follow-up process to identify "missed" unstable angina and AMI, and most respondents were uncomfortable estimating the frequency of these events. Ninety-one respondents were willing to estimate their rate of "missed" unstable angina (median estimate = 5.0%; interquartile range, 2.0%- 5.0%) and 106 estimated their rate of "missed" AMI (median, 2.0%; IQR, 1.0%-3.0%). Most (55/106; 52%) felt their rate of "missed" AMI was at least 2%.

Overall, 216 (92%) of 235 subjects provided an estimate for the average length of stay (LOS) for chest discomfort patients discharged from the ED with non-cardiac diagnoses (Fig. 1). The median estimated LOS was 4.0 hours (IQR, 3.0-6.0) and 193 (89%) of the estimates were greater than 2 hours. Only 118 (50%) respondents were able to estimate the percentage of patients admitted from their ED who are subsequently confirmed to have an ACS. Figure 2 shows that these estimates ranged from 1% to 85% and that 71 (60%) of these estimates were below 50%.

When asked about the potential utility of a clinical prediction rule to identify chest discomfort patients who are safe to discharge after a brief (~2 hour) ED assessment, 165 (70%) subjects felt that such a rule would be very useful, 43 (18%) felt it would be useful, 19 (8%) felt it would be somewhat useful, 6 (3%) felt it would be not useful and 2 (1%) were undecided (Fig. 3). Almost all subjects (220/235; 94%) felt that a prediction rule would be most helpful if it identified more patients who are safe for discharge, but did not increase the rate of "missed" AMI above 2%. Fifteen respondents felt that such a rule would not be useful, and 12 specified reasons for this opinion, including the following: a 2% AMI miss rate is too high (n = 6); prediction rules are of doubtful validity (n = 2); prediction rules are cumbersome and difficult to recall (n = 1); guidelines are preferable to rules (n = 1); rules exclude clinical judgement (n = 1); and rules could miss non-ACS diagnoses that require admission (n = 1).

Of 234 respondents, 138 (59%) felt that a clinical prediction rule should suggest a course of action, 71 (30%) felt it should provide a probability of disease, 14 (6%) felt it should provide both and 6 (3%) were undecided. Five had additional suggestions: the probability of no disease; appropriate outpatient investigations; morbidity/ mortality comparisons; the negative predictive value of the rule; and the ability to differentiate benign from dangerous chest pain.

Fig. 1. Estimated emergency department length of stay

Discussion

Our data suggest that Canadian emergency physicians rarely follow-up patients with chest discomfort and that few have feedback mechanisms to identify "missed" cases of ACS. Those who were willing to estimate believe that they "miss" 3%-5% of patients with unstable angina and 1%-3% of patients with AMI -- this despite ED investigation times of 3-6 hours. The physicians also expressed substantial uncertainty about the proportion of admitted patients who ultimately prove to have ACS. Their estimates ranged from 1% to

 
Fig. 2. Estimates of percentage of patients admitted to rule out acute coronary syndromes (ACS) whose final diagnosis is ACS.

Fig. 3. Emergency physicians' opinions regarding the value of an emergency department prediction rule (n = 235)

85%, and most believe that fewer than 50% of patients admitted to rule-out ACS actually prove to have unstable angina or AMI. These data suggest that patients with chest discomfort create much diagnostic uncertainty despite prolonged ED lengths of stay and substantial resource utilization. This level of diagnostic uncertainty suggests a need for better clinical tools to guide early decision-making for patients with chest discomfort.

ED patients with symptoms suggesting ACS can be placed into 1 of 3 categories.²⁰ Those with objective evidence of ischemia, including ST-segment elevation, new ST-depression, dynamic ST deviations or elevated serum markers have "definite ACS." They require admission, antiplatelet therapy, anti-ischemic medications and, in many cases, reperfusion therapy. Those who lack objective signs of ischemia but whose clinical presentation suggests unstable angina or AMI ("possible ACS") require further investigation in actual or "virtual" CPUs for diagnostic and risk stratification purposes. The third ("non-ACS") group, who have relatively benign causes of chest discomfort, can be safely discharged with appropriate follow-up after a brief ED assessment.

Patients with "definite ACS" seldom pose a diagnostic challenge, but distinguishing "possible ACS" patients (who require urgent investigation) from "non-ACS" patients (who don't) is a difficult task. To maximize patient safety and avoid risk, emergency physicians tend to investigate or admit many low risk patients. In many US hospitals, it is common practice to evaluate virtually all chest pain patients over a 9- to 12-hour period. At the same time, Canadian coronary care units (CCUs) are becoming more efficient by admitting fewer patients with non-ACS diagnoses. The FASTRAK^® Acute Coronary Syndromes Registry estimates that only 10% of CCU admissions do not have a cardiac-related discharge diagnosis. The inevitable result of these conflicting forces is that patients with chest pain who have no objective electrocardiogram changes or serum marker elevations are held in the ED for diagnostic work-up, prolonging ED lengths of stay and downloading investigational costs. While it is important to minimize the number of "missed ACS" cases, excessively cautious disposition practices may result in inefficient utilization of limited health care resources. We believe it is possible to develop a clinical prediction rule that will identify "non-ACS" patients who are safe to discharge after a brief ED assessment. Most of our survey respondents also recognized the need to improve diagnostic efficiency, and a large majority felt that such a prediction rule would be useful.

Clinical prediction rules are more likely to influence care if they suggest a course of action rather than providing a probability of disease,¹⁰ and this is concordant with the preferences of most respondents. Prediction rules with dichotomous outcomes (e.g., discharge vs. investigate) provide clear guidance, while prediction rules that estimate disease likelihood do not. Likelihood estimates are abstract concepts and different physicians will respond differently to the same risk estimate, depending on their risk tolerance level. In addition, a clinical prediction rule does not preclude use of other diagnostic information or clinical judgement.

One of the initial steps in designing a clinical prediction rule is to establish the failure (false-negative) rate that clinicians are willing to accept. In this study, 94% of emergency physicians indicated that a rule would be useful if it did not increase the rate of missed AMI and unstable angina above 2%. Interestingly, 90% of respondents who estimated their current missed AMI rate to be less than 2% also felt that such a rule would be useful. In settings where the miss rate is >2% a rule could improve both safety and efficiency.

A somewhat alarming finding of this study is that emergency physicians had little knowledge about the safety and efficiency of their current practice. Only 5% had follow-up mechanisms for patients with chest discomfort, and most were unaware what percentage of patients are discharged from their ED with "missed" ACS. Similarly, most were unaware of what proportion of patients admitted from their ED ultimately prove to have ACS. This suggests an impressive lack of structured feedback regarding the appropriateness of disposition decisions, which precludes accurate estimates of current miss rates and may be a roadblock to clinical improvement.

Limitations and future questions

In all surveys, including this one, the validity of the data depends on the respondents' honesty. In addition, while 82% is an excellent response rate, it is possible that non-responders may have different practices and attitudes. It is also possible that CAEP members (our study population) may not be representative of all Canadian emergency physicians; therefore there may be a sampling bias. The vast majority of physicians studied expressed favourable attitudes toward an early discharge clinical prediction rule for patients with chest discomfort; however, these attitudes may not translate into changed behaviour if a prediction rule can be developed.

The study data show that emergency physicians have substantial uncertainty around the diagnosis of ACS and that they would welcome a prediction rule to help them make early disposition decisions. In the next phase of our research, we plan to identify a reliable combination of clinical predictors that identify patients who are at very low risk of ACS or adverse outcomes after an ED visit for chest discomfort. Future studies should also address the question of whether better diagnostic feedback mechanisms (e.g., improved ED information systems, formal follow-up strategies) could improve ED diagnostic accuracy for patients with chest discomfort.

Conclusions

Canadian emergency physicians support the concept of a clinical prediction rule for the early discharge of patients with chest discomfort. Most believe that such a rule would be useful if it identified patients who are safe for discharge after a brief assessment, while maintaining current levels of safety. Future research should be aimed at deriving a clinical prediction rule to identify low risk patients who can be safely discharged after a limited emergency department evaluation.

References

1. McCarthy BD, Beshansky JR, D'Agostino RB, Selker HP. Missed diagnoses of acute myocardial infarction in the emergency department: results from a multicenter. Ann Emerg Med 1993;22:579-82.
2. Pope J, Aufderheide T, Ruthazer R, Woolard R, Feldman J, Beshansky J, et al. Missed diagnoses of acute cardiac ischemia in the emergency room. N Engl J Med 2000;342:1163-70.
3. Gibler WB, Runyon JP, Levy RC, Sayre MR, Kacich R, Hattemer CR, et al. A rapid diagnostic and treatment center for patients with chest pain in the emergency department. Ann Emerg Med 1995;25:1-8.
4. Farkouh M, Smars P, Reeder G, Zinsmeister A, Evans R, Meloy T, et al. A clinical trial of a chest-pain observation unit for patients with unstable angina. N Engl J Med 1998;339:1882-8.
5. Dallara J, Severance HW, Davis B, Schulz G. Differences between chest pain observation service patients and admitted "rule-out myocardial infarction" patients. Acad Emerg Med 1997;4:693.
6. Mikhail MG, Smith FA, Gray M, Britton C, Frederiksen SM. Cost-effectiveness of mandatory stress testing in chest pain center patients. Ann Emerg Med 1997;29:88-98.
7. Gomez MA, Anderson JL, Karagounis LA, Muhlestein JB, Mooers FB. An emergency department-based protocol for rapidly ruling out myocardial ischemia reduces hospital time and expense: results of a randomized study (ROMIO). J Am Coll Cardiol 1996;28:25-33.
8. Gaspoz J, Lee T, Cook F, Weisberg M, Goldman L. Outcome of patients who were admitted to a new short-stay unit to "rule-out" myocardial infarction. Am J Cardiol 1991;68:145-9.
9. Hoekstra JW, Gibler WB, Levy RC, Sayre M, Naber W, Chandra A, et al. Emergency-department diagnosis of acute myocardial infarction and ischemia: a cost analysis of two diagnostic protocols. Acad Emerg Med 1994;1:103-10.
10. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodologic standards. JAMA 1997;277:488-94.
11. Pozen M, D'Agostino R, Mitchell J, Rosenfeld M, Guglielmino J, Schwartz M, et al. The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Ann Intern Med 1980;92:238-42.
12. Pozen M, D'Agostino R, Selker H, Sytkowski P, Hood W. A predictive instrument to improve coronary care unit admission practices in acute ischemic heart disease. N Engl J Med 1984;310:1273-8.
13. Selker H, Beshansky J, Griffith J, Aufderheide T, Ballin D, Bernard S, et al. Use of the Acute Cardiac Ischemia Time-Insensitive Predictive Instrument (ACI-TIPI) to assist with triage of patients with chest pain or other symptoms suggestive of acute cardiac ischemia: a multicenter, controlled clinical trial. Ann Intern Med 1998;129:845-55.
14. Goldman L, Weinberg M, Weisberg M, Olshen R, Cook EF, Sargent RK, et al. A computer-derived protocol to aid in the diagnosis of emergency room patients with acute chest pain. N Engl J Med 1982;307:588-96.
15. Goldman L, Cook E, Brand D. A computer protocol to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988;318:797-803.
16. Reilly B, Durairaj L, Husain S, Acob C, Evans A, Hu T, et al. Performance and potential impact of a chest pain decision prediction rule in a large public hospital. Am J Med 1999;106:285-91.
17. Goldman L, Cook EF, Johnson PA, Brand DA, Rouan GW, Lee TH. Prediction of the need for intensive care in patients who come to the emergency departments with acute chest pain. N Engl J Med 1996;334:1498-504.
18. Graham I, Stiell I, Laupacis A, O'Connor A, Wells G. Emergency physicians' attitudes toward and use of clinical decision rules for radiography. Acad Emerg Med 1998;5:134-40.
19. Dillman D. Mail and telephone surveys: the total design method. New York: Wiley Interscience Publishing; 1978.
20. Christenson J. Acute coronary syndromes: we must improve diagnostic efficiency in the emergency department. CJEM 1999;1:22-5.