Survey of emergency physicians' requirements for a clinical decision rule for acute respiratory illnesses in three countries

Original Research

CJEM 2012;14(2):83-89

Abstract

Résumé

Acute respiratory conditions such as chronic obstructive pulmonary disease (COPD), community-acquired pneumonia (CAP), and heart failure (HF) are common diagnoses in the emergency department (ED). In the United States, the incidence of pneumonia has been estimated at 12 per 1,000 and results in approximately 1 million admissions to hospital annually.^1,2 HF is a common problem frequently associated with the need for hospitalization. Once discharged, up to 60% of HF patients are readmitted within 6 months.^3,4 COPD is a common condition in North America that has a prevalence of over 4.7% of patients over 55 years and accounts for over 1,500,000 ED visits annually in the United States.⁵ All three of these conditions are more common and generally more severe in older patients. However, there is considerable variability in the rates of hospital admission for patients over 50 years presenting to the ED with these conditions.^3,6 There are currently no widely accepted guidelines to determine which patients over 50 years with acute respiratory conditions require hospital admission.

A clinical decision rule is an algorithmic decision-making tool that is derived from original research using strict methodological guidelines.^7–9 Previous studies have attempted to assess patient risks and improve decision making for hospital admission for patients in the aforementioned three groups.¹⁰ However, many of these rules have limitations, such as the use of retrospective data, and are difficult to apply in the setting of a busy ED. A prediction rule to identify patients with CAP requiring admission attempted to address some of these limitations by categorizing patients by risk.¹¹ However, this rule does not capture social issues and does not include any functional measures, and further study is required to ensure its efficacy and safety. A comparison of four existing clinical prediction rules for estimating risk in HF showed that all had varying ability to predict short-term death or serious outcomes.¹¹ Currently, it is unknown how sensitive clinicians would require clinical decision rules for respiratory illnesses to be to be clinically acceptable. Moreover, there is no assessment of a patient's ability to ambulate in any of the available clinical decision rules, and it is unknown how often physicians assess this functional ability prior to disposition.

This study was an international survey of members of three national emergency medicine physician associations: the Australasian College of Emergency Physicians, the Canadian Association of Emergency Physicians, and the American College of Emergency Physicians (ACEP). Our objectives were 1) to determine the willingness of physicians to use clinical decision rules to help predict whether hospital admission was required for patients over 50 years for three respiratory conditions: COPD, CAP, and HF; 2) to determine the required sensitivity of these rules to predict a return visit to the ED or death within 14 days for each condition and, for HF only, to predict myocardial infarction within 14 days; and 3) to assess the current use of any form of ambulation assessment (“walk test”) in the ED for patient disposition.

Methods

Study Design And Setting

We conducted a self-administered postal survey of emergency physicians (EPs) in Australasia, Canada, and the United States. We used a modified Dillman's tailored design method for survey design and administration.¹² This study was coordinated by the Clinical Epidemiology Program of the Ottawa Hospital Research Institute between May 2007 and December 2007. This study was approved by The Ottawa Hospital Research Ethics Board.

Selection Of Participants

A total of 1,500 EPs were selected to participate. This was composed of a random sample of 500 members of the Australasian College of Emergency Physicians (total membership 772), a random sample of 500 members of the Canadian Association of Emergency Physicians (total membership 1,317), and a random sample of 500 members of the ACEP (total membership 25,614). Lists of randomly selected members were computer generated and provided by each organization. Members returning their surveys who reported that they were not physicians or not currently practicing in adult emergency medicine were excluded from the analysis.

Survey Content

Survey participants received a three-page questionnaire consisting mainly of closed questions. This study was a substudy of a larger study that involved one additional page of questions pertaining to transient ischemic attack. The section on respiratory illness management contained questions on the need for clinical decision rules for acute respiratory illnesses and the required sensitivity for death, myocardial infarction, and repeat ED visit within 14 days. This study was initially pilot-tested on approximately 45 EPs at The Ottawa Hospital and subsequently revised based on feedback received.

Participants were specifically asked to indicate their level of agreement, using a 6-point Likert-type scale from strongly agree to strongly disagree, with the following statement: “A clinical decision rule is required to guide hospital admission for patients over 50 years with 1) COPD, 2) HF, and 3) CAP.” They were then asked to indicate if they currently use some form of a walk test prior to discharge for patients over 50 years with these three conditions. If they answered yes, they were asked to describe their walk test. Participants were also asked if they would consider using a highly sensitive and well-validated clinical decision rule to guide the need for admission for patients over 50 years for each respiratory condition (yes/no response). Physician respondents were subsequently asked how sensitive each decision rule would have to be to predict 1) return ED visit within 14 days, 2) death within 14 days, and 3) for HF patients only, myocardial infarction within 14 days (response choices of < 80%, 80–84%, 85–89%, 90–94%, 95–96%, 97–98%, 99%. or 100%).

Survey Administration

All potential participants received a prenotification letter that described the study and asked for their participation. One week later, they received a cover letter describing the study, assuring them of confidentiality, and providing instructions for survey completion along with the survey instrument. Nonresponders were sent a minimum of two reminder letters along with the survey at 4-week intervals. All postal surveys included a postage-paid, preaddressed reply envelope. No incentives were provided.

Study Outcomes

All data were entered into a Microsoft Excel (Microsoft Corp., Seattle, WA) database. Our primary outcome measures were 1) the percentage of EPs who would consider using a clinical decision rule to predict the need for admission of patients over 50 years with an ED diagnosis of a) COPD, b) CAP, or c) HF and 2) the required sensitivity of each rule to predict death within 14 days of their index ED visit. Secondary outcomes were 1) to determine the required sensitivity for each rule to predict a return visit to the ED within 14 days of their index visit, 2) to determine the required sensitivity for a myocardial infarction within 14 days for the HF rule only, and 3) to determine whether EPs are currently using some form of walk test to help them evaluate if patients require hospitalization.

Analysis

Data were analyzed using SPSS version 16 (SPSS Inc., Chicago, IL). Means and standard deviations were calculated for continuous variables; frequency distributions were generated for categorical variables. Likert-scale responses were dichotomized as agree (slightly, moderately, or strongly) or disagree. Primary and secondary outcomes were calculated by country, and overall weighted estimates were obtained by taking into account the stratified sampling design; 95% confidence intervals with finite population correction factors were calculated around each estimate.

Sample Size Calculation

We used a stratified random sample design with 500 members from each organization surveyed. The number of 500 per organization was chosen to yield a desired precision of ± 4% around anticipated estimates of proportions in each country, using a conservative proportion of affirmative answers of 50%. We assumed a response rate of 50%, similar to previous surveys.

Results

Of the 1,493 surveys sent, 801 EPs responded (7 surveys came back “return to sender”), resulting in an aggregate response rate of 54%. Country-specific response rates were Australasia, 55% (277 of 500); Canada, 60% (296 of 493); and the United States, 46% (228 of 500). The numerator represents all returned surveys, whereas the denominator represents all surveys sent, less the 7 returned because the surveys were undeliverable.

Physician demographic, professional, and practice setting characteristics are summarized in Table 1. Most respondents were male. Ninety-five percent of Australasian respondents worked at a teaching hospital, in contrast to 75% of Canadian respondents and 44% of American respondents.

Table 2 reports the proportion of EPs who believe that a clinical decision rule is required and the proportion who would consider using a clinical decision rule for each acute respiratory condition, stratified by country. EPs in Australasia and Canada had a greater belief that clinical decision rules are required for all three of the respiratory conditions compared to those in the United States. The overall weighted estimate of the proportion of physicians who would consider using a clinical decision rule if one existed for patients over 50 years presenting with HF was 91% (95% CI 87.9–94.9); the weighted estimate of the proportion of physicians who would consider using such a rule for patients with CAP was 92% (95% CI 88.4–95.2), and the weighted estimate of the proportion of physicians who would consider using such a rule for patients with COPD was 91% (95% CI 86.8–94.1).

Table 3 presents the median required sensitivity for each of the proposed clinical decision rules categorized by ED diagnosis and outcome measure. Physicians in all countries indicated that they required a highly sensitive rule for all three respiratory conditions for death, with an overall median required sensitivity of 97% (interquartile range [IQR] 95–99%). The median required sensitivity for return ED visit was 95% (IQR 90–96) for all three conditions, and the median required sensitivity for subsequent myocardial infarction following HF was 97% (IQR 95–98%).

The estimated proportion of physicians using some form of nonstandardized walk test assessment tool to help them determine patient disposition with the three acute respiratory conditions was 63.2% (95% CI 57.2–69.2). Open-ended descriptions of these assessments included a walk around the ED with oxygen saturation monitoring, oxygen saturation ≥ 88% after walking a prescribed distance, and mobility assessment by physiotherapy or respiratory therapy.

Discussion

Our survey indicates that EPs would consider using clinical decision rules for patients over 50 years of age who present to the ED with COPD, CAP, or HF to determine the need for hospital admission. The median required sensitivity for death within 14 days was 97 to 98% for all three conditions. EPs were pragmatic, acknowledging that any rule may not be absolutely perfect. It is clear, however, that any such rules must be highly sensitive. Having a highly sensitive and well-validated rule may help decrease health care costs by identifying low-risk patients who can be treated safely without the requirement of a hospital admission. In addition, patients who are at high risk could be admitted for treatment in a monitored medical environment, thereby potentially decreasing the risk of adverse outcomes.

This survey was an international study including three of the largest English-speaking areas in the world. Our results indicate that EPs believe in clinical decision rules, given that the vast majority reported that they would use a clinical decision rule for each of the three acute respiratory conditions should a well-validated, highly sensitive rule be developed.

There are few studies on the outcome of patients discharged from the ED with acute HF. These studies have identified that many relapse soon after discharge, with high morbidity and mortality in this population.^3,4,13–18 Similarly, COPD is an important cause of morbidity and mortality in North America, and up to one-third of patients with an ED visit for a COPD exacerbation are admitted.^19–22 CAP is also a common ED condition. Previous guidelines have been developed to guide admission based on a variety of factors (e.g., age, gender, comorbidities, and severity of illness).^11,23,24 Admission rules developed and validated by the Patient Outcome Research Team (PORT) group suggested that patients can be categorized into five classes, from low risk (class I; mortality < 1%) to high risk (class V; mortality ≈ 30%).¹¹ The PORT rule is composed of demographic, clinical, laboratory, and radiographic findings at presentation and was generated using administrative databases. Because they arose from database methodology, these rules are cumbersome and difficult to use, fail to capture other important features that may be present (e.g., intravenous drug use, homelessness, support systems, ability to drink fluids), and fail to include any ambulation or functional assessment. We are not aware of any surveys assessing clinician's perspectives or requirements before any of these previous studies were conducted.

Some clinical decision rules have good clinical uptake, whereas others do not.^25–28 It is not always apparent as to why some rules are successfully implemented into clinical practice and others do not result in substantial change. There is evidence that stated intentions are correlated with actual behaviour.^29,30 Hence, although there are no absolute guarantees that physicians who indicate that they are agreeable to using a clinical decision rule will actually follow through and use it, the available evidence suggests that they are more likely to do so. Conducting surveys assessing stated intentions increases the likelihood that a completed clinical decision rule will ultimately be successful and, as such, is a necessary precursor to the development of a clinical decision rule.

Limitations

Our target population was EPs as clinical decision rules are often used by these front-line caregivers. We chose to sample members of three national emergency medicine associations. We chose these three associations because in Canada and Australasia, they are the only EP associations, and in the case of the United States, the ACEP is the largest emergency medicine association. We do not have definitive data regarding the total number of EPs working in each country and the demographics of these physicians. In Canada, the Canadian Medical Association publication of physicians and specialties provides some assistance by providing age and sex statistics for full-time specialty-trained EPs; however, it does not include family physicians with nonspecialist certification or those who work part-time in the ED. Given that EPs who belong to these organizations are likely more academically inclined, the possibility of coverage error exists (i.e., some physicians in the target population never getting a chance to answer the survey). However, our Canadian respondents had a mean age of 42.3 and 73.9% were male, which closely compares to the median age range of 45 to 54 years and 81% male gender reported in the statistics for EPs by the Canadian Medical Association in 2007.^31,32

Our response rate was relatively low, and the possibility of nonresponse bias cannot be excluded (i.e., different results from nonrespondents than respondents). To respect the privacy and confidentiality of their members, the associations would not provide us with demographic information regarding nonresponders. However, we obtained data about the membership as a whole for Canada and the United States. In Canada, the membership data were for 2009, which demonstrated for their 1,317 members that 58% work full-time in the ED, 63.2% work in urban teaching hospitals, and 19.4% work in rural teaching centres compared to the 78.5% of our survey respondents indicating that they work in a teaching hospital. In the United States, the ACEP has 25,614 members with a mean age of 42.8 years, 74.6% male gender, and 57% working in urban centres; this is similar to our survey results of 44.6 years, 79.5% male gender, and 44.4% working in teaching centres. Although our response rate was less than the 80% considered ideal, it is consistent with previous physician studies that have indicated an average response rate for published physician surveys of 52 to 54%.^33,34

We surveyed predominantly English-speaking countries as most clinical decision rules have been developed in English and the use of these tools would be most familiar to an English-speaking audience. Hence, the results of this survey cannot necessarily be generalized to other geographic or cultural areas. The United Kingdom was approached to participate; however, the EP association would not provide us with a list of members or distribute surveys on our behalf, citing privacy concerns. Finally, because the study has a survey design, most of the questions on the study questionnaire were closed-ended and did not allow physicians to expand or elaborate on their responses.

Conclusion

This large international survey indicates that EPs will consider using clinical decision rules for COPD, CAP, and HF if they are highly sensitive and well validated to identify those patients who are likely to have a return ED visit or die within 14 days from their initial visit. These results will inform and inspire the development of clinical decision rules for patients over 50 years of age with acute respiratory conditions.

Acknowledgement

We would like to thank the hundreds of physicians who completed our survey and the following research personnel at the Ottawa Hospital Research Institute: Catherine Clement, My-Linh Tran, and Angela Marcantonio.