Survey of emergency physicians' requirements for a clinical decision rule for acute respiratory illnesses in three countries
Jeffrey J. Perry, MD, MSc; Reena Goindi, MD; Cheryl Symington, RN; Jamie Brehaut, PhD; Monica Taljaard, PhD; Sandra Schneider, MD; Ian G. Stiell, MD, MSc;
From the *Department of Emergency Medicine, Department of Epidemiology and Community Medicine, Clinical Epidemiology Program, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, ON; †University of Rochester, Rochester, NY.
Presented at the Canadian Association of Emergency Physicians, CAEP June 2008, Ottawa, ON.
This article has been peer reviewed.
© Canadian Association of Emergency Physicians
There are currently no widely used guidelines to determine which older patients with acute respiratory conditions require hospital admission. This study assessed the need for clinical decision rules to help determine whether hospital admission is required for patients over 50 years for three common respiratory conditions: chronic obstructive pulmonary disease (COPD), heart failure (HF), and community-acquired pneumonia (CAP).
Emergency physicians (EPs) from the United States, Canada, and Australasia.
A random sample of EPs from the United States, Canada, and Australasia.
A modified Dillman technique with a prenotification letter and up to three postal surveys.
EP opinions regarding the need for and willingness to use clinical decision rules for emergency department (ED) patients over 50 years with COPD, HF, or CAP to predict hospital admission. We assessed the required sensitivity of each rule for return ED visit or death within 14 days.
A total of 801 responses from 1,493 surveys were received, with response rates of 55%, 60%, and 46% for Australasia, Canada, and the United States, respectively. Over 90% of EPs reported that they would consider using clinical decision rules for HF, CAP, and COPD. The median required sensitivity for death within 14 days was 97 to 98% for all conditions.
EPs are likely to adopt highly sensitive clinical decision rules to predict the need for hospital admission for patients over 50 years with COPD, HF, or CAP.
Il n'existe actuellement aucunes lignes directrices largement utilisées pour déterminer les patients âgés présentant des troubles respiratoires aigus devant être admis à l'hôpital. La présente étude vise à évaluer la nécessité de disposer de règles pour les décisions cliniques afin d'établir plus facilement si une admission à l'hôpital est requise pour les patients de 50 ans et plus présentant trois troubles respiratoires fréquents: maladie pulmonaire obstructive chronique (MPOC), insuffisance cardiaque (IC), et pneumonie acquise dans la communauté (PAC).
Médecins d'urgence (MU) des États-Unis, du Canada, et de l'Australie/Nouvelle-Zélande.
Un échantillon aléatoire de MU des États-Unis, du Canada, et de l'Australie/Nouvelle-Zélande.
Technique Dillman modifiée avec lettre de préavis et jusqu'à trois sondages postaux.
Avis des MU concernant la nécessité de disposer de règles pour les décisions cliniques et désir de les utiliser au service des urgences (SU) pour les patients âgés de 50 ans et plus présentant une MPOC, une IC, ou une PAC, afin de prévoir les admissions dans les hôpitaux. Nous avons évalué le degré de sensibilité requis pour chaque règle dans le cas d'une nouvelle visite au SU et d'un décès dans un délai de 14 jours.
Au total, 801 réponses ont été reçues sur un total de 1 493 sondages, représentant des taux de réponse de 55 %, 60 %, et 46 % pour l'Australie/Nouvelle-Zélande, le Canada, et les États-Unis, respectivement. Plus de 90 % des MU ont déclaré qu'ils étudieraient la possibilité d'utiliser les règles pour les décisions cliniques en présence d'une MPOC, d'une IC, ou d'une PAC. Le degré de sensibilité moyen requis pour un décès dans un délai de 14 jours était de 97 à 98 % pour tous les troubles.
Les MU sont susceptibles d'adopter des règles à degré de sensibilité élevé pour les décisions cliniques afin de prévoir la nécessité d'admettre à l'hôpital des patients âgés de 50 ans et plus présentant une MPOC, une IC, ou une PAC.
Acute respiratory conditions such as chronic obstructive pulmonary disease (COPD), community-acquired pneumonia (CAP), and heart failure (HF) are common diagnoses in the emergency department (ED). In the United States, the incidence of pneumonia has been estimated at 12 per 1,000 and results in approximately 1 million admissions to hospital annually.1,2 HF is a common problem frequently associated with the need for hospitalization. Once discharged, up to 60% of HF patients are readmitted within 6 months.3,4 COPD is a common condition in North America that has a prevalence of over 4.7% of patients over 55 years and accounts for over 1,500,000 ED visits annually in the United States.5 All three of these conditions are more common and generally more severe in older patients. However, there is considerable variability in the rates of hospital admission for patients over 50 years presenting to the ED with these conditions.3,6 There are currently no widely accepted guidelines to determine which patients over 50 years with acute respiratory conditions require hospital admission.
A clinical decision rule is an algorithmic decision-making tool that is derived from original research using strict methodological guidelines.7–9 Previous studies have attempted to assess patient risks and improve decision making for hospital admission for patients in the aforementioned three groups.10 However, many of these rules have limitations, such as the use of retrospective data, and are difficult to apply in the setting of a busy ED. A prediction rule to identify patients with CAP requiring admission attempted to address some of these limitations by categorizing patients by risk.11 However, this rule does not capture social issues and does not include any functional measures, and further study is required to ensure its efficacy and safety. A comparison of four existing clinical prediction rules for estimating risk in HF showed that all had varying ability to predict short-term death or serious outcomes.11 Currently, it is unknown how sensitive clinicians would require clinical decision rules for respiratory illnesses to be to be clinically acceptable. Moreover, there is no assessment of a patient's ability to ambulate in any of the available clinical decision rules, and it is unknown how often physicians assess this functional ability prior to disposition.
This study was an international survey of members of three national emergency medicine physician associations: the Australasian College of Emergency Physicians, the Canadian Association of Emergency Physicians, and the American College of Emergency Physicians (ACEP). Our objectives were 1) to determine the willingness of physicians to use clinical decision rules to help predict whether hospital admission was required for patients over 50 years for three respiratory conditions: COPD, CAP, and HF; 2) to determine the required sensitivity of these rules to predict a return visit to the ED or death within 14 days for each condition and, for HF only, to predict myocardial infarction within 14 days; and 3) to assess the current use of any form of ambulation assessment (“walk test”) in the ED for patient disposition.
Study Design And Setting
We conducted a self-administered postal survey of emergency physicians (EPs) in Australasia, Canada, and the United States. We used a modified Dillman's tailored design method for survey design and administration.12 This study was coordinated by the Clinical Epidemiology Program of the Ottawa Hospital Research Institute between May 2007 and December 2007. This study was approved by The Ottawa Hospital Research Ethics Board.
Selection Of Participants
A total of 1,500 EPs were selected to participate. This was composed of a random sample of 500 members of the Australasian College of Emergency Physicians (total membership 772), a random sample of 500 members of the Canadian Association of Emergency Physicians (total membership 1,317), and a random sample of 500 members of the ACEP (total membership 25,614). Lists of randomly selected members were computer generated and provided by each organization. Members returning their surveys who reported that they were not physicians or not currently practicing in adult emergency medicine were excluded from the analysis.
Survey participants received a three-page questionnaire consisting mainly of closed questions. This study was a substudy of a larger study that involved one additional page of questions pertaining to transient ischemic attack. The section on respiratory illness management contained questions on the need for clinical decision rules for acute respiratory illnesses and the required sensitivity for death, myocardial infarction, and repeat ED visit within 14 days. This study was initially pilot-tested on approximately 45 EPs at The Ottawa Hospital and subsequently revised based on feedback received.
Participants were specifically asked to indicate their level of agreement, using a 6-point Likert-type scale from strongly agree to strongly disagree, with the following statement: “A clinical decision rule is required to guide hospital admission for patients over 50 years with 1) COPD, 2) HF, and 3) CAP.” They were then asked to indicate if they currently use some form of a walk test prior to discharge for patients over 50 years with these three conditions. If they answered yes, they were asked to describe their walk test. Participants were also asked if they would consider using a highly sensitive and well-validated clinical decision rule to guide the need for admission for patients over 50 years for each respiratory condition (yes/no response). Physician respondents were subsequently asked how sensitive each decision rule would have to be to predict 1) return ED visit within 14 days, 2) death within 14 days, and 3) for HF patients only, myocardial infarction within 14 days (response choices of < 80%, 80–84%, 85–89%, 90–94%, 95–96%, 97–98%, 99%. or 100%).
All potential participants received a prenotification letter that described the study and asked for their participation. One week later, they received a cover letter describing the study, assuring them of confidentiality, and providing instructions for survey completion along with the survey instrument. Nonresponders were sent a minimum of two reminder letters along with the survey at 4-week intervals. All postal surveys included a postage-paid, preaddressed reply envelope. No incentives were provided.
All data were entered into a Microsoft Excel (Microsoft Corp., Seattle, WA) database. Our primary outcome measures were 1) the percentage of EPs who would consider using a clinical decision rule to predict the need for admission of patients over 50 years with an ED diagnosis of a) COPD, b) CAP, or c) HF and 2) the required sensitivity of each rule to predict death within 14 days of their index ED visit. Secondary outcomes were 1) to determine the required sensitivity for each rule to predict a return visit to the ED within 14 days of their index visit, 2) to determine the required sensitivity for a myocardial infarction within 14 days for the HF rule only, and 3) to determine whether EPs are currently using some form of walk test to help them evaluate if patients require hospitalization.
Data were analyzed using SPSS version 16 (SPSS Inc., Chicago, IL). Means and standard deviations were calculated for continuous variables; frequency distributions were generated for categorical variables. Likert-scale responses were dichotomized as agree (slightly, moderately, or strongly) or disagree. Primary and secondary outcomes were calculated by country, and overall weighted estimates were obtained by taking into account the stratified sampling design; 95% confidence intervals with finite population correction factors were calculated around each estimate.
Sample Size Calculation
We used a stratified random sample design with 500 members from each organization surveyed. The number of 500 per organization was chosen to yield a desired precision of ± 4% around anticipated estimates of proportions in each country, using a conservative proportion of affirmative answers of 50%. We assumed a response rate of 50%, similar to previous surveys.
Of the 1,493 surveys sent, 801 EPs responded (7 surveys came back “return to sender”), resulting in an aggregate response rate of 54%. Country-specific response rates were Australasia, 55% (277 of 500); Canada, 60% (296 of 493); and the United States, 46% (228 of 500). The numerator represents all returned surveys, whereas the denominator represents all surveys sent, less the 7 returned because the surveys were undeliverable.
Physician demographic, professional, and practice setting characteristics are summarized in Table 1. Most respondents were male. Ninety-five percent of Australasian respondents worked at a teaching hospital, in contrast to 75% of Canadian respondents and 44% of American respondents.
|(n = 277)||(n = 296)||(n = 228)||(N = 801)|
|Mean age, yr (range)||42.3 (32–63)||42.3 (27–66)||44.6 (31–77)||43.0 (27–77)|
|Years of practice, mean (range)||12.6 (1–34)||12.0 (1–35)||13.2 (1–33)||12.0 (1–35)|
|Teaching hospital, %||93.1||75.8||44.4||73.2|
|Annual emergency department visits > 50,000, %||42.4||48.9||39.5||44.0|
Table 2 reports the proportion of EPs who believe that a clinical decision rule is required and the proportion who would consider using a clinical decision rule for each acute respiratory condition, stratified by country. EPs in Australasia and Canada had a greater belief that clinical decision rules are required for all three of the respiratory conditions compared to those in the United States. The overall weighted estimate of the proportion of physicians who would consider using a clinical decision rule if one existed for patients over 50 years presenting with HF was 91% (95% CI 87.9–94.9); the weighted estimate of the proportion of physicians who would consider using such a rule for patients with CAP was 92% (95% CI 88.4–95.2), and the weighted estimate of the proportion of physicians who would consider using such a rule for patients with COPD was 91% (95% CI 86.8–94.1).
|Australasia, n (%)||Canada, n (%)||United States, n (%)||Overall estimate, % (95% CI)|
CAP = community-acquired pneumonia; COPD = chronic obstructive pulmonary disease; HF = heart failure.
|Agree a clinical decision rule is required to guide hospital admission for patients > 50 yr with COPD||165 (63.0)||196 (74.5)||91 (44.3)||46.9 (40.8–53.1)|
|Agree a clinical decision rule is required to guide hospital admission for patients > 50 yr with HF||168 (64.4)||201 (70.4)||95 (46.3)||48.8 (42.7–55.0)|
|Agree a clinical decision rule is required to guide hospital admission for patients > 50 yr with CAP||191 (73.2)||196 (74.5)||109 (53.2)||55.1 (49.0–61.3)|
|Would consider using a rule for HF patients||240 (92.0)||244 (94.2)||187 (91.2)||91.4 (87.9–94.9)|
|Would consider using a rule for CAP patients||239 (91.6)||243 (93.5)||188 (91.7)||91.8 (88.4–95.2)|
|Would consider using a rule for COPD patients||237 (90.8)||242 (93.4)||183 (90.1)||90.5 (86.8–94.1)|
|Currently using some form of walk test||119 (45.8)||156 (59.5)||130 (64.0)||63.2 (57.2–69.2)|
Table 3 presents the median required sensitivity for each of the proposed clinical decision rules categorized by ED diagnosis and outcome measure. Physicians in all countries indicated that they required a highly sensitive rule for all three respiratory conditions for death, with an overall median required sensitivity of 97% (interquartile range [IQR] 95–99%). The median required sensitivity for return ED visit was 95% (IQR 90–96) for all three conditions, and the median required sensitivity for subsequent myocardial infarction following HF was 97% (IQR 95–98%).
IQR = interquartile range.
|Median required sensitivity for death within 14 days of index visit, % (IQR)|
|Heart failure||97–98 (95–99)||97–98 (95–99)||99 (95–99)||97–98 (95–99)|
|Community-acquired pneumonia||97–98 (97–99)||97–98 (95–99)||99 (95–99)||97–98 (95–99)|
|Chronic obstructive pulmonary disease||97–98 (95–99)||97–98 (95–99)||99 (95–99)||97–98 (95–99)|
|Median required sensitivity for return emergency department visit within 14 days of index visit, % (IQR)|
|Heart failure||95–96 (90–96)||95–96 (90–96)||95–96 (90–98)||95–96 (90–96)|
|Community-acquired pneumonia||95–96 (90–96)||95–96 (90–96)||95–96 (90–96)||95–96 (90–96)|
|Chronic obstructive pulmonary disease||95–96 (90–96)||95–96 (90–96)||95–96 (90–96)||95–96 (90–96)|
|Median required sensitivity for myocardial infarction within 14 days of index visit, % (IQR)|
|Heart failure||97 (95–98)||95 (95–98)||98 (95–99)||97 (95–98)|
The estimated proportion of physicians using some form of nonstandardized walk test assessment tool to help them determine patient disposition with the three acute respiratory conditions was 63.2% (95% CI 57.2–69.2). Open-ended descriptions of these assessments included a walk around the ED with oxygen saturation monitoring, oxygen saturation ≥ 88% after walking a prescribed distance, and mobility assessment by physiotherapy or respiratory therapy.
Our survey indicates that EPs would consider using clinical decision rules for patients over 50 years of age who present to the ED with COPD, CAP, or HF to determine the need for hospital admission. The median required sensitivity for death within 14 days was 97 to 98% for all three conditions. EPs were pragmatic, acknowledging that any rule may not be absolutely perfect. It is clear, however, that any such rules must be highly sensitive. Having a highly sensitive and well-validated rule may help decrease health care costs by identifying low-risk patients who can be treated safely without the requirement of a hospital admission. In addition, patients who are at high risk could be admitted for treatment in a monitored medical environment, thereby potentially decreasing the risk of adverse outcomes.
This survey was an international study including three of the largest English-speaking areas in the world. Our results indicate that EPs believe in clinical decision rules, given that the vast majority reported that they would use a clinical decision rule for each of the three acute respiratory conditions should a well-validated, highly sensitive rule be developed.
There are few studies on the outcome of patients discharged from the ED with acute HF. These studies have identified that many relapse soon after discharge, with high morbidity and mortality in this population.3,4,13–18 Similarly, COPD is an important cause of morbidity and mortality in North America, and up to one-third of patients with an ED visit for a COPD exacerbation are admitted.19–22 CAP is also a common ED condition. Previous guidelines have been developed to guide admission based on a variety of factors (e.g., age, gender, comorbidities, and severity of illness).11,23,24 Admission rules developed and validated by the Patient Outcome Research Team (PORT) group suggested that patients can be categorized into five classes, from low risk (class I; mortality < 1%) to high risk (class V; mortality ≈ 30%).11 The PORT rule is composed of demographic, clinical, laboratory, and radiographic findings at presentation and was generated using administrative databases. Because they arose from database methodology, these rules are cumbersome and difficult to use, fail to capture other important features that may be present (e.g., intravenous drug use, homelessness, support systems, ability to drink fluids), and fail to include any ambulation or functional assessment. We are not aware of any surveys assessing clinician's perspectives or requirements before any of these previous studies were conducted.
Some clinical decision rules have good clinical uptake, whereas others do not.25–28 It is not always apparent as to why some rules are successfully implemented into clinical practice and others do not result in substantial change. There is evidence that stated intentions are correlated with actual behaviour.29,30 Hence, although there are no absolute guarantees that physicians who indicate that they are agreeable to using a clinical decision rule will actually follow through and use it, the available evidence suggests that they are more likely to do so. Conducting surveys assessing stated intentions increases the likelihood that a completed clinical decision rule will ultimately be successful and, as such, is a necessary precursor to the development of a clinical decision rule.
Our target population was EPs as clinical decision rules are often used by these front-line caregivers. We chose to sample members of three national emergency medicine associations. We chose these three associations because in Canada and Australasia, they are the only EP associations, and in the case of the United States, the ACEP is the largest emergency medicine association. We do not have definitive data regarding the total number of EPs working in each country and the demographics of these physicians. In Canada, the Canadian Medical Association publication of physicians and specialties provides some assistance by providing age and sex statistics for full-time specialty-trained EPs; however, it does not include family physicians with nonspecialist certification or those who work part-time in the ED. Given that EPs who belong to these organizations are likely more academically inclined, the possibility of coverage error exists (i.e., some physicians in the target population never getting a chance to answer the survey). However, our Canadian respondents had a mean age of 42.3 and 73.9% were male, which closely compares to the median age range of 45 to 54 years and 81% male gender reported in the statistics for EPs by the Canadian Medical Association in 2007.31,32
Our response rate was relatively low, and the possibility of nonresponse bias cannot be excluded (i.e., different results from nonrespondents than respondents). To respect the privacy and confidentiality of their members, the associations would not provide us with demographic information regarding nonresponders. However, we obtained data about the membership as a whole for Canada and the United States. In Canada, the membership data were for 2009, which demonstrated for their 1,317 members that 58% work full-time in the ED, 63.2% work in urban teaching hospitals, and 19.4% work in rural teaching centres compared to the 78.5% of our survey respondents indicating that they work in a teaching hospital. In the United States, the ACEP has 25,614 members with a mean age of 42.8 years, 74.6% male gender, and 57% working in urban centres; this is similar to our survey results of 44.6 years, 79.5% male gender, and 44.4% working in teaching centres. Although our response rate was less than the 80% considered ideal, it is consistent with previous physician studies that have indicated an average response rate for published physician surveys of 52 to 54%.33,34
We surveyed predominantly English-speaking countries as most clinical decision rules have been developed in English and the use of these tools would be most familiar to an English-speaking audience. Hence, the results of this survey cannot necessarily be generalized to other geographic or cultural areas. The United Kingdom was approached to participate; however, the EP association would not provide us with a list of members or distribute surveys on our behalf, citing privacy concerns. Finally, because the study has a survey design, most of the questions on the study questionnaire were closed-ended and did not allow physicians to expand or elaborate on their responses.
This large international survey indicates that EPs will consider using clinical decision rules for COPD, CAP, and HF if they are highly sensitive and well validated to identify those patients who are likely to have a return ED visit or die within 14 days from their initial visit. These results will inform and inspire the development of clinical decision rules for patients over 50 years of age with acute respiratory conditions.
We would like to thank the hundreds of physicians who completed our survey and the following research personnel at the Ottawa Hospital Research Institute: Catherine Clement, My-Linh Tran, and Angela Marcantonio.
Garibaldi RA. Epidemiology of community-acquired respiratory tract infections in adults. Incidence, etiology, and impact. Am J Med 1985;78(6B):32-7, doi:10.1016/0002- 9343(85)90361-4.
Mandell LA, Bartlett JG, Dowell SF, et al. Update of practice guidelines for the management of community-acquired pneumonia in immunocompetent adults. Clin Infect Dis 2003. 37:1405-33, doi:10.1086/380488.
Tsuyuki RT, Shibata MC, Nilsson C, Hervas-Malo M. Contemporary burden of llness of congestive heart failure in Canada. Can J Cardiol 2003. 19:436-8
Krumholz HM, Parent EM, Tu N, et al. Readmission after hospitalization for congestive heart failure among Medicare beneficiaries. Arch Intern Med 1997. 157:99-104, doi:10.1001/archinte.1997.00440220103013.
Mannino DM, Homa DM, Akinbami LJ, et al. Chronic obstructive pulmonary disease surveillance—United States, 1971–2000. Atlanta, GA: US Department of Health and Human Services; 2000.
Tsai A, Kallsen G. Epidemiology of pediatric prehospital care. Ann Emerg Med 1987. 16:284-92, doi:10.1016/S0196-0644(87)80173-7.
Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999. 33:437-47, doi:10.1016/S0196-0644(99)70309-4.
Laupacis A, Sekar N, Stiell IG. Clinical Prediction Rules: a review and suggested modifications of methodological standards. JAMA 1997. 277:488-94, doi:10.1001/jama.1997.03540300056034.
Wasson JH, Sox HC, Neff RK, Goldman L. Clinical Prediction Rules: applications and methodological standards. N Engl J Med 1985. 313:793-9, doi:10.1056/NEJM198509263131306.
Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 1997. 336:243-50, doi:10.1056/NEJM199701233360402.
McAlister FA, Lawson FM, Teo KK, Armstrong PW. A systematic review of randomized trials of disease management programs in heart failure. Am J Med 2001. 110:378-84, doi:10.1016/S0002-9343(00)00743-9.
Dillman DA. Mail and Internet surveys: the tailored design method. 2nd ed. New York: Wiley; 2000.
Auble TE, Hsieh M, Gardner W,et al. A prediction rule to identify low-risk patients with heart failure. Acad Emerg Med 2005. 12:514-21, doi:10.1111/j.1553-2712.2005.tb00891.x.
Cowie MR, Wood DA, Coats AJ, et al. Survival of patients with a new diagnosis of heart failure: a population based study. Heart 2000. 83:505-10, doi:10.1136/heart.83.5.505.
Lee DS, Johansen H, Gong Y,et al. Regional outcomes of heart failure in Canada. Can J Cardiol 2004. 20:599-607
Tsuyuki RT, Richter CA, Rowe BH. Practice patterns and outcomes in patients presenting to the emergency department with acute heart failure. Can J Cardiol 2005. 21(C):93C
Brewer AV, Burton JH, Strout TD. Emergency department patients with acutely decompensated congestive heart failure: is discharge a safe disposition? Acad Emerg Med 2003. 10:544, doi:10.1197/aemj.10.5.544-a.
Rame JE, Sheffield MA, Dries DL, et al. Outcomes after emergency department discharge with a primary diagnosis of heart failure. Am Heart J 2001. 142:714-9, doi:10.1067/mhj.2001.118473.
Feinleib M, Rosenberg HM, Collins JG, et al. Trends in COPD morbidity and mortality in the United States. Am Rev Respir Dis 1989. 140(3 Pt 2):S9-18
Seemungal TAR, Donaldson GC, Paul EA, et al. Effect of exacerbation on quality of life in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1998. 157:1418-22
American Thoracic Society. Standards for the diagnosis and care of patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1995. 152(5 Pt 2):S77-120
Rosychuk RJ, Metes D, Voaklander DC, et al. COPD presentations to emergency departments in Alberta, Canada: a population-based study [abstract]. Can J Emerg Med 2009. 11:269.
Black ER, Mushlin AI, Griner PF, et al. Predicting the need for hospitalization of ambulatory patients with pneumonia. J Gen Intern Med 1991. 6:394-400, doi:10.1007/BF02598159.
Fine MJ, Smith DN, Singer DE. Hospitalization decision in patients with community-acquired pneumonia: a prospective cohort study. Am J Med 1990. 89:713-21, doi:10.1016/0002-9343(90)90211-U.
Stiell IG, Wells GA, Hoag RA, et al. Implementation of the Ottawa Knee Rule for the use of radiography in acute knee injuries. JAMA 1997. 278:2075-8, doi:10.1001/jama.1997.03550230051036.
Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa Ankle Rules. JAMA 1994. 271:827-32, doi:10.1001/jama.1994.03510350037034.
Stiell IG, Wells GA, Laupacis A, et al. A multicentre trial to introduce clinical decision rules for the use of radiography in acute ankle injuries. Br Med J 1995. 311:594-7, doi:10.1136/bmj.311.7005.594.
Stiell IG, Clement C, Rowe BH, et al. Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA 2005. 294:1511-8, doi:10.1001/jama.294.12.1511.
Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof 2006. 26:13-24, doi:10.1002/chp.47.
Godin G, Belanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals' intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement Sci 2008. 3:36, doi:10.1186/1748-5908-3-36.
Canadian Medical Association. Number of physicians by specialty and age, Canada, 2007 Ottawa: Canadian Medical Association; 2007.
Canadian Medical Association. Number and percent distribution of physicians by specialty and sex, Canada 2007 Ottawa: Canadian Medical Association; 2007.
Cummings SM, Savitz LA, Konrad TR. Reported response rates to mailed physician questionnaires. Health Serv Res 2001. 35:1347-55
Asch D, Jedrziewski K, Christakis N. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997. 10:1129-36, doi:10.1016/S0895-4356(97)00126-1.
Dr. Jeffrey J. Perry, Clinical Epidemiology Unit, F6, Ottawa Hospital Research Institute, The Ottawa Hospital, Civic Campus, 1053 Carling Avenue, Ottawa, ON K1Y 4E9; email@example.com.