Key indicators of overcrowding in Canadian emergency departments: a Delphi study

EM Advances

CJEM 2007;9(5):339-346

Abstract

Résumé

Introduction

One of the most challenging issues currently facing the Canadian health care system is overcrowding in hospital emergency departments (EDs). The Canadian Association of Emergency Physicians and the National Emergency Nurses Affiliation have defined ED overcrowding as a situation in which the demand for ED services exceeds the ability to provide care in a reasonable length of time.1 A recent survey of 243 Canadian ED directors found that 62% reported overcrowding as a major or severe problem in 2004 and 2005.2 Although emergency health care providers and hospital administrators may have an intuitive sense of when an ED is becoming overcrowded, there is no consensus on what overcrowding is or how it can best be measured. A systematic review3 found wide variation in content, with only 43% of the studies providing explicit definitions. The authors concluded that a single definition or isolated indicators do not provide a comprehensive assessment of ED overcrowding and may not be suitable for all situations. A more consistent approach that focuses on standardized indicators of events occurring in the ED would help distinguish between the causes, characteristics and outcomes of overcrowding.3

Several studies4-9 have developed lists of proxy measures, scales or indicators of ED overcrowding. Yet, these have not been widely accepted owing to the multiple attributes of overcrowding. Moreover, it is unclear what indicators of ED overcrowding are important to ED providers, researchers and administrators within Canada.

The objective of this study was to identify the level of consensus among ED experts across Canada on the importance of a set of potential indicators of ED overcrowding. A secondary objective was to describe the relative importance of these indicators.

Methods

Study design

A 2-round modified Delphi study was conducted from February 2005 to April 2005. The Delphi approach is a consensus technique that involves the participation of a group of experts and has 4 distinguishing features: anonymity, iteration (the procedure involves at least 1 round), controlled feedback (results of each round are analyzed separately and responses fed back to Delphi participants) and statistical group response (expression of the degree of consensus among the group).10 The Delphi technique has been used to identify indicators in a variety of biomedical areas.11-13 The study was reviewed and approved by the Health Research Ethics Board at the University of Alberta.

Figure 1 shows the flow of the Delphi process. A purposeful sample of 38 Canadian ED experts involved in various aspects of ED operations was invited to participate. An expert was defined as any Canadian ED administrator, medical director, physician or nurse with a known or stated interest in ED overcrowding. Names of potential Delphi participants were obtained from an advisory panel that comprised 10 researchers, clinicians, administrators and opinion leaders in emergency medicine across Canada. Potential participants were individually contacted by email. Each received a letter of consent describing the Delphi process and the expectations regarding their participation.

An initial list of potential indicators of ED overcrowding along with their operational definitions was generated from literature searches. The list was revised by members of the advisory panel, who were also invited to add other indicators and operational definitions. After discussions with the advisory panel, a group of 36 indicators was included in the first-round questionnaire. Operational definitions were provided in the survey questionnaire for all of the 36 potentially relevant indicators. The instrument was pretested on a convenience sample of 11 emergency physicians. Responses from the pretesting were not included in the Delphi analysis.

In the first-round, participants were asked to rate the importance of the 36 measures as indicators of ED overcrowding on a 7-point Likert-type scale (1 = extremely unimportant and 7 = extremely important). In round 2, participants were presented with a list of 10 indicators for which consensus on importance had been achieved in round 1. They were also provided with their first-round ratings as well as the group rating for each indicator. This provided an opportunity for them to amend their ratings in light of the opinions expressed by the group. Participants ranked the indicators according to their relative importance in documenting ED overcrowding. It was established a priori that a group mean of 5.5 (important to very important) or, in the presence of skewed data, a group median of 6 (very important) in the Likert scale would be deemed an indicator of consensus.11

Both first- and second-round questionnaires were distributed electronically. The second-round questionnaire was sent to all the individuals regardless of their participation in the first round. Nonrespondents were sent 3 reminders at 2-week intervals after the initial distribution.

Proportions and percentages were generated for categorical data. Information from the Likert scales was treated as continuous data and reported as means with standard deviations (SDs) or, in the presence of skewed data, medians with interquartile ranges (IQRs). SAS version 6 (SAS Institute, Carey, NC) was used for all the analyses.

Results

Delphi participants

Of the 38 experts who were contacted, 32 (84% response rate) completed the first Delphi round and 33 (87% response rate) completed the second round. Twenty-nine participants had an affiliation with a university hospital. The annual ED census of their institutions ranged from 25 000 to 210 000 patients (median 60 000, IQR 50 000-70 000).

Round 1

Ten indicators of ED overcrowding were selected by consensus as being relevant and clinically important (Table 1). Four indicators were considered extremely important (i.e., with a median Likert scale value of 7). They were the total number of ED patients, the percentage of ED occupied by inpatients, the total time in the ED and the percentage of time the ED was at or above its capacity. Indicators considered very important (Likert scale ≥ 6) were the overall bed occupancy, the time from bed request to bed assignment, the number of staffed acute care beds, the time from triage to emergency practitioner, the time from bed ready to transfer to wards and emergency physician satisfaction.

Round 2

The rank ordering of the top 10 indicators of ED overcrowding selected from round 1 is shown in Table 2. The percentage of the ED occupied by inpatients was ranked as the most important indicator of ED overcrowding (mean 6.53 on 7-point Likert scale, SD 0.80). The other 9 indicators, in order of the importance attributed from most to least, were the total number of ED patients (mean 6.35, SD 0.75), the total time in the ED (mean 6.16, SD 1.04), the percentage of time that the ED is at or above capacity (mean 6.16, SD 1.08), the overall bed occupancy (mean 6.19, SD 0.93), the time from bed request to bed assignment (mean 6.06, SD 1.08), the time from triage to care (mean 5.84, SD 1.08), physician satisfaction (mean 5.84, SD 1.22), the time from bed availability to ward transfer (mean 5.53, SD 1.72) and the number of staffed acute care beds (mean 5.53, SD 1.57).

Data reporting

Data from the first round were positively skewed and the median is the appropriate measure to describe the importance of the indicators. The responses were normally distributed in the second round and therefore the mean is the appropriate measure of the relative importance of the indicators selected from round 1.

Discussion

ED overcrowding is an important issue in many countries.14 One conceptual model used to understand ED overcrowding is the input-throughput-output model.15 The input component refers to measures of the number of patients seeking ED care. The throughput component refers to factors related to ED efficiency, workload and capacity. The output component involves measurement of the efficiency and capacity of the inpatient system to admit patients requiring hospital care and of the ambulatory care system to provide timely care after discharge.4,15,16 The use of such a model can provide an approach to grouping ED overcrowding indicators identified in this study.

This study identified and ranked 10 indicators that are considered important for reporting ED overcrowding in Canada. The most important indicator identified by Canadian experts was the percentage of the ED occupied by inpatients. This factor has also been referred to as access block, emergency inpatients (EIP) or "being boarded" in the ED, and it constitutes an area of emerging research activity.17-20 It describes a situation in which admitted patients requiring hospitalization remain in the ED for an extended period of time before being transferred to an inpatient bed.

Other indicators in the top categories, such as the total ED patients, the total time in the ED, the percentage of time the ED is at or above capacity and the overall bed occupancy, likely represent similar concepts. The indicator of the total number of ED patients must be interpreted with caution, as it is clearly dependent on ED capacity. For example, 50 patients in a 30-bed ED represents severe overcrowding, whereas 50 patients in a 75-bed ED may not. Moreover, it could be argued that the overall bed occupancy and the percentage of time that an ED is at or above capacity represent an additional form of access block. Clearly, for experts in emergency medicine, the delays regarding flow through the department (throughput indicators) and into admitted beds (an output indicator) are important contributors to ED overcrowding. The majority of indicators selected by consensus addressed factors affecting the ability of the ED to cope with the inflow of patients (input indicators), followed by indicators of the ability of the inpatient and ambulatory care systems to provide care after ED discharge (output indicators).

This study is part of a recent trend in the emergency medicine research community to develop reliable measures of ED overcrowding. Some investigators have used the terms overcrowding and crowding interchangeably to describe this problem.3 We used the term "overcrowding" in this study; however, we do not wish to imply that "crowded" emergency departments are acceptable. Composite crowding measures have been developed4-9,21,22; however, they are often cumbersome and to date have rarely been applied outside their places of development. Our study reflects the state of knowledge and belief on Canadian ED overcrowding in 2005-2006, and the list of indicators may require revision as newer evidence emerges.

Studies using consensus methodology have demonstrated that experts' opinions may be a reliable measure of the importance of certain indicators of ED overcrowding. Solberg and colleagues4 used a group consensus method with 74 experts from the United States and reported that "ED throughput time" and "ED boarding time" were the most important throughput and output measures, respectively. Two of the indicators included in our study ("total time in the ED"— a throughput indicator — and "time from bed request to bed assignment"— an output indicator) — had similar definitions and were ranked number 2 and number 6 out of 10, respectively, by the experts in our study. This suggests that Canadian ED operations experts have similar attitudes toward the importance of certain types of indicators.

A qualitative Canadian study by Estey and colleagues23 identified 2 important factors to characterize ED overcrowding: inefficient access to ED beds due to slow throughput of patients and staff shortages. Schull and colleagues5 assembled a group of clinical and administrative experts in pre-hospital, ED and hospital settings in Canada to develop a standard definition and a list of determinants for ED overcrowding.5 Ambulance diversion was deemed an appropriate operational definition; however, in many Canadian centres ambulance diversion is not possible or is not permitted. Other factors that were considered as potentially important determinants included the number of admitted patients held in the ED, intermittent surges in numbers of newly arriving ambulances and ambulatory patients, ED physician staffing, ED physician characteristics, ED nurse staffing, the availability of ED social work and geriatric teams, response times to ED consultation requests, the enforcement of ED consultation timeliness policies, ED design (e.g., the number of stretchers and cardiac monitors, and the size of department) and the availability of radiologic imaging off-hours.

The top 10 indicators we report differ from previous studies and this may reflect the changing nature of our understanding of the problem of ED overcrowding. Our indicators loosely concur with measures that researchers in English-speaking countries outside of North America have considered important. Access block and daily total patient care time are considered the key indicators of ED overcrowding in Australia.24,25 In England in 2004, the National Health Service set a 4 hour target for the maximum total time patients spend in the ED (the third ranked measure in our study).26-28 In the United States, access block caused by boarding inpatients has been considered a major contributor to crowding.4,29,30

The results of this Delphi study suggest that experts consider the "satisfaction of emergency physicians working in the ED" to be an indicator of ED overcrowding; whatever form overcrowding takes, it inevitably leads to a reduction in physician satisfaction. Although physician dissatisfaction cannot be considered a direct measure of ED overcrowding, it certainly can be an indicator of the problem. Weiss and colleagues31 found that the perceptions of ED physicians, and nurses and their feeling of being rushed, correlate with 7 "objective" measures of ED overcrowding: the number of patients in the waiting room, at triage and at registration; the number of full rooms; hallway patients; patients awaiting beds; and the total number of patients registered. The low rating assigned to staffing indicators differs from Estey's results and is concordant with the results of Schull and colleagues,32 in which the numbers of nurse-hours were not significantly associated with overcrowding.

A central limitation of this study is that, while we identified the perceived best indicators of ED overcrowding, we could not, in all cases, identify which measure of these indicators is best for quantifying and comparing overcrowding between locations. For example, "total ED volume" or the "number of ED nurses" are not, themselves, direct measures of ED overcrowding; they become indicators when placed in the context of a hospital's capacity. Other indicators, such as "the percentage of ED occupied by inpatients" are standardized and do not require a site-specific context to make them meaningful. For this reason, some authors have considered the former to be surrogate, rather than direct measures of ED overcrowding.31 Similarly, while "total time in the ED" was identified as a leading indicator in this study, our results do not identify the most appropriate reporting metric. For example, "Length of Stay" (LOS) may be evaluated based on patient characteristics (e.g., all patients v. triage level 3), cut-points (e.g., 4 v. 6 v. 8 h) and various statistical techniques (e.g., 90 th percentile, mean, median). Since recent editorials suggest reporting simpler outcomes that all sites can generate, such as LOS,33 consensus is required to identify the best methods of reporting such indicators. Thus, while we have identified the best "indicators" to document ED overcrowding, these indicators do not each constitute a direct measure of overcrowding, but rather a variable that provides descriptive or predictive information about the problem. The next critical research step is to identify, for each indicator, the best measures that correlate with perceived overcrowding.

Our study has other limitations that should be considered when interpreting the results. Owing to our sampling method, the Delphi group was predominantly represented by experts from large, urban, academic hospitals, thereby limiting our ability to generalize our findings to smaller settings. Data from a recent survey of Canadian ED directors indicated that ED overcrowding in Canada is more likely to occur in EDs with 50 000 visits per year, communities with a population of at least 150 000, university-affiliated hospitals, trauma centres and EDs with 30 or more treatment spaces.2 The participants in this Delphi study were derived from similar sampling to the one we employed, and since overcrowding may be a problem in some rural settings, this represents a potential bias.30 Our method of sampling also undoubtedly affected the type of ED provider represented in our study. The Delphi group was predominantly represented by emergency physicians, and the responses obtained with the Delphi method may not represent the opinions of other ED providers (e.g., nurses, administrators and others) affected by ED overcrowding.

Regional variations in the perception of the causes and the impact of ED overcrowding across Canada may not be captured completely by the set of indicators that were chosen. Measures selected from the medical literature may be influenced by variations in time, their availability and the settings where the studies were conducted. The importance of some indicators may vary according to local criteria and may change over time.

There is no universally accepted method of achieving consensus on both the number of rounds or the numerical values that might indicate consensus in Delphi studies.10,34 Based on recommendations on the theoretical foundations of the Delphi,35 we decided to stop after 2 rounds by using a "statistical group response" as an indicator of consensus. The Delphi method described here used an email questionnaire, whereas classic Delphi studies use mailed questionnaires. Despite the reported low response rate of email surveys,36 the high response rate in this study suggests that the email format did not discourage recipients from responding.

Conclusion

The indicators presented in this study have face validity and may help to develop and guide improvements in uniform ED data collection systems to track overcrowding across Canada. This study contributes to the body of Canadian research on important ED overcrowding measures, it reaffirms the perceived importance of length of stay and volume measures in the Canadian context, and it sets a potential agenda for future research reporting and administrative oversight.

We identified 10 indicators that were considered by consensus among a group of Canadian ED experts to be relevant and clinically important indicators of ED overcrowding. There remains an urgent need for rigorous assessment of these indicators in Canada. Such an assessment would optimize the effectiveness, applicability and adoption of appropriate indicators by Canadian clinicians, administrators, policy makers and ED overcrowding researchers. Most of the indicators we identified can be obtained in systems with sophisticated ED information systems.37 Moreover, further extension of our work should facilitate intra- and inter-institutional comparisons and assist in designing and implementing interventions to reduce overcrowding.

References

1. Canadian Association of Emergency Physicians / National Emergency Nurses Affiliation. Joint position statement: access to acute care in the setting of emergency department overcrowding. Can J Emerg Med 2003;5:81-6.
2. Rowe BH, Bond K, Ospina MB, et al. Frequency, determinants, and impact of emergency department overcrowding in Canada. [Technology Report No. 67.3]. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; 2006.
3. Hwang U, Concato J. Care in the emergency department: How crowded is overcrowded? Acad Emerg Med 2004;11:1097-101.
4. Solberg LI, Asplin BR, Weinick RM, et al. Emergency department crowding: consensus development of potential measures. Ann Emerg Med 2003;42:824-34.
5. Schull MJ, Slaughter PM, Redelmeier DA. Urban emergency department overcrowding: defining the problem and eliminating misconceptions. Can J Emerg Med 2002;4:76-83.
6. Weiss SJ, Derlet R, Arndahl J, et al. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004;11:38-50.
7. Derlet RW, Weiss SJ, Ernst AA, et al. Development of an emergency department overcrowding scale. Results of the national ED overcrowding study (NEDOCS). Acad Emerg Med 2002; 9:366.
8. Reeder TJ, Burleson DL, Garrison HG. The overcrowded emergency department: a comparison of staff perceptions. Acad Emerg Med 2003;10:1059-64.
9. Bernstein SL, Verghese V, Leung W, et al. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med 2003;10:938-42.
10. Crisp J, Pelletier D, Duffield C, et al. The Delphi method? Nurs Res 1997;46:116-8.
11. Broomfield D, Humphries GM. Using the Delphi technique to identify the cancer education requirements of general practitioners. Med Educ 2001;35:928-37.
12. Beattie E, Mackway-Jones K. A Delphi study to identify performance indicators for emergency medicine. Emerg Med J 2004; 21: 47-50.
13. Taylor WJ. Preliminary identification of core domains for outcome studies in psoriatic arthritis using Delphi methods. Ann Rheum Dis 2005;64(Suppl 2):ii110-22.
14. Graff L. Overcrowding in the ED: an international symptom of health care system failure. Am J Emerg Med 1999;17:208-99.
15. Asplin BR, Magid DJ, Rhodes KV, et al. A conceptual model of emergency department crowding. Ann Emerg Med 2003; 42: 173-80.
16. Fatovich DM. Emergency medicine. BMJ 2002;324:958-62.
17. Australasian College for Emergency Medicine. Access block and overcrowding in emergency departments. 2004. p 1-22. Available: www.acem.org.au/media/Access_Block1.pdf (accessed 2007 Mar 7).
18. Australasian College for Emergency Medicine. The relationship between emergency department overcrowding and alternative after-hours GP services. West Melbourne: The College; 2004. Available: www.medeserv.com.au/acem/open/documents/after_hoursgp.pdf (accessed 2007 Mar 7).
19. Fatovich DM, Nagree Y, Sprivulis P. Access block causes emergency department overcrowding and ambulance diversion in Perth, Western Australia. Emerg Med J 2005;22:351-4.
20. Falvo T, Grove L, Stachura R, et al. The opportunity loss of boarding admitted patients in the emergency department. Acad Emerg Med 2007;14:332-7.
21. Reeder TJ, Tucker JL, Cascio ES, et al. Trends in emergency department utilization. Effect of changing demographics. Acad Emerg Med 2001;8:577.
22. Richardson DB. A new definition of emergency department overcrowding using point occupancy. Acad Emerg Med 2004; 11: 462.
23. Estey A, Ness K, Saunders LD, et al. Understanding the causes of overcrowding in emergency departments in the Capital Health Region in Alberta: a focus group study. Can J Emerg Med 2003;5:87-94.
24. Cameron PA, Campbell DA. Responses to access block in Australia: Royal Melbourne Hospital. Med J Aust 2003;178:109-10.
25. Richardson DB. Prospective validation of point occupancy definition of overcrowding. Acad Emerg Med 2004;11:462-3.
26. Locker TE, Mason SM. Analysis of the distribution of time that patients spend in emergency departments. BMJ 2005;330:1188-9.
27. Alberti G. Transforming Emergency Care in England. COI Communications for the Department of Health, United Kingdom; 2004. Available: www.dh.gov.uk/assetRoot/04/09/17/81/04091781.pdf (accessed 2007 Mar 7).
28. NHS Modernisation Agency. Emergency services collaborative. improvements in emergency care: case studies. NHS Modernisation Agency; 2002. Available: www.modern.nhs.uk/esc/8237/Case%20Studies%20%20Low%20Res%20.pdf (accessed 2007 Mar 7).
29. General Accounting Office. Hospital emergency departments: crowded conditions vary among hospitals and communities. Washington (DC): The Office; 2003.
30. Institute of Medicine of the National Academies. Hospital-based emergency care: at the breaking point. Washington (DC): The Institute; 2006. Available:www.iom.edu/CMS/3809/16107/35007.aspx (accessed 2007 Mar 7).
31. Weiss SJ, Arndahl J, Ernst AA, et al. Development of a site sampling form for evaluation of ED overcrowding. Med Sci Monit 2002;8:549-53.
32. Schull MJ, Lazier K, Vermeulen M, et al. Emergency department contributors to ambulance diversion: a quantitative analysis. Ann Emerg Med 2003;41:467-76.
33. Asplin BR. Measuring overcrowding: time for a paradigm shift. Acad Emerg Med 2006;13:459-61.
34. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008-15.
35. Rowe G, Wright G, Bolger F. Delphi: a re-evaluation of research and theory. Technol Forecast Soc Change 1991;39:235-51.
36. Harris DR, Connolly H, Christenson J, et al. Pitfalls of email survey research. Can J Emerg Med 2003;5(3). Available: http://caep.ca/template.asp?id=E6946BBBF1804F4AAEF600DAF7F37B63#079 (accessed 2007 Mar 7).
37. Rowe BH, Bond K, Ospina MB, et al. Data collection on patients in emergency departments in Canada. Can J Emerg Med 2006; 8:417-26.
38. Beveridge R, Clarke B, Janes L, et al. Canadian Emergency Department Triage and Acuity Scale: implementation guidelines. Can J Emerg Med 1999;1(Suppl 3). Available: www.caep.ca.