Perceptions of emergency medicine residents and fellows regarding competence, adverse events and reporting to supervisors: a national survey

EM Advances

CJEM 2010;12(6):491-499

Abstract

Résumé

Introduction

Background

In Canadian hospitals, 7.5% of admissions are associated with an adverse event (AE) and a trend exists toward higher numbers of AEs in teaching hospitals.1 Almost 40% of AEs have been reported to be preventable. In Canada AEs result in 23 750 deaths per year and $750 million in expenses.2,3 Similar results have been reported in US centres.4,5

Studies suggest high rates of AEs involving residents. In a survey of 689 residents, 6.6% reported being responsible for an AE in the prior week.6 Inadequate experience and knowledge have been cited as important causes of AEs involving residents.7,8 Residents and fellows may be predisposed to involvement in AEs because of the nature of their educational experience.9 Many aspects of medical learning involve trial and error in a culture that is often characterized by the mantra “See one, do one, teach one.” Furthermore, patients in teaching hospitals often have complex illnesses, present late in their illness and require more sophisticated care.10 Finally, residents and fellows take on new responsibilities and perform increasingly complex procedures with little or no supervision, and often work long hours, resulting in sleep deprivation.8,9 In 2008, the Institute of Medicine recommended new safety measures that included alleviating fatigue and sleep loss among residents, increasing their supervision by senior physicians and improving transfers of care.11

Importance

Reporting of AEs is essential to the identification and improvement of hazardous processes, and offers the additional advantages of emotional relief, enhanced learning, improvement in personal practice and decreased likelihood of legal liability.12,13 However, current methods of AE reporting create barriers and hence poor rates of reporting. Physicians are reluctant to report AEs to a superior, and studies indicate 50%–96% of AEs are unreported.14-16 Barriers to reporting include a high workload, time-consuming protocols, lack of feedback, belief that the incident was trivial and fears regarding litigation with resultant career impact.17,18 Physician compliance with other forms of mandatory reporting — such as reporting of communicable disease — is similarly poor.19

Residents and fellows may be reluctant to report feeling not competent or “safe” to their supervisor for similar reasons. In a study of 654 internal medicine residents, only 60% felt prepared to be a member of a cardiac arrest team and 55% suspected that they had made errors during cardiac arrests.20 Self-awareness and responsibility for safe care are tied directly to competencies within common residency training paradigms, such as the “Manager” and “Professional” roles within the CanMEDS framework.10,21 The moral and professional duty of a trainee to report feeling not competent conflicts with fears of recrimination or negative professional impact if such feelings are reported. Authority gradients can act as barriers to reporting when senior physicians involved in the reporting structure have influence in the career advancement of the trainee.22,23

Goals of this investigation

The purpose of this study was to characterize the perceptions of emergency medicine (EM) residents and fellows of their clinical and procedural competence, as well as their attitudes, practices and perceived barriers to reporting these perceptions to their supervisors.

Methods

Study design

A survey instrument was designed using Survey Monkey. Quantitative data were collected using a 5-point agreement scale (ranging from strongly disagree to strongly agree) and a 5-point frequency scale (ranging from rarely to almost always) as appropriate. Qualitative data was captured as free text. The survey was piloted by an EM residency program director and 4 EM residents for face and content validity, and revised accordingly.

An AE was defined as “a complication, injury, or harm to a patient resulting from medical management (not from the patient’s underlying condition or disease).”

Setting and participants

An anonymous Web-based survey was carried out from January through March 2008 of all Canadian EM residents outside of Quebec in the EM programs of the Royal College of Physicians and Surgeons of Canada and The College of Family Physicians of Canada. The program directors were contacted by email and/or telephone and asked to forward an email to all EM residents and fellows. The email contained an overview of the study and electronic link to the study website; the website contained an introductory letter to the study, a detailed consent form and the survey instrument.

Data collection and processing

Using a modified Dillman approach, each participating program director was sent 3 weekly reminder emails to reforward the invitation.24 Participating program directors provided the investigators the number of trainees to whom they sent the initial invitation and follow-up reminders, but not their identity. Data collection was structured to preserve anonymity, and participants were asked to refrain from submitting any identifying information for themselves, their patients or their institutions. Participants were instructed to complete the survey only once. Program directors were contacted after a 4-week period and at the end of the study period to confirm their compliance with the protocol.

Data analysis

Data analysis was performed using Microsoft Excel 2003. Proportions with 95% confidence intervals (CIs) were generated for descriptive statistics. We used t tests for continuous variables and χ2 for categorical variables for statistical testing.

The study was approved by the University Health Network Research Ethics Board.

Results

Response and demographic characteristics

Nine of 10 FRCP(EM) and 12 of 13 CFPC(EM) program directors participated; one declined to participate, and one could not be reached despite numerous attempts. All participating program directors confirmed that they had forwarded the initial invitation letter and 2 follow-up emails to their trainees. The response rate was 37.3%; 220 EM trainees were invited and 82 completed all or part of the survey (78 residents and 1 fellow completed all of the survey and 3 did not complete the demographics section).

Demographics

Demographic information is provided in Table 1. Of the 79 respondents who completed the demographic section of the survey, 45 (57.0%) were in an FRCP(EM) program and 34 (43.0%) were in a CCFP(EM) program. The CCFP(EM) program runs as a single year after 2 years of training in family medicine, thus all CCFP(EM) participants were in their third postgraduate year, and represented 82.9% of the 41 respondents in their third postgraduate year.

Table 1. Demographic characteristics of the study population

Characteristic	No. (%) of respondents*
*Unless otherwise indicated. †Some respondents did not complete the demographics section of the survey.
Sex (n = 79)
Male	43	(54.4)
Female	36	(45.6)
Mean (range) age, yr (n = 77†)	29.5	(25–37)
Postgraduate year (n = 79)
1	13	(16.5)
2	7	(8.9)
3	41	(51.9)
4	11	(13.9)
5	6	(7.6)
≥ 6	1	(1.3)
Medical training (n = 79)
Canada	70	(88.6)
United States	1	(1.3)
Elsewhere	8	(10.1)
Language spoken at home (n = 78†)
English	70	(89.7)
French	1	(1.3)
Other	7	(9.0)

Perceptions of personal competence

Trainees were asked, “When was the last time you performed a procedure or undertook a clinical responsibility with minimal or no supervision which you did not feel confident performing independently?” Twelve trainees (14.8% [95% CI 6.5%–23.2%]) had experienced this in the past month, and 32 (39.5% [95% CI 28.2%–50.8%]) indicated that this had occurred within the past 6 months.

Table 2 provides information on procedures or responsibilities performed unsupervised despite trainees not feeling competent or “safe” to perform these independently. Nights on-call, admission decisions and invasive procedures were most frequently undertaken despite trainees not feeling competent.

Table 2. Responses to the question, “What are the procedures or responsibilities that you have performed unsupervised even though you did not feel competent or ‘safe’ to perform independently?”

Response option	Response, %	Response count*	SE, %	95% CI
CI = confidence interval; SE = standard error. *This question was answered by 79 respondents and skipped by 3 respondents.
Providing on-call care overnight	38.0	30	5.5	26.6%–49.3%
Complex wound care	27.8	22	5.0	17.3%–38.4%
This situation has not occurred	25.3	20	4.9	15.1%–35.5%
Inserting a central line	16.5	13	4.2	7.6%–25.3%
Deciding whether to discharge or admit a patient	16.5	13	4.2	7.6%–25.3%
Supervising a resuscitation or cardiac arrest	11.4	9	3.6	3.8%–19.0%
Breaking bad news	10.1	8	3.4	2.8%–17.4%
Endotracheal intubation	8.9	7	3.2	2.0%–15.8%
Lumbar puncture	3.8	3	2.2	0.0%–8.6%
Obtaining an arterial blood gas	2.5	2	1.8	0.0%–6.6%
Cardioversion/defibrillation	1.3	1	1.3	0.0%–4.4%
Other	12.7	10	3.7	4.7%–20.6%

Reporting

Table 3 provides information on reporting. Almost one-third of the study participants (25/82, 30.5% [95% CI 19.9%–41.1%]) agreed with the statement, “I sometimes feel unsafe or unqualified when undertaking certain unsupervised responsibilities or procedures that I am expected to do, but I do not report this to my senior physician.” Moreover, 34 of 82 (41.5% [95% CI 30.2%–52.7%]) reported their lack of competence to a supervisor half the time or less.

Table 3. Responses to the question, “When you feel unsafe or not competent to undertake a clinical responsibility independently, how often do you report this to your supervising physician (resident or staff)?”

Response option	Response, %	Response count*	SE, %	95% CI
CI = confidence interval; SE = standard error. *This question was answered by 82 respondents and skipped by 0 respondents.
Rarely (0%–20% of the time)	9.8	8	3.3	2.7%–16.8%
Infrequently (21%–40% of the time)	9.8	8	3.3	2.7%–16.8%
About half the time (41%–60% of the time)	22.0	18	4.6	12.4%–31.5%
Very frequently (61%–80% of the time)	17.1	14	4.2	8.3%–25.8%
Almost always (81%–100% of the time)	41.5	34	5.4	30.2%–52.7%

Barriers to reporting and suggestions for improvement

The primary causes for trainees not reporting feeling unsafe or not competent, as selected from a list or entered as free text, were being worried about loss of trust, autonomy or respect by the supervising physician (38/80, 47.5% [95% CI 35.9%–59.1%]), loss of reputation with the supervising physician (32/80, 40.0% [95% CI 28.6%–51.4%]), a negative impact on evaluation (17/89, 21.3% [95% CI 11.7%–30.8%]), being too busy (23/80, 28.9% [95% CI 18.2%–39.3%]) and not wanting to bother the supervising physician (21/80, 26.3% [95% CI 16.0%–36.5%]). Only 19 of 80 respondents (23.8% [95% CI 13.8%–33.7%]) endorsed the statement, “This scenario never occurs.”

Table 4 outlines suggestions of trainees for improving reporting. These suggestions most frequently included encouragement to report without penalty and a less judgmental environment.

Table 4. Responses to the question, “What would you change in order to make you feel more comfortable reporting to a supervising physician that you do not feel safe or competent performing a procedure or undertaking a clinical responsibility?”*

Response option	Response, %	Response count†	SE, %	95% CI
CI = confidence interval; SE = standard error. *Respondents could check all that applied. †This question was answered by 82 respondents and skipped by 0 respondents.
Encouragement to report without penalty	50.0	41	5.5	38.6%–61.4%
A less judgmental environment	39.0	32	5.4	27.9%–50.2%
Periodic anonymous surveys of trainees regarding perceived competence and safety	24.4	20	4.7	14.5%–34.3%
There is no need to improve the system	19.5	16	4.4	10.3%–28.7%
Specific education in roots of medical error and/or communication	18.3	15	4.3	9.3%–27.3%
Anonymous reporting	14.6	12	3.9	6.4%–22.9%
Other	4.9	4	2.4	0.0%–10.2%

Perceptions regarding AEs and reporting

Almost two-thirds of the study participants (52/82, 63.4%) indicated that they had, in the past year, felt responsible for an AE after providing medical care or performing a procedure they had felt competent to perform independently, with more than one-third (28/82, 34.1% [95% CI 23.3%–45.0%]) reporting that this had occurred within the past 6 months.

Table 5 provides information on the consequences of the most serious AEs trainees felt responsible for. Most reported that the consequences were minor, although significant outcomes — including death — were described.

Table 5. Responses to the question, “What was the patient consequence of the most significant event?”*

Response option	Response, %	Response count†	SE, %	95% CI
CI = confidence interval; SE = standard error. *Respondents could check all that applied. †This question was answered by 80 respondents and skipped by 2 respondents.
I cannot recall responsibility for an adverse event/unknown/none apparent	58.8	47	5.5	47.3%–70.2%
Delayed treatment	16.3	13	4.1	7.5%–25.0%
Pain	13.8	11	3.9	5.6%–21.9%
Impairment with expected recovery	13.8	11	3.9	5.6%–21.9%
Increased hospital stay	12.5	10	3.7	4.6%–20.4%
Delayed diagnosis	12.5	10	3.7	4.6%–20.4%
Permanent impairment with little disability (e.g., scar)	5.0	4	2.4	0.0%–10.4%
Death	3.8	3	2.1	0.0%–8.5%
Permanent impairment with great disability (e.g., amputation, myocardial infarction)	2.5	2	1.7	0.0%–6.5%
Other	5.0	4	2.4	0.0%–10.4%

When asked to describe the most significant medical AE for which they felt some responsibility, trainees most frequently described complications related to procedures — primarily insertions of chest tubes, central lines and paracentesis (22/55, 40.0%), errors in diagnosis or treatment (15/55, 27.3%) and errors related to management of medication (8/55, 14.5%).

Trainees described heterogeneous rates of reporting of AEs. The majority stated that they had reported the most serious AE for which they felt responsible to their attending physician (40/63, 63.5% [95% CI 50.8%–76.2%] and/or to their senior resident (22/63, 34.9% [95% CI 22.4%–47.5%]). Less than one-quarter (15/63, 23.8% [95% CI 12.5%–35.1%]) reported the event to the patient or to the patient’s family. Only 6 of 63 (9.5%) reported the event through their hospital reporting system. Of note, 12 of 63 trainees (19% [95% CI 8.6%–29.5%]) did not report the worst AE for which they felt responsible to either their senior resident or their attending physician. The response rate to this question was 76.8% (63/82).

Barriers to reporting AEs

Table 6 provides information on barriers to reporting. The most frequently identified barrier to reporting AEs was worry about loss of trust, autonomy or respect from the supervising physician. Social barriers related to authority gradients were more frequently described than those related to administrative or systems functions such as not knowing what events to report or being too busy and forgetful. Occasional respondents identified fear of appearing incompetent (1 respondent, 1.3%) and worry about public humiliation and shame (1 respondent, 1.3%), as exemplified by the statement, “I also find many nurses are publicly very derogatory towards residents if an error/poor judgment occurs on the service.”

Table 6. Responses to the question, “What are the barriers which prevent you from reporting such adverse events to your supervising physician (resident or staff)?”*

Response option	Response, %	Response count	SE, %	95% CI
CI = confidence interval; SE = standard error. *Respondents could check all that applied.
I report all events	45.0	36	5.6	33.5%–56.5%
Worried about loss of trust, autonomy or respect by your senior	35.0	28	5.3	23.9%–46.1%
Don’t know which events to report	18.8	15	4.4	9.6%–27.9%
Too busy and/or forget to report	15.0	12	4.0	6.6%–23.4%
Residents are unfairly blamed for errors	12.5	10	3.7	4.6%–20.4%
It may negatively impact my evaluation	12.5	10	3.7	4.6%–20.4%
Competitive environment	8.8	7	3.2	1.9%–15.6%
Worried about impacts on career	5.0	4	2.4	0.0%–10.4%
Worried about litigation (a lawsuit)	3.8	3	2.1	0.0%–8.5%
Lengthy reporting procedure	2.5	2	1.7	0.0%–6.5%
My senior does not want me to report	2.5	2	1.7	0.0%–6.5%
Reporting increases my workload	1.3	1	1.2	0.0%–4.3%
Other (please describe)	7.5	6	2.9	1.1%–13.9%
None	3.8	3	2.1	0.0%–8.5%

Discussion

Perceptions of not being competent or safe

Approximately 40% of EM residents and trainees had, within the 6 months before completing the survey, felt not competent performing their duties but did not report this to their supervisor. Trainees frequently reported feeling unsafe when on-call overnight, deciding to admit or discharge patients and undertaking invasive procedures. These responses — coupled with a frequent reluctance to bring such feelings to the attention of supervising physicians — are patient safety concerns, as trainees are front-line providers integral in providing off-hour care in teaching hospitals.

Several studies have found increased morbidity and mortality on the weekends in teaching hospitals, when trainees are less closely supervised.25,26 This may relate to procedural and clinical competence. In a Canadian survey of internal medicine residents, 44.3% indicated having felt overwhelmed leading resuscitations in cardiac arrest, and 55.3% worried that they had made errors during such events.20 In a survey of Canadian pediatric EM fellows, the median self-estimated success rate for internal jugular and subclavian central line insertion was 50% or less.27

Although it could be argued that feeling unsafe is a normal part of the learning process and may not reflect actual abilities, resident experiences with AEs suggest that their perceptions may often be valid. Singh and colleagues28 analyzed malpractice claims in which trainees were judged to have played an important role in harmful errors. Teamwork breakdowns — primarily a lack of supervision and hand-off problems — were contributing factors in 70% of cases, and lack of technical competence were contributing factors in 58% of cases.

Reporting practices

In our study, rates of residents failing to report AEs were comparable to rates of residents failing to report feelings of not being competent. Farnan and colleagues29 interviewed residents about important clinical decisions made while on call and about their experiences with clinical uncertainty. They found reluctance among residents surveyed to seek advice from staff that often prompted initial reliance on peers in a manner that, on occasion, resulted in delays in indicated care and even patient harm. Similar low rates of resident reporting of AEs have been described in other studies.6,7,12

Improving safety culture in the training environment

Residents and fellows endorsed a training environment that is less punitive and places more emphasis on reporting. Medical education appears to be evloving in that direction. Whereas Leape30 remarked in 1998 that “physicians are socialized in medical schools to strive for error-free practice,” Cuschieri31 noted in 2006 that efforts across multiple health delivery systems were being undertaken to move from the defensive “blame and shame culture” to an open and transparent health care delivery system.

Proposed changes to improve reporting are described in the literature. Curricula for teaching patient safety to medical students and trainees have been developed, focusing on awareness of error, cognitive sources of error and cognitive forcing strategies to mitigate error.32 Thompson and colleagues33 described a multidisciplinary system-based curriculum of safety and AE reporting introduced for first-year medical students that showed that perceptions of patient harm and the role physicians play in patient safety can be changed.

Error-reducing strategies have been described for practice at the residency level. Gutierrez and colleagues34 note that physicians may be more inclined to report near misses — as they occur more frequently and with fewer consequences to the patient — and advocate integrating the reporting of near misses into the shift sign-out routine. These authors also advocate analyzing reported events and incorporating reporting of errors into medical school curricula as early as first year. The addition of an online log of error reporting as part of residents sign-out routine has been described as a timely and convenient process to encourage reporting.35 Rodriguez-Paz and colleagues36 proposed a new training paradigm that integrates competency-based knowledge and clinical skills with deliberate attitudinal and behavioural changes focused on patient safety in a simulated environment. The Canadian Patient Safety Institute published a safety framework that identified 20 key competencies and 140 enabling competencies for the patient safety education of health care professionals.37

Most competency-based frameworks include professionalism and management skills and include principles of patient safety and quality improvement. Deriving specific enabling competencies to describe self-awareness in practice, and designing instruments such as evaluation forms and multisource feedback tools to address these competencies may increase awareness and acceptability. Similarly, modifying trainees’ evaluations of teachers to include receptiveness to being contacted and openness to a trainee’s declaration of lack of competence may have similar affect. Web-based reporting systems are being trialed in some hospital environments in an effort to increase reporting by all hospital staff, and trainees should be included in these initiatives. Finally, as these issues arise from cultural constructs, systematic change from the top down is a prerequisite for lasting change, and necessitates leaders, teachers and supervisors to act as, and be perceived as, role models. Accordingly, educational leaders must emphasize that supervisors should be receptive to trainees’ concerns and respond appropriately and constructively. Further to this, the system must enable leaders to address teachers who do not perform appropriately. Remediation structures, guidelines for supervision and policies regarding intimidation and harassment of residents should be routine in academic centres.

Medicolegal considerations

One theme arising in this study was that trainees frequently indicated they did not feel it was ultimately their responsibility to disclose their perceived lack of competence; rather, it was ultimately the supervisor’s job to determine this. This should serve as a cautionary finding for attending physicians, as the medicolegal implications of residents performing assigned duties despite not feeling competent can be shared by trainees and their supervisors.

The Canadian Medical Protective Association serves as the primary medicolegal advisors for 95% of Canadian physicians. In an article circulated to all its members, the association wrote the following:

In legal actions [that consider the responsibility of supervising physicians], the court generally considers whether the delegation, supervision and support provided by the supervising physician were reasonable in the circumstances. Supervising physicians should therefore consider the following: “Is the task appropriate to delegate to an individual with the trainee’s level of training? Does the specific trainee have the required knowledge, skill and experience to perform the task? What degree of supervision is required?” ... Open and supportive communication by the supervisor and readiness to help the trainee will allow the trainee to voice any concerns about a task. The trainee should feel free to clarify instructions, voice concerns and ask the supervisor for help.38

The article also addresses the medicolegal responsibility of trainees:

In legal actions, the court generally considers whether the trainee performed at a standard which could reasonably be expected. Trainees and residents should therefore: Recognize the limits of their knowledge. Exercise caution and consider their inexperience. Notify their supervisors of their knowledge, skill and experience with the delegated task. Keep the supervisor informed of their actions. Inform patients of their status as medical trainees.38

Limitations

Several limitations should be considered when interpreting our results. First, our response rate of 37.3% is low and our resulting sample may not reflect the population of trainees as a whole. We suspect our low response rate arose both because of the sensitive nature of the questions and by our dependence on 2 tiers of participation (recruiting program directors to recruit trainees), as confidentiality precluded release of the email addresses of trainees to the researchers. This said, our response rate is comparable to or better than those seen in similarly designed Web-based studies of medical trainees and in the safety reporting literature.20,39,40

Our study population involved Canadian EM trainees outside of Quebec. As a result, our findings are not necessarily generalizable to residents and fellows in Quebec or in training programs outside of Canada.

Nonresponder bias may have influenced recruitment given the sensitive nature of the study topic, and the experiences of the subgroup that participated may be different from those who chose not to participate in the survey. Moreover, participants may have felt compelled to give socially acceptable responses.

However, even if all nonresponders perceived full competence and experienced AE-free practice, our data would still suggest that the frequency of trainee underreporting to superiors is significant, and a need exists to improve the training culture.

Conclusion

Emergency medicine residents and fellows report that they frequently don’t feel competent when undertaking responsibilities without supervision. Barriers to expressing these concerns and reporting AEs relate to social pressures and authority gradients. Modification of the training culture is warranted as part of efforts to improve patient safety. Supervising physicians, as educators, should readily assess the level of experience of trainees and foster a culture that facilitates reporting. A low response rate was a limitation to this study and its findings.