Diagnostic accuracy and clinical utility of emergency department targeted ultrasonography in the evaluation of first-trimester pelvic pain and bleeding: a systematic review

Review Articles

CJEM 2009;11(4):355-364

Abstract

Résumé

INTRODUCTION

Emergency department targeted ultrasonography (EDTU) describes a diagnostic test in which an emergency physician (EP) performs a focused ultrasonography examination on emergency department (ED) patients with specific clinical problems. EDTU is used to rapidly identify life-threatening conditions in order to expedite treatment.1

The primary purpose of EDTU in evaluating patients with first-trimester pain or bleeding is to exclude the diagnosis of ectopic pregnancy,2,3 which is effectively ruled out if an intrauterine pregnancy (IUP) is identified. This is done by visualizing an intrauterine gestational sac that contains either a yolk sac or fetal pole.4,5

To avoid missing patients with ectopic pregnancies, EDTU must be highly specific for the detection of IUP. The proportion of false positives must be small so that patients with an ectopic pregnancy will not be erroneously discharged from the ED. False negatives are of lesser consequence, since patients who have no IUP visible on EDTU will undergo further evaluation to exclude ectopic pregnancy.

The objectives of this systematic review were 2-fold. First, we comprehensively reviewed the literature describing the diagnostic accuracy of EDTU, and we calculated a pooled estimate of the specificity and sensitivity of EDTU for IUP. Second, we evaluated the clinical impact of the use of EDTU in the assessment of first-trimester pelvic pain or bleeding, including time to diagnosis and time to treatment of ectopic pregnancy, patient length of stay (LOS) in the ED and cost-effectiveness.

METHODS

One author (A.M.) performed the literature review, screened abstracts and selected articles for full-text review. The search strategy, which was developed in conjunction with a health sciences librarian, is described in Table 1.4

Two authors (H.M., M.E.) independently reviewed the retrieved articles to determine eligibility for inclusion. These 2 authors also independently abstracted results from included studies. Disagreements were resolved by consensus with input from the third author (A.M.), who also independently reviewed the articles.

The prespecified inclusion criteria for studies examining the diagnostic accuracy of EDTU were based on guidelines for a quality assessment tool of diagnostic accuracy studies (QUADAS) for systematic reviews of diagnostic tests6and standard questions for the critical appraisal of reports of a diagnostic test.7The inclusion criteria were as follows: 1) prospective enrolment; 2) unselected, pregnant emergency patients presenting with pelvic pain or bleeding in the first trimester; 3) the use of transabdominal, transvaginal, or both EDTU techniques, performed in EDs by EPs; 4) comparison of EP EDTU interpretation with a gold standard incorporating any of the following: ultrasonography performed in radiology departments or by gynecology consultants, surgical findings, pathology reports, or telephone or clinical follow-up; and 5) presentation of sufficient data to allow calculation of sensitivity and specificity, classifying a positive finding of IUP as the observation of an intrauterine gestational sac with yolk sac or fetal pole. The first 3 of these criteria were used in the initial abstract screening.

The number of true-positive, true-negative, false-positive and false-negative EDTU scans from each study were entered into a spreadsheet. For each study, the specificity and sensitivity, and their 95% confidence intervals (CIs), were recalculated de novo.8The sensitivity and specificity of EDTU for IUP was tested for statistical heterogeneity between selected studies using Fisher exact test and the I² statistic. A simple pooled estimation of the specificity and sensitivity was planned if no evidence of heterogeneity was found (Fisher exact test p > 0.10 and I²< 50%).9,10

The inclusion and screening criteria for studies examining the clinical impact of EDTU in symptomatic first-trimester pregnancy were as follows: 1) retrospective or prospective studies; 2) examined objective outcomes including mortality, ED LOS, time to diagnosis, time to surgery among patients treated operatively or costs; 3) compared outcomes in patients examined with EDTU versus patients undergoing sonography in radiology departments or by gynecology consultants.

Statistical analysis was performed using Microsoft Excel 2003 (Microsoft) and SAS Version 9 (SAS Institute). Forest plots were generated using StatsDirect Version 2.5.7 (StatsDirect, Ltd.).

RESULTS

The electronic literature search yielded 128 studies whose abstracts we screened. Of these, 19 were selected for full-text review. Five additional studies were identified on manual search of the literature, all of which were selected for full-text review. Three studies11-13 met inclusion criteria for the review of diagnostic accuracy of EDTU for IUP. Nine papers met inclusion criteria for the clinical impact systematic review (Fig. 1).12,14-21

Diagnostic accuracy of EDTU for IUP

The results of the 3 studies meeting inclusion criteria are shown in Figure 2 and Table 2.11-13,17,22 The methodological quality of these studies is described using the QUADAS criteria in Table 3.11-13

Mateer and colleagues12

This study enrolled a convenience sample of 300 patients presenting to a single ED with first-trimester pain or bleeding. A protocol employing transvaginal EDTU was used to rule out ectopic pregnancy. Patients with IUPs identified on EDTU were discharged with outpatient follow-up. Those with obvious ectopic pregnancies, or those with indeterminate EDTU findings and a serum β human chorionic gonadotropin (βhCG) level greater than 2000 mIU/mL were referred to the gynecology service. Patients with abnormal IUPs or with indeterminate EDTU findings and a βhCG less than 2000 mIU/mL were discharged with clinic follow-up within 72 hours.

Emergency physicians who performed EDTU completed 10-12 hours of training and 10-12 proctored scans. The composite gold standard included formal ultrasonography, operative reports, pathology reports, or clinical or telephone follow-up.

All 169 EDTU scans that were classified as definite IUPs were true positives. EDTU scans classified as negative for IUP included abnormal IUPs (31), ectopic pregnancies (5), and no definite IUP with βhCG either above (28) or below (67) 2000 mIU/mL. Of these 131 negative EDTU scans, 86 were true negatives, including 40 ectopic pregnancies identified on formal ultrasonography or follow-up. In this study the specificity of EDTU for IUP was 100% and the sensitivity was 79%.

Durham and coworkers11

This study enrolled 125 consecutive patients presenting to a single ED with first-trimester pain and bleeding, excluding those with a clinical diagnosis of incomplete abortion. A protocol employing transabdominal and transvaginal EDTU was used to rule out ectopic pregnancy. Patients with IUPs identified on EDTU were discharged with outpatient follow-up, and those with nondiagnostic EDTU scans underwent formal ultrasonography or specialist consultation. One patient with an ectopic pregnancy was erroneously discharged from the ED. This patient did not have a definite IUP identified on EDTU.

Emergency physicians who performed EDTU completed a 24-hour ultrasonography course, including 20 proctored scans. They had performed, on average, 50 pelvic ultrasonography procedures before the study began. The composite gold standard included formal ultrasonography and clinical or telephone follow-up.

All 87 EDTU scans that were classified as definite IUPs were true positives. Nondiagnostic EDTU scans included 12 findings of only a gestational sac, 8 ectopic pregnancies, 2 blighted ova, 1 molar pregnancy and 15 indeterminate results. The specificity of EDTU for IUP in this study was 100% and the sensitivity was 91%.

Wong and colleagues13

This study enrolled 151 consecutive patients presenting to a single ED with pain or vaginal bleeding in the first 16 weeks of pregnancy. Complete data were available on 143 patients. Transabdominal sonography was used exclusively. EDTU credentialing standards for EPs was not described. The gold standard in this study was the result of ultrasonography procedures performed by gynecologic consultants.

Of 89 EDTU scans, 88 that demonstrated a definite IUP were true positives. One false positive was a misclassified ectopic pregnancy. This ectopic pregnancy was identified on close follow-up using serial ultrasounds and βHCG levels. The patient underwent right salpingectomy with an uneventful recovery. Nondiagnostic EDTU findings included abnormal IUPs (20) or no definite IUP (34). The specificity of EDTU for IUP in this study was 92%, and the sensitivity was 67%.

There was evidence of statistical heterogeneity of the specificity (p = 0.0945, I2= 78%) and sensitivity (p < 0.05, I2= 88%) of EDTU for IUP between these included studies. For this reason, a pooled analysis of the diagnostic accuracy was not performed.

Other studies

The literature search identified 2 other studies17,22 of adequate methodological quality, but their diagnostic criterion for IUP was not clearly described. The specificity of EDTU for IUP in these studies was 98.5%22and 100%,17with sensitivities of 54%22and 94%17(Table 1). Although the specificity of EDTU for IUP was high in these studies, we did not include them in the systematic review, because we could not be certain that their diagnostic criterion for IUP met current practice guidelines.4,23

Clinical impact of EDTU in the assessment of first-trimester pelvic pain or bleeding

No studies compared mortality between patients who were evaluated either with EDTU or formal ultrasonography. Nine studies were identified that examined the effect of EDTU use on time to diagnosis or treatment of ectopic pregnancy, ED LOS or cost-effectiveness (Fig. 1).

Mateer and coworkers12

This study is also included in the review of the diagnostic accuracy of EDTU for IUP. An EDTU protocol was used to evaluate first-trimester pelvic pain or bleeding. The authors compared the proportion of missed ectopic pregnancies in the 2 years before, and the 3 years following initiation of the protocol. Of the 300 EDTU patients with complete data, 40 of these patients had an ectopic pregnancy as determined using formal sonography, operative findings or clinical follow-up. The historical control group included all 56 patients with ectopic pregnancy who were admitted through the ED in the 2 pre-protocol years. The proportion of patients with an ectopic pregnancy who were discharged from the ED was reduced from 43% preprotocol to 28% using the protocol. This effect was not statistically significant.

The proportion of patients with ectopic pregnancies who were discharged from the ED and who had ruptured on surgical exploration was reduced from 21.4% to 2.5% (p < 0.05). This study’s methodological strengths included a relatively large patient sample with only a small proportion lost to follow-up. However, patient identification and chart data abstraction methods are not adequately described to allow assessment of their validity and reliability.

Using this EDTU protocol, 11 of the 40 (27.5%) patients with ectopic pregnancy were discharged from the ED. Although this represents fewer patients with ectopic pregnancy than in the preprotocol years, it is concerning to see so many ectopic pregnancies missed. Patients erroneously discharged with ectopic pregnancies had either EDTU findings classified as “abnormal IUP” or had a nondiagnostic EDTU with a βhCG less than 200 mIU/mL. This practice is not supported by current guidelines, which recommend the use of strict diagnostic criteria for identifying an IUP (i.e., intrauterine gestational sac plus yolk sac or fetal pole).4,5

Rodgerson and coworkers14

This retrospective chart review examined the effect of adding right upper quadrant (RUQ) views to the pelvic EDTU. It included 37 patients who were admitted through a single ED with ruptured ectopic pregnancy and at least 400 mL of hemoperitoneum at the time of surgery. Patients were identified using ED and hospital-wide patient databases. Data abstractions were performed by 1 or 2 reviewers with a high degree of inter-rater reliability (κ = 0.90). Sixteen patients were examined with EDTU. All had positive findings of fluid in the RUQ. The other 21 underwent formal ultrasonography. Mean time to diagnosis was 139 minutes less for those patients with RUQ free fluid that was identified using EDTU (p < 0.0001). Mean time to surgery was 211 minutes faster for patients with free fluid identified on EDTU (p < 0.0001). These results are not generalizable to patients with unruptured ectopic pregnancies. Furthermore, the reduction in time to diagnosis and time to surgery may be a result of selection bias. The EDTU group had mean systolic blood pressure measurements 8 mm Hg lower than the formal ultrasonography group, and, on average, 100-200 mL more blood in the peritoneum. Although groups were similar with respect to heart rate, hemoglobin levels and transfusion requirements, it may be that patients in the EDTU group underwent surgery more quickly simply because they were more hemodynamically unstable.

Blaivas and Bell15

This retrospective chart review examined data from 94 patients presenting to the ED with an ectopic pregnancy requiring surgical intervention. EDTU use in 24 patients decreased mean time to surgery by 2 hours and 25 minutes, compared with 70 patients who were evaluated with sonography in the radiology department (95% CI 1 h 41 min to 3 h 9 min). Comparison groups were similar with respect to the presence of hemoperitoneum, number of days in hospital and infection rates. This report has only been published in abstract form, and offers no additional information to allow assessment of methodological quality.

Emergency department length of stay (1 randomized controlled trial, 1 prospective study and 3 retrospective studies)

Pierce and coworkers16

In this randomized trial, a convenience sample of 63 hemodynamically stable patients with first-trimester pelvic pain or bleeding were randomly assigned to undergo usual care or usual care plus EDTU (29 patients in each arm). The primary outcome was ED LOS. Both groups had similar diagnoses: 24 versus 21 IUPs, 3 versus 6 intrauterine fetal death and 2 versus 2 ectopic pregnancies in the EDTU group versus the usual care group. Patients evaluated with EDTU had shorter mean ED stays by 2.1 hours (p < 0.0001), perhaps because of a lower rate of gynecologic consultation. This study has only been published in abstract form, limiting the evaluation of methodological quality.

Shih17

This prospective, observational study enrolled 125 women with pain or bleeding in the first trimester. Patients were evaluated either with EDTU or with formal ultrasonography, depending on whether the attending emergency physician was EDTU-credentialed. Physicians were credentialed following 24 hours of training, including 10 proctored scans. The authors of this study chose to exclude 10 patients with ectopic pregnancies, “… because their LOSs would be determined by many other variables, independent of consultation status.”17None of the remaining 115 patients had an ectopic pregnancy. The 48 patients evaluated with EDTU had a mean ED stay 120 minutes shorter than the 67 patients who had ultrasonography procedures performed by radiologists or obstetrics and gynecology consultants (p < 0.001). The shorter LOS was not observed among patients with nondiagnostic scans, or among patients requiring consultation. In this nonrandomized study, it is unclear whether patients’ presenting complaints, diagnoses or demographic characteristics were similar between EDTU and standard-care groups.

Blaivas and coworkers18

This was a retrospective chart review of patients with pelvic pain or bleeding in the first 12 weeks of pregnancy. Only patients found to have an IUP were included in the analysis. Patients were evaluated either with EDTU (n = 277) or with sonography in the radiology department (n = 1142), depending on whether their physicians were EDTU-credentialed. Residents were credentialed after completing a 2-4 week ultrasonography rotation. Credentialing standards for attending physicians were not described. Patients evaluated with EDTU had a mean ED LOS 59 minutes shorter than those evaluated with formal ultrasonography (95% CI 49-77 min, p = 0.0001). This effect was more pronounced during evening and weekend hours (the mean reduction in LOS was 77 min, 95% CI 55-97 min), and was still significant during daytime hours, with a mean reduction in LOS of 48 min (95% CI 35-71 min). It is unclear whether EDTU and non-EDTU groups were similar with respect to the presenting complaint, clinical stability or other potential confounders. The only adjusted analysis was the examination stratifying by time of day.

Burgher and coworkers19

This single-centre retrospective chart review examined the records of 84 patients with first-trimester pelvic pain or bleeding who presented during a 20-week period. During the first 10 weeks of the study, patients requiring pelvic ultrasonography underwent examination and transvaginal sonography by obstetrics and gynecology consultants. During the last 10 weeks of the study, patients had transvaginal sonography performed by emergency physicians, who were credentialed following a 3-day course and 5 proctored scans. The 46 patients evaluated with transvaginal EDTU had a mean ED LOS that was 70 minutes shorter than the 38 patients whose ultrasonography procedures were performed by obstetrics and gynecology consultants (p < 0.0003). The distribution of diagnoses and sonographic findings was similar between groups, including 5 ectopic pregnancies per group, all of which were either identified on initial EDTU, or obstetrics and gynecology sonography or were admitted after having nondiagnostic initial sonograms. The methods for patient identification and chart data abstraction were not described, leaving open the possibility of missed patients and misclassification of data.

Jang and Aubin20

This abstract describes a retrospective chart review of patients with a gestation of less than 20 weeks who presented with bleeding or pain. Mean LOS for patients evaluated using EDTU was 149 minutes shorter compared with those patients whose ultrasonography procedures were performed by obstetrics and gynecology consultants, and 168 minutes shorter than those patients who were evaluated with both EDTU and ultrasonography procedures performed by obstetrics and gynecology consultants (both p < 0.05). There were no missed ectopic pregnancies among patients evaluated solely with EDTU. This study is published in abstract form only and offers no additional information on sample size or other indicators of methodological quality.

Cost savings from EDTU use (1 randomized controlled trial, 1 retrospective study)

Pierce and colleagues16

This randomized trial, described above, compared LOS and billed costs for patients randomly assigned to groups receiving examination with usual care or usual care with EDTU. The average billed charges were US$391.17 less in the group evaluated with EDTU ($535.30 v. $926.47, p = 0.18). The comparison groups were similar with respect to the use of ancillary laboratory testing and patient satisfaction.

Durston and coworkers21

This was a retrospective chart review of all women with ectopic pregnancy who were treated during a 6-year period in a single ED. The study was divided into 3 time periods during which different protocols for evaluation of symptomatic first-trimester pregnancy were employed. During the first period, on-call ultrasound technicians were available, but emergency physicians were discouraged from ordering formal ultrasonography without obstetrics and gynecology consultation. During the second period, on-call ultrasound technicians were available with no restrictions on ordering formal ultrasonography from the ED. During the third period, EDTU was employed. The study included 120 women with ectopic pregnancy who were evaluated in the ED and had complete data. There were no differences in the proportion of patients with missed ectopic pregnancies, or in time to treatment of ectopic pregnancy between these 3 protocols. Depending on the cost of EDTU implementation, the authors estimated that EDTU use in lieu of 24-hour on-call ultrasound technicians would save US$229-$1244 per case of ectopic pregnancy.

DISCUSSION

This systematic review is the first to specifically examine the evidence describing the diagnostic accuracy and clinical utility of EDTU in symptomatic first-trimester pregnancy. Another systematic review, prepared for Canadian Agency for Drugs and Technologies in Health (CADTH), examined the diagnostic accuracy of EDTU for intraperitoneal free fluid, abdominal aortic aneurysm, pericardial fluid and ectopic pregnancy.24This review included 2 studies that did not meet our inclusion criteria. One study employed unclear diagnostic criteria for IUP.17The other evaluated EDTU use in multiple clinical scenarios rather than specifically examining EDTU use for first-trimester pain or bleeding.25The authors generated summary statistics for the identification of ectopic pregnancy, including a positive likelihood ratio of 14.57 and a negative likelihood ratio of 0.08, indicating that EDTU is likely useful in identifying and excluding ectopic pregnancy. However, this pooled analysis was performed without testing for heterogeneity between studies. Moreover, the CADTH review examined the diagnostic accuracy of EDTU for ectopic pregnancy, rather than for the identification of IUP, which is how EDTU is taught and used in Canadian EDs.

Our literature search identified 5 studies examining the diagnostic accuracy of EDTU for IUP. Three of these used diagnostic criteria for IUP that has been employed in practice guidelines4,5 and taught in Canadian EDTU courses.23The available evidence indicates that EDTU is highly specific for the identification of IUP in unselected ED patients with first-trimester pelvic pain or bleeding. This is precisely the population in which pelvic EDTU will be used. The excellent specificity of EDTU for IUP means that patients with pelvic pain or bleeding in the first trimester of pregnancy who have a definite IUP on EDTU may be safely discharged from the ED. Patients who do not have a definite IUP on EDTU require formal ultrasonography, including radiological assessment of the uterus and adnexae, specialist consultation for further observation and care, or both.2

It is important that strict diagnostic criteria for IUP be used, namely the identification of an intrauterine gestational sac with yolk sac or fetal pole. These criteria are supported by current practice guidelines4,5 and radiology texts,26and taught in Canadian EDTU courses.23Use of this precise definition of an IUP will limit the potential for false-positive results such as an extrauterine gestation or pseudogestational sac.4,5,23

One possible pitfall of using EDTU to exclude ectopic pregnancy is the potential for missing a heterotopic twin pregnancy, which is the simultaneous gestation of an IUP and an ectopic pregnancy.3The prevalence of heterotopic twin pregnancies is estimated between 1 in 3600 and 1 in 30 000 pregnancies,27,28 but approaches 1 in 100 in patients who have achieved pregnancy using assisted reproductive technologies.29For these higher-risk patients, the identification of an IUP using EDTU is not sufficient to exclude a heterotopic twin pregnancy. These patients should be further evaluated with formal ultrasonography or specialist consultation or both.

We were not able to perform pooled analyses of the specificity or the sensitivity of EDTU for IUP because of statistical heterogeneity between the selected studies. Differences in credentialing standards and operator ability may have contributed to this heterogeneity. Current Canadian standards require 50 proctored scans of each anatomical region (pelvis, aorta, pericardium, abdomen),5,23 whereas the reviewed studies required between 10 and 20. It is reassuring that emergency physicians who were trained with fewer proctored scans were able to achieve a high specificity of EDTU for IUP.

Our systematic review identified evidence that the use of EDTU in the evaluation of first-trimester complaints may lead to improvements in quality of care. EDTU reduces the number of missed ectopic pregnancies discharged from the ED and may decrease the time to treatment of diagnosed ectopic pregnancies.12,14,15

Five studies indicated that EDTU use in evaluating first-trimester pain and bleeding reduces ED LOS, particularly for those patients found to have IUPs.16-20 These studies each have methodological flaws or report insufficient data to demonstrate unequivocally that EDTU use reduces LOS for patients with first-trimester pain or bleeding. The finding of reduced LOS, however, is consistent between all studies, and makes clear intuitive sense, particularly for patients with an IUP. Since first-trimester pain and bleeding are common ED presentations, this represents an important potential decrease in ED patient LOS, leading to reduced wait times overall.

EDTU use in symptomatic first-trimester pregnancy may result in overall cost savings.16,21 Although there are substantial costs associated with implementation of EDTU programs and physician training, potential cost savings over formal ultrasonography for patients with first-trimester pain or bleeding will likely favour widespread adoption of EDTU use.

Limitations

Studies examining the diagnostic accuracy of EDTU were required to meet predefined methodological criteria, and to use the definition of IUP endorsed in practice guidelines4,5 and taught in Canadian EDTU courses.23As a result, 2 methodologically adequate studies that did not describe their criteria for IUP were excluded from our review.

We included studies using a combination of gold standards, rather than a single gold standard such as formal ultrasonography or surgical findings. Since not all patients are treated operatively, and not all patients will undergo formal ultrasonography, it is not realistic to use any single gold standard. The composite gold standard of formal ultrasonography results, surgical findings or clinical follow-up is valid, reflects real life emergency medicine and maximizes the number of patients with usable data.

Our systematic review may be vulnerable to publication bias. As EDTU has been adopted into emergency practice, small quality-assurance studies were likely performed but not published. It is unclear whether non-publication of preliminary studies might have caused us to overestimate or underestimate the diagnostic accuracy of EDTU for IUP. We chose to include data from both published studies and abstracts, in an effort to limit publication bias in our assessment of the diagnostic accuracy and clinical utility of pelvic EDTU.30,31

CONCLUSION

Published evidence indicates that EDTU is highly specific for the detection of IUP in ED patients presenting with first-trimester pelvic pain or bleeding. Patients who have definite IUP identified by EDTU may be safely discharged from the ED with outpatient follow-up. There is consistent evidence that EDTU use lowers ED LOS, reduces delays in the diagnosis and treatment of ectopic pregnancy, and may decrease health care costs. The findings of this systematic review strengthen the argument for the routine use of EDTU in the evaluation of pelvic pain or bleeding in the first trimester of pregnancy.

REFERENCES

1. Rosen CL, Wolfe RW. Ultrasound in emergency medicine. Emerg Med Clin North Am 2004;22:XV-XVI.
2. Murray H, Baakdah H, Bardell T, et al. Diagnosis and treatment of ectopic pregnancy. CMAJ 2005;173:905-12.
3. Moore C, Promes SB. Ultrasound in pregnancy. Emerg Med Clin North Am 2004;22:697-722.
4. American College of Emergency Physicians. ACEP policy statement: emergency ultrasound guidelines. Available: www.acep.org/practres.aspx?id=32182&ekmensel=c580fa7b_90_202_32182_21 (accessed 2008 Aug 08).
5. Canadian Emergency Ultrasound Society. Canadian emergency ultrasound society: recommended standards. Available: www.ceus.ca/002-standards/002-00.standards.htm (accessed 2008 Jul 29).
6. Whiting PF, Weswood MF, Rutjes AW, et al. Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies [abstract]. BMC Med Res Methodol 2006;6:9.
7. Guyatt G, Cook D, Devereaux PJ, et al. Therapy. In: Guyatt G, Rennie D, editors. User’s guides to the medical literature: a manual for evidence-based clinical practice. Chicago (IL): AMA Press; 2002. p. 55-79.
8. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 1927;22:209-12.
9. Deville WL, Buntinx F, Bouter LM, et al. Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Med Res Methodology 2002;2:9.
10. Simon S. Stats: meta-analysis for a diagnostic test. Available: www.childrens-mercy.org/stats/training/hand67.asp (accessed 2009 May 27).
11. Durham B, Lane B, Burbidge L, et al. Pelvic ultrasound performed by emergency physicians in the detection of ectopic pregnancy in complicated first-trimester pregnancies. Ann Emerg Med 1997;29:338-47.
12. Mateer JR, Valley VT, Aiman EJ, et al. Outcome analysis of a protocol including bedside endovaginal sonography in patients at risk for ectopic pregnancy. Ann Emerg Med 1996;27:283-9.
13. Wong TW, Lau CC, Yeung A, et al. Efficacy of transabdominal ultrasound examination in the diagnosis of early pregnancy complications in an emergency department. J Accid Emerg Med 1998;15:155-8.
14. Rodgerson JD, Heegaard WG, Plummer D, et al. Emergency department right upper quadrant ultrasound is associated with a reduced time to diagnosis and treatment of ruptured ectopic pregnancies. Acad Emerg Med 2001;8:331-6.
15. Blaivas M, Bell G. Benefit from emergency physician identified ectopic pregnancy using bedside ultrasound [abstract]. Acad Emerg Med 2000;7:500.
16. Pierce DL, Friedman KD, Killian A, et al. Emergency department ultrasonography (EUS) in symptomatic first-trimester pregnancy [abstract]. Acad Emerg Med 2001;8:546.
17. Shih CH. Effect of emergency physician-performed pelvic sonography on length of stay in the emergency department. Ann Emerg Med 1997;29:348-51.
18. Blaivas M, Sierzenski P, Plecque D, et al. Do emergency physicians save time when locating a live intrauterine pregnancy with bedside ultrasonography? Acad Emerg Med 2000;7:988-93.
19. Burgher SW, Tandy TK, Dawdy MR. Transvaginal ultrasonography by emergency physicians decreases patient time in the emergency department. Acad Emerg Med 1998;5:802-7.
20. Jang TB, Aubin CD. Resident ultrasonography in symptomatic first-trimester pregnancy and emergency department length of stay. Ann Emerg Med 2003;42:S88-9.
21. Durston WE, Carl ML, Guerra W, et al. Ultrasound availability in the evaluation of ectopic pregnancy in the ED: Comparison of quality and cost-effectiveness with different approaches. Am J Emerg Med 2000;18:408-17.
22. Todd WM, Moore CL, O’Brien E, et al. Risk stratification of suspected ectopic pregnancy by transabdominal emergency physician-performed ultrasonography. Ann Emerg Med 2004;44:S82-3.
23. Canadian Association of Emergency Physicians. CAEP road-shows: emergency department targeted ultrasound. Available: www.caep.ca/template.asp?id=8B793B0F46CE4083AB4E775BE6D6A412 (accessed 2009 May 27).
24. Chen S, Hussereau D, Noorani H, et al. Portable ultrasound devices in emergency departments. Technology report issue 63. Ottawa (ON): Canadian Coordinating Office for Health Technology Assessment; 2006.
25. Schlager D, Lazzareschi G, Whitten D, et al. A prospective study of ultrasonography in the ED by emergency physicians. Am J Emerg Med 1994;12:185-9.
26. Mettler F. Jr. Intrauterine and ectopic pregnancy. In: Essentials of radiology. 2nd ed. Philadelphia (PA): Elsevier; 2005. p. 240-3.
27. Stephen R, Richards LES, Carlton BD. Heterotopic gestation: reappraisal of incidence. Am J Obstet Gynecol 1982; 142:928-30.
28. DeVoe RW, Pratt JH. Simultaneous intrauterine and extrauterine pregnancy. Am J Obstet Gynecol 1948;56:1119-26.
29. Tal J, Haddad S, Gordon N, et al. Heterotopic pregnancy after ovulation induction and assisted reproductive technologies: a literature review from 1971 to 1993. Fertil Steril 1996;66:1-12.
30. Montori V, Guyatt G. Publication bias. In: Guyatt G, Rennie D, editors. Users’ guides to the medical literature. Chicago (IL): AMA Press; 2002:529-537.
31. Hopewell S, McDonald S, Clark M, et al. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 2007 Apr 18;(2):MR000010.