Canadian C-Spine Rule study for alert and stable trauma patients: II. Study objectives and methodology

Pedagogical Tools and Methods

CJEM 2002;4(3):185-193

Abstract

Résumé

Introduction | Methods | Discussion | Conclusion | References

Introduction

Clinical decision (or prediction) rules are decision-making tools that incorporate 3 or more variables from the history, physical examination or simple tests. They are derived from original research rather than consensus or opinion and they help clinicians make diagnostic or therapeutic decisions by standardizing the collection and interpretation of clinical data and reducing uncertainty.^1-4 There is growing interest in the methodological standards for their development and validation.^5-9 These are summarized in Box 1.^10-16

The Canadian C-Spine Rule was derived in a prospective cohort study involving 8924 patients.¹⁷ The overall goal of this study was to derive a clinical decision rule that is highly sensitive for detecting acute cervical spine (C-spine) injuries and that will allow physicians to be more efficient in their use of C-spine radiography in alert, stable trauma patients. The component objectives of the C-spine derivation study are listed in Box 2.

After derivation of the C-Spine Rule, we used cross-validation techniques to show that it was 100% sensitive (95% confidence interval [CI], 98%-100%) and 42.5% specific (95% CI, 40%-44%) in identifying 151 clinically important C-spine injuries. In the derivation set, it would have led to radiography in 58.2% of cases, representing a potentially important utilization improvement.

Part I of this 2-article series described the background and rationale for the Canadian CT Head and C-Spine (CCC) Study -- the largest prospective emergency department (ED) study yet conducted in Canada.^17,18 This article describes the study objectives and the methodology used to derive the Canadian C-Spine Rule.

Methods for C-spine rule derivation

Study setting and subjects

The study took place in 10 Canadian community and teaching hospitals with a combined annual ED volume of approximately 400 000 patient visits (Box 3). Consecutive alert, stable adults who presented to one of the participating EDs after sustaining acute blunt trauma to the head or neck were eligible for inclusion. We chose to study alert, stable patients because they constitute the vast majority of trauma patients seen in our EDs and because clinicians are unlikely to apply a decision rule to unconscious, uncooperative or severely injured multiple trauma patients. Box 4 details the eligibility criteria.^19,20

Ethics and consent

Normal patient management was not altered. Patients were not subjected to new therapy, invasive procedures, undue risk or discomfort, or imaging beyond that normally required. Radiography was ordered according to standard practice, based on the treating physician's clinical judgement. The research ethics committees of the participating hospitals therefore determined that informed consent was unnecessary at the time of study enrolment. Patients who were followed up by telephone were asked to provide verbal consent to the follow-up process. Confidentiality was maintained throughout the study, and patient names were removed from all records.

Study procedures and data collection

Clinical variables

At a research formulation workshop before the study, the investigators used clinical experience and prior literature to identify variables associated with acute C-spine injury. Box 5^21-31 and Box 6³² list the predictor variables assessed in this study.

Standardized patient assessment

Certified emergency physicians or supervised emergency medicine residents made all patient assessments. Participating physicians were trained to assess the clinical variables in a uniform manner during a 1-hour demonstration session. When evaluating the neck, they were advised to loosen cervical collars and palpate directly unless the patient was uncooperative. In addition, they assessed active rotation and flexion in cooperative patients with no neurological signs or symptoms. Clinical findings were recorded on standard data collection sheets prior to sending patients for radiography.

Physicians' judgement

Physicians were asked to estimate the probability of C-spine injury and the probability of unstable fracture or dislocation (both to the closest decile), and to state their theoretical comfort with ordering "no C-spine radiography" for each patient, on a 5-point scale from Very comfortable to Very uncomfortable.

Interobserver reliability

On a subset of patients, a second emergency physician blinded to the results of the first assessment independently assessed the clinical variables. These interobserver reliability assessments were performed in all centres on a feasibility basis whenever 2 study physicians were available.

Outcome measures

Primary outcome

Acute C-spine injury was defined as any fracture, dislocation or ligamentous instability (angulation of intervertebral space greater than 11 degrees)²⁰ demonstrated on any radiographic investigation.³³ The injuries were further classified as "clinically unimportant" or "clinically important," with the latter being the primary study outcome. Clinically important C-spine injuries were defined as those that generally require active treatment with internal fixation, halo, brace or rigid collar; whereas clinically unimportant injuries require only symptomatic treatment. Clinically unimportant injuries included: a) isolated avulsion fracture of osteophyte, b) isolated fracture of transverse process not involving facet joint, c) isolated fracture of spinous process not involving lamina, and d) simple compression fracture less than 25% of vertebral body height. These explicit definitions were based on the results of a survey of 129 neurosurgeons, spine surgeons, neuroradiologists and emergency physicians at 8 study sites.³⁴ During the study, all fractures classified as "clinically unimportant" were independently reviewed by the study neurosurgeon and neuroradiologist, who were blinded to the data collection sheets and to the other's interpretation. Disagreements were resolved by consensus.

Outcome assessment

Within the study, standard C-spine radiographs were ordered based on the judgement of the treating physician. Radiography entailed at least 3 views (lateral, anterior-posterior and odontoid), and the lateral view demonstrated all cervical vertebrae to the C7-T1 junction.²⁰ Flexion-extension views were recommended if the patient exhibited disproportionate neck pain or if initial radiography revealed abnormal soft tissue swelling or loss of lordosis. Oblique views, tomograms and computed tomography (CT) were ordered at the discretion of the treating physician. All patients with identified C-spine injury underwent computed tomography. Imaging studies were interpreted by independent staff radiologists who were provided routine clinical information on the requisition but were blinded to the contents of the data collection sheet. To assess the reliability of interpretation, a second radiologist blinded to the primary interpretation reviewed all abnormal radiographs (n = 94) and a 1% random sample of normal radiographs (n = 129). The radiologists showed 100% agreement in these cases.

Proxy outcome assessment

A review of baseline practice showed that 30% of eligible trauma patients at the study hospitals did not undergo C-spine radiography, and the investigators felt that the study protocol could not ethically mandate radiography for all patients. Although patients who are not referred for radiography have less severe injuries and are unlikely to have acute abnormalities on C-spine radiography, all enrolled patients who did not have radiography underwent telephone follow-up to assure no significant injuries were missed. Patients were classified as having no acute C-spine injury if they met all the following explicit criteria at 14 days: a) pain in neck is rated as none or mild, b) restriction of movement of neck is rated as none or mild, c) patient does not require use of a neck collar, and d) neck injury has not prevented return to usual occupational activities (work, housework, or school). Patients who could not fulfill these criteria were recalled for clinical reassessment and C-spine radiography. The validity of these criteria to exclude acute C-spine injury was confirmed by having the telephone follow-up questionnaire applied to a random sample of study patients with and without C-spine injury and who had all undergone radiography.³⁵ The questionnaire has proven to be 100% sensitive for identifying 66 abnormal cases among the 389 radiography patients reached by telephone (i.e., not one of the patients with clinically important C-spine injury would have been classified as "no acute C-spine injury" by the follow-up criteria). Patients who had neither radiography nor adequate follow-up were excluded from the final analysis.

Data analysis

Interobserver agreement

The reliability each potential predictor variable was determined by calculating the kappa coefficient for interobserver agreement beyond chance, along with 95% CIs.^36,37 For variables with 3 or more ordered categories, a weighted kappa was calculated.³⁸ Acceptable interobserver agreement was defined as a kappa coefficient of at least 0.6, which is considered to represent "substantial agreement."

Univariate analysis

Univariate analyses were used to determine the strength of association between each variable and the primary outcome, acute C-spine injury. This process aided selection of the best variables for the multivariate analyses. The appropriate univariate technique was chosen according to the type of data: for nominal data, the chi-squared test with continuity correction; for ordinal variables, the Mann-Whitney U test; and, for continuous variables, the unpaired 2-tailed t-test, using pooled or separate variance estimates as appropriate.

Multivariate analysis

Multivariate analyses combined multiple predictor (independent) variables into a single analysis in order to derive a model to predict the outcome (dependent) variable, acute C-spine injury. Variables found to be both reliable (* > 0.6) and strongly associated with the outcome measure (p < 0.05) were combined using one of 2 different multivariate techniques: recursive partitioning or logistic regression. Second order interaction among predictor variables was evaluated using the Mantel-Haenszel and logistic modeling procedures. Appropriate composite variables were considered for incorporation into the multivariate analyses.

A clinically acceptable decision rule should provide near perfect sensitivity for detecting clinically important C-spine injuries, while maximizing specificity; and it should contain as few variables as possible, for easy use by clinicians. The goal of the multivariate analysis was, therefore, to identify a combination of predictor variables that fulfilled the following criteria: a) 99% or greater sensitivity, b) sufficient specificity to lead to a 20% relative reduction in use of C-spine radiography, and c) no more than 6 component variables. Assuming more than 1 model would meet the minimum acceptable criteria, the best model would be the one with the highest specificity and the fewest component variables.

To determine the best combination of predictor variables for the C-spine decision rule, we performed recursive partitioning using KnowledgeSEEKER Version 2.1 Software (Angoss Software International, Toronto, Ont.).^39-42 In essence, recursive partitioning uses a series of chi-squared analyses to successively split the data set into homogeneous groups that share a common clinical (predictor) characteristic. Our experience suggests that recursive partitioning may be more suitable than logistic regression when the objective is to correctly classify one outcome group at the expense of the other. In this case it was critical to correctly identify all patients with acute C-spine injuries (high sensitivity) at the expense of misclassifying some patients without significant injury (imperfect specificity). To accomplish this, it is possible to add predictor variables to the model until it achieves near perfect sensitivity, recognizing that each added variable reduces specificity. Recursive partitioning techniques allow investigators to deliberately avoid complex models with significant interactions that are difficult for clinicians to interpret or apply, and permit cases with some missing predictor data to be retained in the analysis.

We also performed multiple logistic regression analysis as an alternate multivariate technique to determine the best predictive model. Logistic regression is well suited for the prediction of a binary outcome (e.g., fracture vs. no fracture), and allows one to calculate the probability of the specified outcome by inserting derived values (regression coefficients) into a mathematical formula. A disadvantage of logistic regression is that cases with missing data for any variable are excluded from the analysis. In this case, the regression model was built using forward stepwise variable selection until no variables met the entry (p < 0.05) or removal (p > 0.10) criteria, based on their univariate association with the primary outcome. In order to provide the simplest model for clinicians, clinically meaningful cutpoints were sought for continuous variables. To illustrate, if the age cut-off with the greatest predictive strength was 63.7 years, this would be adjusted upward to 65 years.

The variables chosen by the best models constituted the decision rule for selecting alert, stable trauma patients for C-spine radiography. The decision rule was presented in clear narrative form that does not require computation or use of statistical aids.

Classification performance

The derived decision rule was cross-validated by comparing the classification of each patient to his or her actual status for the primary outcome. This allowed estimation of rule sensitivity and specificity, with 95% CIs. A more robust prospective validation will be carried out on a new set of (validation) patients in phase II.

Physicians' judgement

Data from the 3 questions on physicians' comfort and predictions were tabulated in a simple descriptive format. In addition, information on the predicted probability was used to calculate receiver operating characteristic (ROC) curves and likelihood ratios for determining acute C-spine injury as well as unstable fracture/dislocation, respectively. The accuracy of the physicians' predictions were compared to that of the derived decision rule by ROC curve analysis.

Sample size

Patients required

Since no hypothesis was being tested, sample size was based on the precision (95% confidence range) around the sensitivity estimate of the derived decision rule, and on the precision of the estimates of interobserver variability and logistic regression coefficients. The sample size had to accommodate the large number of potential clinical variables (22), the large number of physicians (more than 60), and the low prevalence of acute C-spine injury (estimated to be 1.8% of eligible patients). Given these assumptions, we estimated that a sample of 8 000 alert, stable trauma patients should yield approximately 120 cases of acute C-spine injury. With this number of injuries, and a 100% sensitive decision rule, the 95% confidence range would be 97%-100%.³⁸ We estimated that it would take 26 months to enroll 8 000 patients with 120 C-spine injuries at the study hospitals.

Interobserver agreement

We aimed to have at least 200 patients assessed by 2 study physicians to determine interobserver agreement for the clinical variables. The number 200 was dictated primarily by considerations of feasibility because obtaining assessments by 2 physicians in a busy ED is difficult. With 200 patients, the 95% CIs for a kappa of 0.6 are 0.73 and 0.47.⁴³

Discussion

Potential limitations of methodology

Accurate real-time collection of 20 clinical predictor variables from thousands of trauma patients, using the services of hundreds of "volunteer" physicians, is a difficult challenge -- a balance between strict methodological rigour and feasibility. Consequently, there are several potential limitations that warrant discussion.

In order to enroll as many fracture cases as possible and to increase the efficiency of the study, we enrolled patients transferred from other hospitals (4.1% of all study cases). Physicians were asked to examine these patients prior to reviewing the radiographs, but we cannot verify that this was always done. We know, however, that many transfer cases proved not to have fractures.

At 1 of the 10 study sites, several physicians were uncomfortable with the concept of evaluating active range of motion and rarely did so. Nevertheless, the vast majority of physicians did, indeed, evaluate range of motion unless there were clear high-risk factors such as paresthesias, advanced age or dangerous mechanism (e.g., diving, fall from a height, MVC rollover).

There may be concern about our use of "clinically important C-spine injury" as the primary outcome. This outcome was, however, accepted by our sample of 129 Canadian academic neurosurgeons, spine surgeons and emergency physicians, and we believe that it represents a pragmatic and safe approach to patient care. The priority for diagnostic imaging in trauma patients should be to identify C-spine injuries that require treatment and follow-up, not to identify "clinically unimportant injuries" that, by definition, require neither stabilization nor specialized follow-up, and which are unlikely to be associated with long term sequelae. Furthermore, the Canadian C-Spine Rule has proven very sensitive for clinically unimportant injuries, missing only 1 small avulsion fracture that required only a cervical collar.

Another potential limitation is that not all study patients underwent C-spine radiography. The Canadian clinicians in our study often withhold diagnostic imaging for trauma patients whom they consider to be at low risk for injury. Consequently, we could not ethically insist upon universal radiography for all patients. Patients were only classified as having no clinically important injury if they satisfied all criteria on the structured 14-day telephone proxy outcome tool administered by a registered nurse. Patients who could not fulfill all criteria were recalled for radiography, and patients who could not be reached were excluded from the final analysis. The proxy outcome tool has been validated and shown to be accurate in identifying patients with clinically important injuries. In addition, we acknowledge that not all eligible patients were enrolled in the study. However, this is not unusual for a clinical study, and we are confident that there was no selection bias in that the characteristics of patients not enrolled were very similar to those of the patients who were enrolled.

The NEXUS criteria

The NEXUS (National Emergency X-Radiography Utilization Study) criteria for cervical spine radiography have recently received prominent attention after the publication of a huge validation study incorporating more than 34 000 patients.^44-46 These guidelines state that no C-spine radiography is required if patients satisfy all of 5 low-risk criteria: absence of midline tenderness, normal level of alertness, no evidence of intoxication, no neurological findings, and no painful distracting injuries. We have concerns about the sensitivity, specificity and reliability of these criteria. The NEXUS authors calculated a specificity of 12%, which is very low and could actually increase radiography utilization in most countries outside of the US. In addition, Canadian clinicians have found some of the NEXUS criteria to be poorly reproducible -- particularly "presence of intoxication" and "distracting painful injuries." We conducted a retrospective validation of the NEXUS criteria based upon our database of 8924 patients and found that these criteria missed 10 of 148 clinically important injuries, yielding a sensitivity of only 93%.⁴⁷ Our recent prospective validation found similar problems with the sensitivity of the NEXUS criteria.⁴⁸ We believe that the NEXUS criteria require further prospective evaluation, to assure sensitivity, specificity and interobserver agreement in multiple sites before they can be accepted for widespread clinical use.

Current research: prospective validation (phase II)

A multicentre study to validate the Canadian C-Spine Rule is currently being completed in multiple Canadian sites with another 8000 alert and stable trauma patients (1999-2002).⁴⁹ The goal of phase II is to prospectively assess the accuracy, reliability and clinical sensibility of the Canadian C-Spine Rule in a new set of alert, stable trauma patients and to determine the potential health care savings. Specific objectives are to determine: a) the accuracy or classification performance of the decision rule when applied prospectively, b) reliability and interobserver agreement for the rule, c) the clinical sensibility (i.e., physicians' accuracy, comfort and ease of use applying the rule), d) the potential of the rule to reduce the use of C-spine radiography, e) the potential for refinement of the rule to achieve better specificity, f) the potential savings associated with widespread implementation of the rule, in a preliminary economic evaluation, and g) the accuracy, reliability and clinical sensibility of the NEXUS criteria.

Future research: implementation (phase III)

The authors also intend to conduct an multicentre Canadian implementation study (phase III) to evaluate the effectiveness and safety of an active strategy to implement the Canadian C-Spine Rule into physician practice. Specific objectives will be to: a) determine clinical impact on C-spine radiography rates, missed fractures, serious adverse outcomes, length of stay in ED and patient satisfaction; b) determine sustainability of the impact over time; c) further evaluate performance of the Canadian C-Spine Rule, with regards to accuracy, physician interpretation and physician comfort with use; and d) conduct an economic evaluation to determine the potential for cost savings with widespread implementation. We are planning a matched-pair cluster design study that will compare outcomes during 3 consecutive 12-month "before," "after" and "decay" periods at 6 pairs of "intervention" and "control" sites. These 12 hospital ED sites will be stratified as "teaching" or "community" hospitals, matched according to baseline C-spine radiography ordering rates, and then allocated within each pair to either intervention or control groups. During the "after" period at the intervention sites, simple and inexpensive strategies will be employed to actively implement the Canadian C-Spine Rule.

Conclusion

Blunt trauma is a common condition associated with excessive and variable radiography use and with prolonged, often unnecessary, patient immobilization in the ED. The Canadian C-Spine Rule has the potential to improve radiography utilization, reduce health care costs and enhance the efficiency of patient flow without jeopardizing patient care. This article describes the methods used to derive the Canadian C-Spine Rule and provides a valuable reference for investigators planning to develop future clinical prediction rules.

References

1. Pozen MW, D'agostino RB, Mitchell JB, Rosenfeld M, Guglielmino JT, Schwartz ML, et al. The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Ann Intern Med 1980;92:238-42.
2. Goldman L, Cook EF, Brand DA. A computer protocol to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988;318:797-803.
3. Heckerling PS, Tape TG, Wigston RS. Clinical prediction rule for pulmonary infiltrates. Ann Intern Med 1990;113:664-70.
4. Bates DW, Cook EF, Goldman L, Lee TH. Predicting bacteremia in hospitalized patients: a prospectively validated model. Ann Intern Med 1990;113:495-500.
5. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and methodological standards. N Engl J Med 1985;313:793-9.
6. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437-47.
7. Feinstein AR. Clinimetrics. New Haven (CT): Yale University Press; 1987.
8. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997;277:488-94.
9. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users' guides to the medical literature: XXII: How to use articles about clinical decision rules. JAMA 2000;284(1):79-84.
10. Fischl MA, Pitchenik A, Gardner LB. An index predicting relapse and need for hospitalization in patients with acute bronchial asthma. N Engl J Med 1981;305:783-99.
11. Rose CC, Murphy JG, Schwartz JS. Performance of an index predicting the response of patients with acute bronchial asthma to intensive emergency department treatment. N Engl J Med 1984;310:573-7.
12. Centor RM, Yarbrough B, Wood JP. Inability to predict relapse in acute asthma. N Engl J Med 1984;310:577-80.
13. Charlson ME, Ales KL, Simon R, MacKenzie CR. Why predictive indexes perform less well in validation studies. Arch Intern Med 1987;147:2155-61.
14. Poses RM, Cebul RD, Collins M, Fager SS. The importance of disease prevalence in transporting clinical prediction rules: the case of streptococcal pharyngitis. Ann Intern Med 1986;105:586-91.
15. Wigton RS, Connor JL, Centor RM. Transportability of a decision rule for the diagnosis of streptococcal pharyngitis. Arch Intern Med 1986;146:81-3.
16. Lee TH. Evaluating decision aids: the next painful step. J Gen Intern Med 1990;5:528-9.
17. Stiell IG, Wells GA, Vandemheen KL, Clement CM, Lesiuk H, De Maio VJ, et al. The Canadian C-Spine Rule for radiography in alert and stable trauma patients. JAMA 2001;286(15):1841-8.
18. Stiell IG, Wells GA, McKnight RD, Brison R, Lesiuk H, Clement CM, et al. Canadian C-Spine Rule study for alert and stable trauma patients: I. Background and rationale. CJEM 2002;4:84-90.
19. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet 1974;2:81-4.
20. Advanced Trauma Life Support instructor manual. 5th ed. Chicago: American College of Surgeons; 1993.
21. Jacobs LM, Schwartz R. Prospective analysis of acute cervical spine injury: a methodology to predict injury. Ann Emerg Med 1986;15:44-9.
22. Roberge RJ, Wears RC, Kelly M, Evans TC, Kenny MA, Daffner RD, et al. Selective application of cervical spine radiography in alert victims of blunt trauma: a prospective study. J Trauma 1988;28:784-8.
23. Cadoux CG, White JD, Hedberg MC. High-yield roentgenographic criteria for cervical spine injuries. Ann Emerg Med 1987;16:738-42.
24. Brison RJ, Hartling L, Pickett W. Need for cervical spine radiographs in persons injured in rear-end MVCs [abstract]. J Emerg Med 1998;16:225.
25. Bachulis BL, Long WB, Hynes GD, Johnson MC. Clinical indications for cervical spine radiographs in the traumatized patient. Am J Surg 1987; 153:473-7.
26. Neifeld GL, Keene JG, Hevesy G, Leikin J, Proust A, Thisted RA. Cervical injury in head trauma. J Emerg Med 1988;6:203-7.
27. Kreipke DL, Gillespie KR, McCarthy MC, Mail JT, Lappas JC, Broadie TA. Reliability of indications for cervical spine films in trauma patients. J Trauma 1989;29:1438-9.
28. Bayless P, Ray VG. Incidence of cervical spine injuries in association with blunt head trauma. Am J Emerg Med 1989;7:139-42.
29. McNamara RM, Heine E, Esposito B. Cervical spine injury and radiography in alert, high-risk patients. J Emerg Med 1990;8:177-82.
30. Saddison D, Vanek VW, Racanelli JL. Clinical indications for cervical spine radiographs in alert trauma patients. Am Surg 1991;57:366-9.
31. McNamara RM, O'Brien MC, Davidheiser S. Post-traumatic neck pain: a prospective and follow-up study. Ann Emerg Med 1988;17:906-11.
32. Shackford SR, Wald SL, Ross SE, Cogbill TH, Hoyt DB, Morris JA, et al. The clinical utility of computed tomographic scanning and neurologic examination in the management of patients with minor head injuries. J Trauma 1992;33:385-94.
33. White AAI, Panjabi M. Clinical biomechanics of the spine. 11th ed. Philadelphia: Lippincott; 1990.
34. Stiell IG, Lesiuk H, Vandemheen K, Worthington JR, Verbeek RP, Reardon M, et al. Obtaining consensus for a definition of "clinically important cervical spine injury" in the CCC Study [abstract]. Acad Emerg Med 1999;6:435.
35. Stiell IG, Vandemheen K, Brison R, Cass DE, Dreyer J, Eisenhauer MA, et al. Validity evaluation of the cervical spine injury proxy outcome assessment tool in the CCC Study [abstract]. Acad Emerg Med 1999;6:434.
36. Kramer MS, Feinstein AR. Clinical biostatistics: LIV. The biostatistics of concordance. Clin Pharmacol Ther 1982;29:111-23.
37. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
38. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York: John Wiley & Sons; 1981.
39. Ciampi A, Hogg SA, McKinney S, Thiffault J. RECPAM: a computer program for recursive partition and amalgamation for censored survival data and other situations frequently occurring in biostatistics. I. Methods and program features. Comput Meth Prog Biomed 1988;26:239-56.
40. Ciampi A, Chang CH, Hogg S, McKinney S. Recursive partition: a versatile method for exploratory data analysis in biostatistics. In: MacNeill IB, Umphrey GJ, editors. Time series and econometric modelling. Vol 5, Biostatistics. Boston: D. Reidel Publishing Company; 1987. p. 23-50.
41. Friedman JH. A recursive partitioning decision rule for nonparametric classification. IEEE Trans Comput 1977;16:404-8.
42. Ciampi A, Thiffault J, Nakache JP, Asselain B. Stratification by stepwise regression, correspondence analysis and recursive partition: a comparison of three methods of analysis for survival data with covariates. Comput Stat Data Anal 1986;4:185-204.
43. Flack VF, Afifi AA, Lachenbruch PA, Schouten HJA. Sample size determinations for the two rater kappa statistic. Psychometrika 1988;53:321-5.
44. Hoffman JR, Schriger DL, Mower W, Luo JS, Zucker M. Low-risk criteria for cervical-spine radiography in blunt trauma: a prospective study. Ann Emerg Med 1992;21:1454-60.
45. Hoffman JR, Mower W, Wolfson AB, Todd K, Zucker M. Validity of a set of clinical criteria to rule out injury to the cervical spine in patients with blunt trauma. N Engl J Med 2000; 343:94-9.
46. Hoffman JR, Wolfson AB, Todd K, Mower WR. Selective cervical spine radiography in blunt trauma: methodology of the National Emergency X-Radiography Utilization Study (NEXUS). Ann Emerg Med 1998; 32(4):461-9.
47. Stiell IG, McKnight RD, Wells GA, Lesiuk H, Vandemheen K, Clement C, et al. Application of the NEXUS low-risk criteria for cervical spine radiography in Canadian emergency departments [abstract]. Acad Emerg Med 2000;7:566.
48. Stiell IG, Clement C, Wells GA, Morrison L, Greenberg GH, Dreyer J, et al. Prospective validation of the NEXUS low-risk criteria for cervical spine radiography [abstract]. Acad Emerg Med 2002: In press.
49. Stiell IG, Clement C, Wells GA, McKnight RD, Brison R, Worthington J, et al. Multicenter prospective validation of the Canadian C-Spine Rule [abstract]. Acad Emerg Med 2002. In press.