Can Telehealth Ontario respiratory call volume be used as a proxy for emergency department respiratory visit surveillance by public health?

EM Advances

CJEM 2008;10(1):18-24

Abstract

Résumé

Introduction

The prompt detection of disease outbreaks has become a major area of interest in public health. It is felt that early detection has the potential to reduce morbidity and mortality.1 Real-time syndromic surveillance uses existing non-traditional data for prompt analysis and feedback to those responsible for investigation and follow-up of potential outbreaks.2 Decreasing false-alarm rates while reducing time to detection could improve sensitivity and specificity in a cost-effective manner. Improving the quality of existing signals and adding new signals are strategies that may improve the timeliness of detection.1 Recent studies have suggested that integrating multiple data sources can significantly improve detection accuracy of syndromic surveillance systems, but more work is needed to explore the most effective means of said integration and what types of data streams give the greatest benefit.3,4 Current data streams that have the potential for integration include: over-the-counter drug sales, emergency department (ED) visits, 911 calls, ambulatory dispatch, school and work absenteeism records, insurance and billing claims, and telephone medical helplines.5 In Ontario, the earliest points of detection of increased access to health care would include the purchase of upper respiratory illness-related products, the Telehealth Ontario hotline, and visits to EDs and primary care physicians.6 These access points can be complementary or dependent on their accessibility to the population.

To our knowledge, no studies have evaluated the use of Telehealth Ontario data as a surveillance tool. However, other countries have both prospectively and retrospectively evaluated the use of their teletriage systems as early warning systems. Most notably, the United Kingdom has been a forerunner in the field of teletriage syndromic surveillance, having used the National Health Service Direct system for this purpose since 1999.7-13 In the absence of available evidence on Telehealth Ontario's usefulness as a syndromic surveillance system, key decision makers may entertain doubts about telephone health hotline effectiveness, which hinder further investments and integration of such data into the health mainstream.14 It should be noted that Telehealth Ontario is a component of the Ontario Ministry of Health and Long-Term Care's (MOHLTC's) influenza pandemic plan to aid in surveillance during the pandemic period of influenza outbreaks. Its main role is to triage callers and provide health information.15 This further indicates the importance of assessing the value of Telehealth Ontario data. Many studies have used data on ED chief complaints, International Classification of Disease (ICD)-coded respiratory illnesses, and/or laboratory-confirmed influenza and respiratory viruses to retrospectively show the validity of syndromic surveillance systems16-19 as well as show the impact of such illnesses on the health care system.20-25

The objective of this study was to examine the temporal relation between Ontario's ED visits and Telehealth calls for respiratory illnesses. We hypothesized that calls to the Telehealth Ontario hotline would be a proxy measure for respiratory visit data from EDs in Ontario, which could warrant the inclusion of Telehealth Ontario data into a real-time syndromic surveillance system.

Methods

Study design

We conducted a retrospective study on respiratory illness data for a 22-month period between June 1, 2004, and March 31, 2006. Anonymized data were obtained from 2 sources: the MOHLTC's Telehealth Ontario program and the Canadian Institute for Health Information's (CIHI's) National Ambulatory Care Reporting System (NACRS). Our data set was limited to this time frame as NACRS data was only available until March 2006. Inclusion of data before June 2004 risked introducing uncontrolled variables owing to the severe acute respiratory syndrome (SARS) outbreak.

Setting

Telehealth Ontario is a toll-free helpline provided by the MOHLTC and is available to all residents of the province. Users are encouraged to call with any general health questions; confidential advice is given regarding any health concerns. The system is available 24 hours per day, with advice coming from trained and experienced registered nurses. Telehealth is available in English and French, with translational support available in 110 languages.26 Aside from English and French, Mandarin, Cantonese, Farsi, Italian and Portuguese are the most requested languages.26 Each call lasts an average of 10 minutes, with nurses directing patients to the most appropriate form of care. This is achieved using a medically approved care advice database that is evidence based and expert driven.27

NACRS was developed in 1997 by the CIHI to capture clinical, administrative and demographic information from all hospital- and community-based ambulatory care.28 Ontario is the only province in Canada that is mandated by the provincial government to submit all abstracts on patient visits in a fiscal year. As of July 31, 2006, there were 186 institutions in Ontario submitting to NACRS.29

Study population

This research was centred on the province of Ontario as its population base, and all citizens were included in the catchment area. The study was part of a broader investigative project that was approved by the Queen's University Research Ethics Board.

Data collection and outcome measure

Telehealth services are provided by a private contractor who was hired by the MOHLTC, which has collected data since December 2001. After studying various syndromic surveillance systems, including Real-time Outbreak and Disease Surveillance (RODS), the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE) and the National Health Service Direct, along with their respective syndrome classifications, we categorized the Telehealth guidelines into 1 of 32 syndrome names (e.g., respiratory upper, respiratory lower, asthma, trauma, gastroenteritis and neurologic-non infectious, among others). These 32 syndromes were based on the formative work of the aforementioned systems, and for the purposes of this study, only calls coded as an upper or lower respiratory syndrome were analyzed (Box 1).

The institution codes and abstracts the data from NACRS at the end of a patient visit. CIHI conducts edit checks on all data submitted to identify duplicate records, missing data or inconsistencies in data transmission. The submitting facility is asked to correct any detected errors.28 This database has little missing information. The reliability of the coding of data collected by CIHI ranges from 74% to 96%, with influenza and pneumonia reliably coded at 81%.23 Since the 2002/03 fiscal year, NACRS has collected diagnosis- and intervention-related information based on the ICD-10-CA (International Classification of Diseases and Related Health Problems, 10th rev, Canadian Enhancement) coding system.29 For the purposes of this study, each health facility had an experienced health coder assign an ICD code. Only those that dealt with a communicable respiratory illness were included in our data set (Table 1).

Statistical analyses

The Telehealth Ontario and NACRS data sources were compared by fitting time-series models and estimating a cross correlogram at different lags (semi-monthly). Both data sets were transformed and detrended by differencing. Autoregressive moving average models were fitted to the differenced series to ensure the residuals were white noise. The autocorrelation and partial autocorrelation functions of the models were examined to determine autoregressive and moving average parts of the models. Residuals were checked for normality against the fitted values and checked for white noise by the Portmanteau test. Spearman rank tests were performed and then cross correlations were estimated for residuals (to account for seasonality and trends) at different lags with the limit of statistically significant correlation being 2/√(N-1). This method of analysis has been previously demonstrated in National Health Service Direct research.30 All statistical procedures were generated with SAS software, version 9.1 (SAS Institute, Cary, North Carolina).

Results

Of the 1.8 million calls to the Telehealth Ontario hotline during the study time period, 216 105 calls (~12%) were for an upper or lower respiratory complaint. In contrast, 819 832 ICD-10-CA coded complaints of the approximately 17.5 million abstracts submitted to NACRS (~5%) corresponded to infectious respiratory illnesses (Table 1). Age breakdowns from each data set are shown in Table 2, with Telehealth Ontario having more calls regarding 0-4-year-olds (49%), while ED visits consisted of mainly those aged 18-64 years (44%).

There were 2 corresponding peaks for respiratory calls and respiratory visits in the data sets, both of which occurred during week 7 of 2004 and 2005. The 2004 winter peak was prolonged and produced higher proportions of calls to Telehealth Ontario and more visits to EDs for respiratory illness, compared with the 2005 season (Fig. 1).

The Spearman rank correlation coefficient was calculated to be 0.97. The Portmanteau test for cross correlations of residuals found that 3 sets of residuals were white noise (p = 0.02), which is a discrete time stochastic process whose terms are independent and identically distributed. Two statistically significant correlations were found between the Telehealth Ontario and the NACRS data series. One was highly significant at lag (semi-monthly) 0, indicating that increases in both series can occur simultaneously; whereas, another was weakly correlated at lag 1, indicating that increases in Telehealth Ontario calls can precede increases in ED visits by as much as 15 days.

Discussion

This study is the first to examine data from Ontario's Telehealth program in an effort to provide evidence of Telehealth's possible effectiveness as a surveillance tool. Our results suggest that the integration of Telehealth Ontario data into a real-time syndromic surveillance system may be a complementary tool for the detection of respiratory illnesses on a provincial basis. This is in line with other studies, which have proposed that multiple non-traditional data sources may be of significant use to augment current syndromic surveillance systems.4,16,31,32

Our comparison of Telehealth Ontario with the NACRS data on respiratory illnesses shows that both curves are highly correlated. The time-series cross-correlogram showed that the Telehealth Ontario data can document increases in respiratory calls simultaneously with NACRS, and it indicated that if threshold levels are set for the start of outbreaks, they may provide up to 15 days advance warnings of ED visits. This also gives a positive indication that Telehealth Ontario can be used as a proxy measure for discharge diagnoses from EDs for respiratory illness.

The more intense and prolonged 2004/05 winter peak was likely due to the fact that 2004/05 was predominantly an Influenza A year (81% of all cases in Canada were Influenza A,33 compared with 61.1% in 2005/0634), with a larger number of cases relative to other years (10 006 confirmed cases in 2004/05 v. 6590 confirmed cases in 2005/06). In the 2004/05 influenza season, the Public Health Agency of Canada reported a peak during week 9 of 2005 for Ontario, 2 weeks later than the Telehealth Ontario and NACRS data; in 2005/06, the Public Health Agency of Canada reported a peak for Ontario was during week 8 of 2006, 1 week after the Telehealth Ontario and NACRS data.33,34 These lag periods reiterate Telehealth Ontario's potential as a viable early warning system for respiratory illness. Evaluation of aberration detection methods on both curves is the next step to ascertain if sufficient specificity values are being obtained from the warnings to show that action by surveillance teams and doctors is warranted.

The strength of the NACRS data is based on physicians' diagnoses, which are then converted to ICD codes. Studies on the accuracy of ICD-9 (International Classification of Diseases, 9th rev) codes for respiratory illness have shown excellent specificity and moderate sensitivity, supporting their use in public health surveillance.35 While our study used the newer classifications (ICD-10-CA), we revised our inclusion of specific codes as a result of recent work that found the most sensitive ICD-9 codes for influenza-like illness surveillance.36 Furthermore, evidence suggests that this is a valid assumption as syndromic surveillance systems are stable (i.e., resilient toward change, such as a shift from coding in ICD-9 to ICD-10).37 Although the NACRS is a concise, clinical-based data set, its flaw is its timeliness. The MOHLTC mandates all Ontario hospitals to submit data, but its only requirement is that abstracts be submitted before the fiscal year-end deadline. This reporting lag makes using these data for a real-time surveillance system impractical, whereas the Telehealth Ontario data are recorded in real-time as an electronic form, which could theoretically be integrated into an existing provincial surveillance system.

Telehealth Ontario receives a large volume of calls for 0-4-year-olds and a marked deficiency in those over the age of 65, compared with ED visits. This inadequate representation of the elderly population may be explained by their hesitation to use the Telehealth Ontario program or by their lack of awareness of it, or their preference to see a family physician or visit the ED. The overrepresentation of calls associated with 0-4-year-olds may in fact be essential to monitor, as children are the primary propagators of respiratory-type illnesses such as influenza; they also have longer durations of viral shedding and generally higher titres of recovered viruses.38-40 By identifying respiratory illness in children early, we may be able to identify circulating viruses and implement appropriate public health measures (such as public messaging, calls for vaccination and school closures) to further mitigate the spread of disease to other children, adults and the elderly. Further, if Telehealth Ontario is able to identify those patients who do not require emergency medical treatment and instead divert them to self-care or their family physician, this may help reduce the burden on EDs. However, if this is to be an effective method, further work will have to be done to ensure that patients follow the action suggested by the Telehealth Ontario nurse. Another positive aspect of the Telehealth Ontario program is its accessibility to those living in remote geographic areas where hospitals or primary care may be difficult to access.

Limitations

Our study has several limitations. First, administrative data has its inherent weaknesses as there is always the possibility for coding and entry errors as well as misdiagnoses. Although experienced physicians, nurses and health coders provide this data, human error can be a factor. Second, the retrospective aspect can also be of consequence as there is no possibility to follow up on missing or incomplete data. Although NACRS has very good filters for this, Telehealth Ontario does not. The age and sex categories of the Telehealth Ontario data set showed minimal missing values (data not shown). These missing values, along with possible minor misclassifications, are not expected to influence results owing to the large amount of data in both series. Third, selection bias may also have been introduced by people who do not seek any form of medical attention. This issue is also thought to be of small consequence to our study because of the toll-free nature of Telehealth Ontario and the free access to health care for all Canadians. Finally, we only had 22 months of data to examine and therefore conclusions based on this can only reflect the seasonality and circulation of respiratory viruses for this time period.

Conclusion

The analysis of routinely collected Telehealth Ontario data provides evidence that it can be a proxy measure for ED-visit data for respiratory illnesses on a provincial basis. This is potentially the only source of province-wide real-time surveillance data in Ontario. The only other method currently used is influenza-like illness surveillance, which is reported by sentinel physicians (1 sentinel physician per 165 000 people).15 The integration of Telehealth Ontario data into existing real-time syndromic surveillance systems may improve the ability of such systems to detect outbreaks of respiratory illness and assess the impact on EDs more quickly than current methods. This would allow public health and emergency management officials a novel means of timely surveillance, which could enable prompt preparation and action toward influenza-like illness outbreaks. Further research on Telehealth Ontario and other non-traditional data sources is needed in an ongoing effort to improve disease detection and to provide evidence for their effectiveness as tools for surveillance, especially during pandemic influenza periods.

References

1. Wagner MM, Tsui F-C, Espino JU, et al. The emerging science of very early detection of disease outbreaks. J Public Health Manag Pract 2001;7:51-9.
2. Henning KJ. Overview of syndromic surveillance — what is syndromic surveillance. MMWR Morb Mortal Wkly Rep 2004; 53(Suppl):5-11.
3. Fienberg SE, Shmueli G. Statistical issues and challenges associated with rapid detection of bio-terrorist attacks. Stat Med 2005; 24:513-29.
4. Wang L, Ramoni MF, Mandl KD, et al. Factors affecting automated syndromic surveillance. Artif Intell Med 2005;34:269-78.
5. Rolland E, Moore K, Robinson VA, et al. Using Ontario's "Telehealth" health telephone helpline as an early-warning system: a study protocol. BMC Health Serv Res 2006;6:10.
6. Moore K. Real-time syndrome surveillance in Ontario, Canada: the potential use of emergency departments and Telehealth. Eur J Emerg Med 2004;11:3-11.
7. Harcourt SE, Smith GE, Hollyoak V, et al. Can calls to NHS Direct be used for syndromic surveillance? Commun Dis Public Health 2001;4:178-82.
8. Chapman RS, Smith GE, Warburton F, et al. Impact of NHS Direct on general practice consultations during the winter of 1999-2000: analysis of routinely collected data. BMJ 2002;325:1397-8.
9. Cooper DL, Smith G, Baker M, et al. National symptom surveillance using calls to a telephone health advice service — United Kingdom, December 2001-February 2003. MMWR Morb Mortal Wkly Rep 2004;53(Suppl):179-83.
10. Cooper DL, Smith GE, Hollyoak VA, et al. Use of NHS direct calls for surveillance of influenza — a second year's experience. Commun Dis Public Health 2002;5:127-31.
11. Cooper DL, Smith GE, O'Brien SJ, et al. What can analysis of calls to NHS Direct Tell us about the epidemiology of gastrointestinal infections in the community? J Infect 2003;46:101-5.
12. Payne F, Jessopp L. NHS Direct: review of activity data for the first year of operation at one site. J Public Health Med 2001; 23: 155-8.
13. Leonardi GS, Hajat S, Kovats RS, et al. Syndromic surveillance use to detect the early effects of heat-waves: an analysis of NHS Direct data in England. Soz Praventivmed 2005;51:194-201.
14. Miller E. A. Solving the disjuncture between research and practice: Telehealth trends in the 21st century. Health Policy (New York) 2007;82:133-41.
15. Ontario Ministry of Health and Long-Term Care. Ontario health plan for an influenza pandemic 2006. Available: www.health.gov.on.ca/english/providers/program/emu/pan_flu/pan_flu_plan.html#whole (accessed 2007 Mar 2).
16. Reis BY, Mandl KD. Time series modeling for syndromic surveillance. BMC Med Inform Decis Mak 2003;3:2.
17. Beitel AJ, Olson KL, Reis BY, et al. Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population. Pediatr Emerg Care 2004; 20:355-60.
18. Bourgeois FT, Olson KL, Brownstein JS, et al. Validation of syndromic surveillance of respiratory infections. Ann Emerg Med 2006;47:265-71.
19. Townes JM, Kohn MA, Southwick KL, et al. Investigation of an electronic emergency department information system as a data source for respiratory syndrome surveillance. J Public Health Manag Pract 2004;10:299-307.
20. Schull MJ, Mamdani MM, Fang J. Community influenza outbreaks and emergency department ambulance diversion. Ann Emerg Med 2004;44:61-7.
21. Schull MJ, Mamdani MM, Fang J. Influenza and emergency department utilization by elders. Acad Emerg Med 2005;12:338-44.
22. Upshur REG, Moineddin R, Crighton EJ, et al. Interactions of viral pathogens on hospital admissions for pneumonia, croup and chronic obstructive pulmonary diseases: results of a multivariate time-series analysis. Epidemiol Infect 2006;134:1174-8.
23. Crighton EJ, Moineddin R, Mamdani MM, et al. Influenza and pneumonia hospitalizations in Ontario: a time-series analysis. Epidemiol Infect 2004;132:1167-74.
24. Upshur REG, Knight K, Goel V. Time-series analysis of the relation between influenza virus and hospital admissions of the elderly in Ontario, Canada, for pneumonia, chronic lung disease, and congestive heart failure. Am J Epidemiol 1999;149:85-92.
25. Menec VH, Black C, MacWilliam L, et al. The impact of influenza-associated respiratory illnesses on hospitalizations, physician visits, emergency room visits, and mortality. Can J Public Health 2003;94:59-63.
26. Ontario Ministry of Health and Long-Term Care. Public information — Telehealth Ontario. Available: www.health.gov.on.ca/english/public/program/telehealth/tele_faq.html (accessed 2007 Mar 2).
27. Clinidata. Symptom based tele-triage and health information services. Available: www.clinidata.com (accessed 2007 Nov 21).
28. Thakore J, Roach J, Flaherty DH. Clinical administratvie databases — privacy impact assessment. Ottawa (ON): Canadian Institute for Health Information; 2005.
29. Executive summary: database background and general data limitations documentation. National Ambulatory Care Reporting System (NACRS) FY 2005-2006. Ottawa (ON): Canadian Institute for Health Information; 2006.
30. Doroshenko A, Cooper D, Smith G, et al. Evaluation of syndromic surveillance based on national health service direct derived data — England and Wales. MMWR Morb Mortal Wkly Rep 2005;54:117-22.
31. Rowe BH, Bond K, Ospina MB, et al. Data collection on patients in emergency departments in Canada. CJEM 2006;8:417-24.
32. Espino JU, Hogan WR, Wagner MM. Telephone triage: a timely data source for surveillance of influenza-like diseases. Paper presented at: AMIA 2003 Symposium Proceedings; November 8, 2003;Washington, DC.
33. Public Health Agency of Canada. Influenza in Canada: 2004-2005 season. Canada Communicable Disease Report 2006; 32(6):57-74.
34. Public Health Agency of Canada. Influenza in Canada: 2005-2006 season. Canada Communicable Disease Report 2007; 33(3):21-44.
35. Espino JU, Wagner MM. Accuracy of ICD-9-coded chief complaints and diagnoses for the detection of acute respiratory illness. Paper presented at: AMIA 2001 Annual Symposium; November 3, 2001; Washington, DC.
36. Marsden-Haug N, Foster VB, Gould PL, et al. Code-based syndromic surveillance for influenzalike illness by International Classification of Diseases, ninth revision. Emerg Infect Dis 2007; 13: 207-16.
37. Buehler JW, Hopkins RS, Overhage JM, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks. MMWR Morb Mortal Wkly Rep 2004; 53 (RR05):1-11.
38. Heikkinen T. Influenza in children. Acta Paediatr 2006;95:778-84.
39. Centers for Disease Control and Prevention. Prevention and control of influenza — recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep 2007;55(RR-10):1-42.
40. Nicholson KG. Human influenza. In: Nicholson KG, Webster RG, Hay AJ, eds. Textbook of influenza. Oxford (UK): Blackwell Science; 1998.