In this study we illustrate that a search algorithm using a single HF ICD-CM code can have acceptable sensitivity, specificity, PPV, NPV and accuracy in identifying children with HF from within electronic medical records, but not surprisingly, algorithm performance dependends on the data source. The specificity of this search algorithm is improved by including HF medication as an additional criterion. These findings are generally similar to the contemporary adult HF literature,[3] in which search strategies using more than 1 HF ICD coded visit and/or medications further improve the PPV of the algorithms at the expense of the sensitivity. Of note, in our cohort, an inpatient admission was also associated with a higher likelihood of having an adjudicated HF diagnosis.
Despite the initially good test characteristics of the algorithms in the Clinic Cohort (a population with a high prevalence of HF), we captured many false positive patients when the algorithms were used in a larger data source with a lower prevalence of HF- the Intermountain Electronic Data Warehouse which closely mirrors the Pediatric Health Information System (PHIS) data source. Based on our findings, up to 46% of patients captured were false positives. A high number of these patients received HF ICD codes in the setting of simple congenital heart disease consisting of left to right shunts without myocardial dysfunction. This suggests that when using administrative data sources for HF population-based research, search algorithms may still include a high proportion of patients without myocardial dysfunction. While this specific concern is unique to pediatric HF, there are numerous other reasons for inconsistent or inaccurate ICD code assignment in large pediatric cohorts, such as limitations in medical documentation, variable coding practices, misspecification, and upcoding.[13] These issues also exist in the congenital heart disease literature, in which ICD code search strategies include a high proportion of false positives (as high as 69.1%) necessitating ICD code combinations in tailored search algorithms.[14] With this context, it is clear that a nuanced approach to case ascertainment is required for pediatric HF research.
Authors in the pediatric HF literature commonly cite ICD code challenges as a methodologic limitation[15–17] because until now, the accuracy of a single ICD code approach has not been reported. This concern is evident in recent work using PHIS to describe pediatric HF epidemiology. In this study, > 50% of children identified as having HF (based on ICD codes) with congenital heart disease had left to right shunts such as ventricular septal defects, or patent ductus arteriosus.[14] While it is possible that these children had true myocardial dysfunction and HF in addition to simple congenital heart disease, it seems more likely that these children received a HF ICD code in the setting of pulmonary over-circulation. Because these patients clearly have a surgically correctable cause of “heart failure” many HF investigators would prefer to exclude them from HF study cohorts, because their outcomes will differ substantially from those with myocardial dysfunction. Based on our findings, the application of an ICD-code search algorithm followed by use of institutional surgical data, or linked data sets such as the Society of Thoracic Surgeons (STS) National Database, can ensure a higher proportion of patients with adjudicated HF from a large data source. Application of STS Primary Procedure codes can also be used to identify children with single surgical interventions for simple shunt lesions, and removal of those patients from the HF study cohort could ensure a cleaner epidemiologic cohort of children with myocardial dysfunction and the syndrome of HF. Other options to characterize patients could include linked cardiac catheterization records or CPT codes for cardiac surgeries and procedures.
This study generates several meaningful contributions. First, the algorithms tested in this study provide an initial framework for case ascertainment in future pediatric HF research, and for the first time, we quantify the performance of ICD code search algorithms. This valuable framework serves as an important cornerstone for next steps in research. Second, we not only describe a high number of false positive HF patients in a n administrative database cohort, but we were able to identify potential means of refining a HF cohort by age or by incorporating surgical or procedural records. In general, these contributions are relevant because optimal case ascertainment using electronic datasets remains critical to improve our understanding of HF epidemiology among children at regional and national levels.
While registries that collect longitudinal pediatric HF data will be powerful future resources for epidemiologic study and may not require ICD codes for case identification, these registries are currently limited in scope.[18–20] Thus, datasets that include electronic health data such as regional datasets like the EDW, health care payor data, and multi-institutional datasets remain an important source of generalizable pediatric HF data. The ICD code-based approaches and refinements we describe in this study could be useful in identifying the population of interest from these datasets.
There are several limitations to this study. The first limitation is that for feasibility, we used a relatively small population for study. While it is possible that a larger data set may result in different algorithm performance, our findings mirror those in the adult literature, and also those described in the CHD literature when using ICD codes for case ascertainment.[14] It is also noteworthy that replicating this research with exclusively ICD-9 or ICD-10 codes may not yield the exact algorithm performance as reported in this study. Fortunately however, crosswalk systems exist to convert between ICD-9 and ICD-10 codes as needed so our work could be replicated using ICD-9 or ICD-10 codes, or both if needed based on the era of interest.
A unique strength of this study is that we were able to assign gold standard HF diagnoses using clinical chart review. Chart review also made it possible to characterize the “false positive” HF patients in the data set, allowing us to propose refinement strategies for the search algorithms. Another strength of this study is that we were able to evaluate the search algorithms in a group of patients with or without a history of hospitalization. This differs from many administrative datasets in which only inpatient data are collected.