An Automated Surveillance Strategy to Ide.jpegy Infectious Complications After Cardiac Implantable Electronic Device Procedures

Abstract

Background. The optimum approach for infectious complication surveillance for cardiac implantable electronic device (CIED) procedures is unclear. We created an automated surveillance tool for infectious complications after CIED procedures.

Methods. Adults having CIED procedures between January 1, 2005 and December 31, 2011 at Duke University Hospital were ide.jpegied retrospectively using International Classification of Diseases, 9th revision (ICD-9) procedure codes. Potential infections were ide.jpegied with combinations of ICD-9 diagnosis codes and microbiology data for 365 days postprocedure. All microbiology-ide.jpegied and a subset of ICD-9 code-ide.jpegied possible cases, as well as a subset of procedures without microbiology or ICD-9 codes, were reviewed. Test performance characteristics for specific queries were calculated.

Results. Overall, 6097 patients had 7137 procedures. Of these, 1686 procedures with potential infectious complications were ide.jpegied: 174 by both ICD-9 code and microbiology, 14 only by microbiology, and 1498 only by ICD-9 criteria. We reviewed 558 potential cases, including all 188 microbiology-ide.jpegied cases, 250 randomly selected ICD-9 cases, and 120 with neither. Overall, 65 unique infections were ide.jpegied, including 5 of 250 reviewed cases ide.jpegied only by ICD-9 codes. Queries that included microbiology data and ICD-9 code 996.61 had good overall test performance, with sensitivities of approximately 90% and specificities of approximately 80%. Queries with ICD-9 codes alone had poor specificity. Extrapolation of reviewed infectious rates to nonreviewed cases yields an estimated rate of infection of 1.3%.

Conclusions. Electronic queries with combinations of ICD-9 codes and microbiologic data can be created and have good test performance characteristics for ide.jpegying likely infectious complications of CIED procedures.

Surveillance for infection after invasive procedures is a key component of quality control programs. For example, surveillance and feedback of surgical site infection data lead to lower rates of subsequent infections [1].

Procedures involving cardiac implantable electronic devices (CIED) increased by almost 100% between 1993 and 2008, during which time rates of infectious complications also increased [2–5]. Although demographics, provider and facility characteristics, device types, adverse event rates, compliance with guideline recommendations, and Centers for Medicare and Medicaid Services data from the majority of CIEDs are reported into the national ICD Registry managed by the American College of Cardiology, there is no validated surveillance strategy for infectious complications for these procedures [6]. Reported infectious complication rates for CIED procedures vary from <1% to >5.6%, although outcome measures have differed between studies [7–12].

The Duke Infection Prevention and Healthcare Epidemiology group has created automated surveillance systems for several nontraditional surgical procedures, including endoscopic retrograde cholangiopancreatography and cardiac catheterization [13, 14]. These computer-based surveillance systems allow prospective monitoring of potential infections to target intensive infection control efforts as threshold rates of infection are reached. The objective of this study was to create an automated surveillance tool for infectious complications after CIED procedures using International Classification of Diseases, 9th revision (ICD-9) codes and microbiology data.

METHODS

Study Population

All patients ages 18 and older who underwent CIED procedures at Duke University Hospital between January 1, 2005 and December 31, 2011 were ide.jpegied using ICD-9 procedure codes via queries of the Duke Enterprise Data Unified Content Explorer (DEDUCE) system [15].

Ide.jpegication of Potential Infectious Complications

Potential infection-related complications among this cohort were ide.jpegied using specific ICD-9 diagnosis codes and microbiology data for the 365 days after the index procedure. The ICD-9 diagnosis codes included for data extraction and analysis are listed in Supplementary Appendix 1. The following microbiology data (as labeled by our microbiology laboratory for the study period) were included for analysis and were flagged if positive: bacterial and fungal blood cultures, aerobic and anaerobic cultures from sources other than blood, acid-fast bacillus cultures from any source, bacterial body fluid cultures, bacterial tissue cultures, fungal cultures from sources other than blood, and bacterial catheter tip cultures, which may have included cultures from leads after extraction. Urine, cerebrospinal fluid, respiratory, eye, ear, and stool cultures were excluded from analysis, as were any cultures that did not have a source ide.jpegied.

Infectious Complication Definition and Validation

We completed in-depth chart reviews to create a criterion standard comparator for the candidate automated surveillance definitions. Three groups of patients were ide.jpegied for further chart review: (1) all cases who had positive microbiology cultures obtained from the sources listed above within 365 days of the index procedure, (2) a random subset of 250 patients who had ICD-9 codes indicative of potential infectious complication assigned within 365 days after index procedure (3.5% of all sampled procedures and 14.8% of those with positive ICD-9 codes), and (3) a random subset of 120 of patients who had neither positive cultures as above nor ICD-9 codes indicative of infection assigned within 365 days after index procedure (1.7% of all sampled procedures and 2.2% of those with neither finding; Figure 1). The numbers of randomly selected charts within groups 2 and 3 were chosen based on estimated reviewer capacity.

Eight experts in infectious diseases and healthcare epidemiology reviewed clinical documentation, microbiologic data, echocardiographic data, and radiographic data from the available electronic medical records for 365 days from the reviewed procedure to determine whether an infectious complication occurred. Cases were reviewed by a single reviewer. Reviewers used the definition proposed by Chamis et al [16] to define infectious complications of CIED procedures. In brief, this defines clinical evidence of cardiac device infections by physical findings including erythema, dehiscence, and fluctuance, with confirmation based on microbiologic or echocardiographic findings. Reviewers further categorized infectious complications as follows: CIED pocket infection, CIED lead infection, endocarditis, and bacteremia. Blood culture contaminants were defined as skin commensal organisms found in only 1 blood culture specimen within a series, as previously described by Bekeris et al [17]. Patients with contaminated blood culture specimens were not included as incident cases.

Cases of CIED pocket infections were defined as “microbiologically confirmed” if cultures or Gram stain from the generator pocket were positive for organisms. Cases of intravascular component (lead) infections were defined as microbiologically confirmed if cultures or Gram stain from the lead were positive for organisms. Cases of endocarditis were defined clinically as patients for whom valvular or lead vegetations were detected by echocardiography or for whom modified Duke criteria for infective endocarditis were met [18, 19]. Cases of bacteremia were defined as patients with a positive blood culture within 1 year of device implantation. Cases were defined as microbiologically confirmed when cultures or Gram stain from the appropriate clinical site were positive for an infecting pathogen. Cases of “culture-negative” infections were defined as patients with the aforementioned clinical signs of infection and without microbiologic confirmation who subsequently underwent lead or device extraction or were treated with a course of intravenous antibiotics.

Statistical Analyses

Multiple series of 5 different queries each were performed to ide.jpegy the best electronic surveillance definition. Each series included queries with microbiology data alone, ICD-9 data alone, both microbiology data and ICD-9 data, limited subsets of both, and inclusion of the performance of either transthoracic or transesophageal echocardiography. The ICD-9 code combinations were tested within the chart-reviewed population as follows: all ICD-9 codes, the infectious code subgroup without code 996.61 (all codes ≤790.7), the top 6 most frequently ide.jpegied codes within the cases (codes 421.0, 790.7, 996.01, 996.04, 996.61, and 996.72), the top 6 most frequently ide.jpegied codes without code 996.61 (codes 421.0, 790.7, 996.01, 996.04, and 996.72), and the top 3 most frequently ide.jpegied codes (codes 790.7, 996.61, and 996.72). Code 996.61 was excluded from certain combinations to determine performance of more specific infectious codes. Microbiology inclusion within the queries involved any microbiology specimen types (“Micro”) and a limited inclusion set for cases with 2 or more positive cultures of the following specimen types: “Limited Micro”, including tissue culture, bacterial blood cultures, and both anaerobic and aerobic bacterial cultures from other sites. Additional queries were created adding the performance of echocardiograms within the 365- and 90-day time frames. Queries were designed to include data at both 365 and 90 days from the time of the index procedure.

Sensitivity, specificity, positive predictive values, and negative predictive values for all specific query combinations were calculated compared with infectious complications ide.jpegied by expert reviewers (our criterion standard). Electronic queries were tested among the chart-reviewed procedures only. Test performance curves were created for the top 3 performing series of ICD-9 codes. Data analyses were performed using Microsoft Excel (Redmond, WA) and Stata version 9.2 (College Station, TX).

Institutional Review Board Statement

This study protocol was approved by the Institutional Review Board at Duke University Health System (Pro00037480).

RESULTS

A total of 6097 patients had 7137 procedures during the study time frame. Overall, the mean age of those undergoing procedures was 65.9 years, and 2695 (37.8%) of the procedures were performed in women. Of the 7137 procedures, 188 (3%) were noted to have positive microbiology data within 365 days, whereas 1672 (23%) procedures had 1 of the designated infection-related ICD-9 codes assigned within 365 days. Overall, 1686 (24%) procedures were ide.jpegied as having a potential infectious complication by 1 of the 2 methods (Figure 1), and 174 (2%) were ide.jpegied by both methods.

We determined the frequency of ide.jpegication of each ICD-9 code and microbiology data type both in the cohort as a whole and in those procedures ide.jpegied as infectious complication cases by expert review (Table 1). Overall, ICD-9 codes were subdivided into “infectious” (codes ≤790.7 and 996.61) and “mechanical” (other codes ≥996.0) subtypes, with 2456 codes ide.jpegied for 1376 patients. The ICD-9 code 996.01 (mechanical complication due to cardiac pacemaker [electrode]) was the code most frequently ide.jpegied within the entire cohort (found in 439 procedures, or 6% of entire cohort, and 13 cases, or 20% of complications), whereas code 996.61 (infection and inflammatory reaction due to cardiac device, implant, and graft) was the most frequently ide.jpegied within the cases (found in 349 procedures, or 5% of entire cohort, and 59 cases, or 91% of complications). Seven hundred ninety-nine positive cultures were ide.jpegied after 188 procedures. Bacterial blood cultures were the most common positive culture type in the entire cohort (119 procedures, or 2% of entire cohort, and 24 cases, or 37%). Bacterial tissue cultures were the most frequent positive culture type among the cases (76 procedures, or 1% of entire cohort, and 41 cases, or 63%).

Of the 558 cases that underwent further chart review, 101 were deemed to be true infectious complications and occurred within 365 days of the procedure. However, 36 of these infections followed procedures initially performed at outside institutions, leaving 65 cases of confirmed infection from our home institution for further analysis. Coagulase-negative staphylococci were the most common pathogens that caused infection, isolated in 27 (42%) of these 65 cases, followed by methicillin-sensitive (15, or 23%) and methicillin-resistant (7, or 11%) Staphylococcus aureus (Supplementary Appendix 2). Sixty cases were ide.jpegied by both ICD-9 codes and microbiologic data, whereas the remaining 5 were ide.jpegied by only ICD-9 codes, corresponding to an infection rate of 31.9% of procedures with associated microbiologic data and 0.3% of those with an ICD-9 code but without microbiologic data. No infectious complications were found in the 120 charts reviewed that were not ide.jpegied by either microbiologic or ICD-9 sources. Extrapolating these estimates to the 6579 remaining procedures that were not reviewed by experts would yield an expected 30 additional cases ide.jpegied by ICD-9 codes alone. In sum, 95 cases after 7137 procedures performed at Duke University Medical Center would produce an estimated infection rate of 1.3%.

Test performance characteristics of different query combinations at 365 days are shown in Table 2, and full results at 365 days and 90 days are available in Supplementary Appendix 3. In general, queries that included microbiology data and the ICD-9 code 996.61 (queries 1b, 3b, and 5b on Table 2 and Supplementary Appendix 3) had good overall test performance with sensitivities of approximately 90% and specificities of approximately 80%. The ICD-9 code combinations without microbiology data had very high sensitivity with generally poor specificity (queries ending in “a” on Table 2 and Supplementary Appendix 3). Inclusion of echocardiography performance and/or using the Limited Micro data increased specificity to >90% but decreased sensitivity (series ending in “c”, “d”, or “e” in Table 2 and Supplementary Appendix 3). Test performance curves are shown at 365 days from the index procedure (Figure 2) and at 90 days from the index procedure (Figure 3) for the 3 highest performing series of ICD-9 codes (corresponding to queries beginning with 1, 3, and 5 on Table 2 and Supplementary Appendix 3).

DISCUSSION

Calculation of infectious complication rates is an integral part of quality measurement for surgical and procedural specialties. However, best practices for determining rates of infections have not been elucidated for many procedures. This study shows that electronic strategies may be feasibly created for infectious surveillance for CIED procedures. In our cohort, queries with combinations of microbiology data and ICD-9 codes exhibited a high sensitivity and high-to-moderate specificity for ide.jpegying patients who developed an infectious complication following these nontraditional procedures.

Given the increasing frequency of CIED and other nontraditional procedures, comprehensive infection control programs need additional strategies to monitor infectious complications other than reliance on provider recognition and reporting. Our primary goal was to create a novel electronic tool that exhibited a high sensitivity for CIED infectious complications, compared with expert chart reviews. This ensures such a tool would miss a minimal number of cases once deployed.

Because just under one third of microbiology-ide.jpegied potential cases were true cases, any search strategy, including those described in this study, would generate many false-positive results. Therefore, implementation of an “enhanced” surveillance strategy may include a subsequent, secondary human review. Although we do not have an a priori “number needed to screen” to make this enhanced strategy feasible for implementation, using the top 3 ICD-9 codes with microdata would leave just over 1 false positive for each ide.jpegied case and miss approximately 10% of cases. Such a strategy may optimize the human reviewer's efficiency with the certainty of ide.jpegication a human reviewer provides. The enhanced tool could then be used prospectively to validate our rate of infections and thereafter to ide.jpegy rate changes that may indicate outbreaks or clusters of infections at the medical center level. In addition, this CIED procedure monitoring strategy could become part of a wider, mu.jpegaceted surveillance package for nontraditional procedures at our institution.

Combinations of both microbiology data and ICD-9 codes provided the best test performance characteristics. Previous work has shown administrative data alone often are inaccurate for ide.jpegying healthcare-associated infections [20, 21]. Taken in isolation, ICD-9 codes greatly overestimated the number of infectious complications within our cohort. This likely was influenced by the inclusion of mechanical ICD-9 codes in our searches, which was done because we thought providers might use these codes for devices removed for potential infections. However, even code 996.61 (infection and inflammatory reaction due to cardiac device, implant, and graft), which had a sensitivity >90% for the reviewed cases, had a positive predictive value of only 17% when used alone in this population. This indicates that the decision to remove a device may be unclear and that providers may have different thresholds for labeling a device removal as secondary to infection. Furthermore, ICD-9 codes without adjunctive microdata have poor specificity because many different potential infectious complications, such as urinary tract infections, may be unrelated to the CIED procedure, and many noninfected devices must be replaced for mechanical or duration-of-life reasons.

Rates of infectious complications within our cohort, even with the most pessimistic assumptions, were at the lower end of previously published estimates [3–5, 7–12]. Authors of the most recent estimate of rates of CIED infections over a 16-year period demonstrated that rates increased from 1.5% to 2.4% because patients with more comorbidities have had more procedures [3]. However, this estimate was based on national inpatient discharge data, so it is likely that individual centers with higher volume and longer experience with implantation may have lower rates of complications.

Because surgical site infection time definitions were recently reduced from 365 to 90 days, we tested both cutpoints for our data. The shorter time frame missed 17 late infections in our population. Although these represented true infections under our study definition, it is unclear whether later infections reflect complications of the initial procedure itself or occur secondarily; thus, infections occurring beyond 90 days may not be meaningful indicators of procedure quality [22].

Advantages of this study include the inclusion of several years of procedure data and an ability to analyze the effect of multiple different screening query combinations by confirming the case definition with chart reviews. By reviewing many ICD-9-only cases and procedures without associated data, we were able to ensure that clinical factors such as previously received antibiotics and culture types excluded from our data capture did not falsely lower the number of cases we ide.jpegied. Furthermore, review of the microbiology-positive cases allowed us to exclude cases with infections such as bacteremia from other sources and those unrelated to the device/procedure. In addition, queries can be quickly constructed and accurately repeated because our electronic database query system is robust.

The limitations of this work include the lack of structured definitions and time frames for infectious complications of nontraditional surgical procedures. This is a shared limitation for many surveillance methodologies, which we addressed by using content experts for chart review. In addition, we had only 1 expert review each case, so we were unable to test reliability. Second, by not reviewing larger fractions of the ICD-9-positive or -negative query cohorts, our estimate may be subject to sampling bias. However, we chose the reviewed charts randomly to minimize this potential bias and sampled 8% of all procedures, including >25% of those ide.jpegied by either method. Therefore, we have no reason to believe that the nonreviewed cohort differs significantly from the rest of our population. Furthermore, although we only examined cases with our chosen microbiologic data and a subset of those with our selected ICD-9 procedure codes, none of the additional 120 reviewed charts with neither ide.jpegication source were classified as infections. Therefore, we believe this series of queries captured the vast majority of true infections. Next, we were unable to determine the “case mix” of initial and follow-up procedures for this cohort, and infection rates have been shown to differ between first-time implanted devices and re-implantations [7, 23]. Future work should focus on procedure type to better define rates for primary versus revision procedures. Finally, this was a retrospective review of procedures from a single academic center; a prospective, multicenter validation of our query tool is needed.

CONCLUSIONS

In conclusion, electronic queries with combinations of specific ICD-9 codes and microbiologic data can be created and have good test performance characteristics for ide.jpegying likely infectious complications of CIED procedures. Future work on prospective validation of these queries and integration of these and similar strategies into broader surveillance efforts should improve patient safety after nontraditional procedures and inform future quality improvement and assessment practices.

Acknowledgments

We thank Ann Marie Navar for assistance with database management.

Potential conflicts of interest. All authors: No reported conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.

References