The Fog May be Lifting Around Antibiotic Use Metrics and Interfacility Comparison Free

(See the Major Article by Yu et al on pages 1677–85.)

Fair is foul, and foul is fair; Hover through the fog and filthy air

– William Shakespeare, Macbeth

As an epidemiologist based at the Centers for Disease Control and Prevention (CDC) for more than 20 years, I worked with an outstanding team on aggregation and dissemination of healthcare-associated infection and antibiotic resistance data to help health departments and hospitals define their healthcare infection problems, create targets for prevention, and track progress. In retrospect, it all seemed relatively straightforward, and compromises were made to be inclusive of all types of healthcare facilities.

In 2017, within 2 months of moving from the CDC to build antimicrobial stewardship activities and research across a large healthcare system, my perspective changed. For one, despite an unleveled playing field regarding some aspects of infection reporting [1], interfacility comparisons did drive action fast. Cross-discipline teams orchestrated by the Office of Quality at Emory Healthcare were very data driven and used rapid-cycle approaches to effect change fast. Even in a setting where comparisons were not necessarily fair, the data were actionable. In part, this was because infections were mostly rare events, providers and prevention teams learned from their occurrence, and sensible reactive measures resulted from reviews of these events.

I also appreciated what others have stated: healthcare delivery is complex. The staff in the system, all passionate, smart, and effective, used the illumination of local data by reasonable metrics to clear the fog of this complex entity. Having some reasonably risk-adjusted metrics of quality (as exist to some degree around healthcare-associated infections) helps clear the fog and avoid unjustified or inaccurate judgments based on improper risk adjustment.

However, the fog is thick as we turn to antibiotic stewardship and antibiotic use, where tracking of such use does not fall into this category of clear, well-defined, reasonably risk-adjusted metrics of quality. Antibiotic use is common (half of hospitalized patients are receiving an antibiotic on any given day), and targeted evaluation cannot occur at large complex facilities in the same way as it does for infection reporting. Some situation-specific feedback and intervention can occur in real time at the bedside thanks to dedicated pharmacists, but targeting patient groups for prospective interventions is much more challenging. Facility-derived antibiotic use measures can and do allow hospitals to interpret their data and make quality improvement decisions for their local facility—that is, to use the data to guide stewardship decisions [2, 3].

Some consensus has emerged around the basics [4]. However, in most publications, comparisons are probably limited to intrafacility or historical comparisons. Exploratory work exists with some hospital groups and such companies as Vizient, Premier, or Becton Dickinson, which provide services that can offer metrics allowing some interfacility comparisons. While at the CDC, I joined in an effort to standardize an approach and avoid competing pathways to benchmarking, establishing a national system for facilities to report, participate, and interpret facility-specific data in comparison with peers. Recently, the CDC launched this effort, the revised Antimicrobial Use option of the Antimicrobial Use and Resistance module. There are many advantages to relying on the National Healthcare Safety Network (NHSN) as a platform for such reporting, which have been commented on elsewhere (eg, standard, multiple-user groups and links to Clostridium difficile counts), but sufficient risk adjustment is not among them at present.

This journal recently published a descriptive summary by van Santen et al [5] of CDC’s proposed (and current) approach to risk adjustment. This article outlines details of the NHSN’s approach to adjusting antibiotic days of therapy at the location–antibiotic group level of observation. The predictive models are a good start, a step forward. The authors reported that their risk adjustments explain variability in some adult inpatient locations more than others, and for some antibiotic groups more than others, but overall these adjustments only modestly explained the variability (eg, the dispersion-based pseudo-adjusted R² was 0.52 for hospital-onset but 0.14 for community-onset broad-spectrum agents) [5].

On the heels of this description of the risk adjustment method is a research study by Yu and colleagues [6] published in this issue of the journal. This research pushes the envelope and challenges NHSN to lean into the risk adjustment method. The authors used data sources in Northern and Southern Kaiser Permanente to approximate the risk adjustment power of the NHSN approach and compare it with a sophisticated adjustment strategy (ie, complex model) as well as a simplified strategy in 35 facilities. Although a head-to-head comparison of a Kaiser-specific benchmarking approach with an NHSN-like approach may not have been their intent, such a comparison is a strength of the study. A simplified risk adjustment strategy (ie, simplified antibiotic stewardship [ASP] ratio) was created to provide parsimonious risk adjustment reliant on assimilation of very basic patient-level factors that most acute-care hospitals should be able to compile monthly as part of routine administrative efforts. Yu et al found that the adjusted metrics and rankings of facility by adjusted antibiotic-use metrics were very comparable between the simplified ASP ratio and the complex model (which included varied patient-level data that would be a challenge for most hospitals to assimilate).

In contrast, the comparison of values produced with the NHSN risk adjustment methods (ie, a fair approximation of the NHSN methods) with those of the complex model correlated poorly with results observed using the simplified ASP ratio. This suggests that the simple model performed as well as the more complex version. Part of the premise for this interpretation is that the complex model accounts for comorbid conditions and indications for prescribing antibiotics, a well-documented predictor of inpatient antibiotic use. The components of the simple ASP model include aggregating administrative data for encounters, including the occurrence of certain diagnosis-related groups, infection present at admission, and type of encounter (ie, they accounted for patients under observation and those undergoing same-day surgery), in addition to location type and patient-days in each location. Some critical access facilities may have more difficulty summarizing these values than large academic centers, but the need for such adjustment may not be felt equally across all types of US hospitals.

The challenge to the NHSN is to begin applying summary metrics of patient mix to better account for the complexity of our current healthcare system. Although the NHSN standardized antimicrobial administration ratio may be sensitive to a stewardship activity [7], the depth of study and the sensitivity of the current metrics to improved prescribing quality are still largely unknown. In this climate of competing priorities by hospital quality offices and heightened transparency of patient safety metrics, rising to this challenge is a very timely response. Otherwise, it will be increasingly difficult to shine a light internally to help our quality office colleagues and muster the resources to really be responsive to data.

Yu and colleagues [6] shed light regarding patient-level factors predictive of antimicrobial use that are feasibly captured in electronic health records and can be leveraged for interfacility comparison. This is good news: a rather simple predictive model using summary encounter–level data improves risk adjustment above the reliance on broad facility-level characteristics and unit location, similarly to current NHSN methods. Measuring antimicrobial use in such a way will be a critical step toward leveraging hospital resources to help reduce unnecessary antibiotic use.

Note

Potential conflicts of interest. The author: No reported conflicts of interest. The author has submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References