Desirability of Outcome Ranking for the Management of Antimicrobial Therapy (DOOR MAT) Reveals Improvements in the Treatment of Bloodstream Infections Caused by Escherichia coli and Klebsiella pneumoniae in Patients from the Veterans Health Administration

Abstract

Background

Reductions in the use of broad-spectrum antibiotics is a cornerstone of antimicrobial stewardship. We aim to demonstrate use of the Desirability of Outcome Ranking Approach for the Management of Antimicrobial Therapy (DOOR MAT) to evaluate the treatment of Escherichia coli and Klebsiella pneumoniae bloodstream infections in patients from the Veterans Health Administration (VHA) across a decade.

Methods

Using electronic records, we determined empiric and definitive antibiotic treatments, clinical characteristics, and 30-day mortality of patients with monomicrobial E. coli and K. pneumoniae bloodstream infections hospitalized in VHA medical centers from 2009 to 2018. Focusing on patients treated with parenteral β-lactams and with available antibiotic susceptibility testing results, we applied a range of DOOR MAT scores that reflect the desirability of antibiotic choices according to spectrum and activity against individual isolates. We report trends in resistance and desirability of empiric and definitive antibiotic treatments.

Results

During the 10-year period analyzed, resistance to expanded-spectrum cephalosporins and fluoroquinolones increased in E. coli but not in K. pneumoniae, while resistance to carbapenems and piperacillin-tazobactam remained unchanged. In 6451 cases analyzed, we observed improvements in DOOR MAT scores consistent with deescalation. Improvement in desirability of definitive treatment compared with empiric treatment occurred in 26% of cases, increasing from 16% in 2009 to 34% in 2018. Reductions in overtreatment were sustained and without negative impact on survival.

Conclusions

DOOR MAT provides a framework to assess antibiotic treatment of E. coli and K. pneumoniae bloodstream infections and can be a useful metric in antimicrobial stewardship.

Clinicians who treat patients with serious bacterial infections base empiric antibiotic therapy on perceived risk for resistance, balancing the goal of effective therapy necessary for survival with the “collateral damage” caused by overtreatment (ie, selection of resistant bacteria and Clostridioides difficile) [1, 2]. Identification of bacteria and antibiotic susceptibility testing (AST) offer the opportunity to formulate definitive antibiotic regimens, escalating to an active agent in the presence of resistance to empiric therapy or deescalating to effective antibiotics with a narrower spectrum of activity. Deescalation, given its impact on individual patient outcomes, population health, and costs, is an aim of antimicrobial stewardship and constitutes high-value care [3].

Antimicrobial stewardship is a focus for healthcare improvement, including at the US Veterans Health Administration (VHA). The VHA National Antimicrobial Stewardship Initiative evolved through a system-wide network of infectious diseases pharmacists and physicians who undertook continued education, review of guidelines, and policy development that have beneficial impact on antibiotic use and resistance [4]. Although not recently assessed, between 2008 and 2011, deescalation occurred in less than one-third of VHA patients treated for pneumonia [5].

Existing tools classify antibiotics according to their spectrum of activity and assess empiric and definitive antibiotic regimens and deescalation [6, 7]. The Desirability of Outcome Ranking Approach for the Management of Antimicrobial Therapy (DOOR MAT) was devised to assess the appropriateness or desirability of treatment selections based on spectrum as well as antibiotic activity in relation to susceptibility profiles [8]. DOOR MAT offers advantages when compared with tools that do not capture whether individual antibiotic choices represent effective therapy for individual infections. Furthermore, DOOR MAT can reflect deescalation and improvement in desirable use of antibiotics in settings where rapid diagnostics are implemented, among other antimicrobial stewardship interventions [9].

Bloodstream infections with Escherichia coli and Klebsiella pneumoniae provide an informative model to study deescalation and apply DOOR MAT. Empirical treatment of such infections often includes β-lactam antibiotics with activity against Pseudomonas aeruginosa (eg, piperacillin-tazobactam and cefepime) or a carbapenem. Additionally, E. coli and K. pneumoniae may harbor extended-spectrum β-lactamases that preclude the use of cephalosporins and that may require definitive treatment with a carbapenem [10]. Conversely, if E. coli or K. pneumoniae are susceptible to narrow-spectrum cephalosporins (eg, cefazolin) or expanded-spectrum cephalosporins (eg, ceftriaxone), deescalation to such agents is advisable after consideration of comorbid conditions, source, and severity of infection.

In this study, our aim was to demonstrate the use of DOOR MAT as an approach to evaluate antibiotic treatment in a large cohort of patients with E. coli and K. pneumoniae bloodstream infections from the VHA. The application of DOOR MAT in this “real-world” population serves to capture changes in empiric and definitive antibiotic treatments and temporal trends in resistance to antibiotics, as well as to assess deescalation in patients with E. coli and K. pneumoniae bloodstream infection.

METHODS

We used the VA Informatics and Computing Infrastructure to access the VHA Corporate Data Warehouse (CDW) in order to study patients hospitalized from 1 October 2008 through 30 September 2018 at VHA facilities, which currently include 170 VA medical centers [11]. The VA Northeast Ohio Healthcare System Institutional Review Board approved the study protocol and granted a waiver of informed consent.

Patients with blood cultures growing E. coli or K. pneumoniae were identified from microbiology data in the CDW. Date and time of cultures and of AST results (interpreted as resistant [R]/intermediate or susceptible [S]) were recorded as reported without consideration of methods or interpretive criteria. Isolates reported as resistant/intermediate to all β-lactams and fluoroquinolones were defined as “difficult to treat” [12]. Date and time of antibiotic administration were obtained from the CDW, as were the following patient-specific data: demographic characteristics, infectious syndromes, and comorbid conditions documented with codes from the International Classification of Diseases, Ninth Revision (ICD-9) and Tenth Revision (ICD-10). Dates of death were extracted from the VHA Vital Status File, and 30-day mortality was calculated from the time that blood cultures were obtained. Inclusion and exclusion criteria and the resulting cohorts for subsequent analysis are summarized in Figure 1. Among patients with multiple blood cultures growing E. coli or K. pneumoniae, cases were separately considered if they occurred >30 days after a prior culture. For each case, we considered antibiotic treatment administered for ≥2 days. We defined empiric antibiotic regimens as those received the day before AST results were available and definitive antibiotic regimens as those received the day after AST results became available.

Application of DOOR MAT

A comparison of empiric and definitive antibiotic treatment of bloodstream infections caused by E. coli and K. pneumoniae was undertaken using DOOR MAT [8]. Cases of E. coli and K. pneumoniae bloodstream infections treated with selected parenteral β-lactam antibiotics were grouped into the following categories of increasing spectrum: cefazolin, ampicillin/sulbactam, ceftriaxone, cefepime, piperacillin-tazobactam, and carbapenems (imipenem, meropenem, and ertapenem). Resistance profiles for each isolate were established according to susceptibility results for cefazolin, ceftriaxone, and imipenem as representative agents routinely tested. A treatment level consisting of antibiotics of “last resort” (eg, polymyxins) was included. As these drugs are not routinely included in susceptibility testing and resistance (R) is rarely observed, we assumed that all isolates were susceptible (S) to these agents. Thus, we defined 4 patterns of escalating resistance: S-S-S-S, R-S-S-S, R-R-S-S, and R-R-R-S.

From treatments and profile combinations, we created a DOOR MAT grid in which we defined 5 ranked categories (Figures 2A and 2B): “desirable treatment,” or treatment matching an isolate’s susceptibility (eg, cefazolin-susceptible E. coli/K. pneumoniae treated with cefazolin); “potentially appropriate,” referring to ceftriaxone-resistant E. coli/K. pneumoniae treated with piperacillin-tazobactam, considering evolving evidence in the past decade regarding appropriateness of that choice vis-à-vis carbapenems [10, 13]; “appropriate but broad,” or treatment with broader spectrum than strictly needed (eg, cefazolin-susceptible E. coli/K. pneumoniae treated with ceftriaxone); “overtreatment,” or relatively undesirable because treatment is much broader than needed (eg, cefazolin-susceptible E. coli/K. pneumoniae treated with carbapenems); and “undertreatment,” or undesirable because of inactive treatment (eg, ceftriaxone-resistant E. coli/K. pneumoniae treated with cefazolin).

In this version of DOOR MAT, the treatments of interest and the classification and ranking of their use for each pattern were created by consensus of the authors. We implemented a “naive” scoring that separates the classifications evenly and compared temporal patterns and clinical outcome associated with this naive scoring vs a shift toward greater penalization of overtreatment vs a shift toward greater penalization of undertreatment. Numerical scores were assigned to each category using the 3 approaches. For all, undertreatment received the lowest score (0) and desirable treatment received the highest score (100; Figure 2B). Approach 1, or naive scoring, had equal distribution of scores among 5 categories. Approach 2 used a strict or less permissive scoring system that penalized all categories other than desirable treatment and scored them closer to undertreatment. Approach 3 used a lax or more permissive scoring system that did not heavily penalize broader treatment than desirable. Scores were analyzed as a continuous variable for desirability of treatment selection for empiric and definitive regimen. Average scores by organism, treatment period, and year were calculated for a range of scoring approaches. A multilevel model was used to further estimate how, adjusting for organism, DOOR MAT scores changed from empiric to definitive periods, increased or decreased over time, and whether the changes from empiric to definitive periods increased or decreased over time. This interaction of treatment period and time was modeled to determine whether changes to therapy in the definitive period (typically deescalation) occurred more frequently or dramatically in recent years.

Clinical Validity of DOOR MAT

We explored whether this implementation of DOOR MAT, that is, the combination of infections, organisms, antibiotics, susceptibility patterns, and scores specified here yields individual patient scores that are unassociated with or, optimally, positively associated with an important patient-level outcome such as 30-day mortality. Using logistic regression models, we estimated the odds ratio and 95% confidence interval (CI) for mortality of a 10-unit increase in DOOR MAT scores. We adjusted for age, organism, Charlson comorbidity index, and a modified Acute Physiology and Chronic Health Evaluation (mAPACHE) score that measures severity of illness using data extracted from CDW about the time of blood culture collection [14]. Separate models were estimated for empiric and definitive DOOR MAT scores and then both were considered in a single model.

RESULTS

We identified 22 904 cases of E. coli and K. pneumoniae bloodstream infections among 21 808 patients hospitalized at VA medical centers from 2009 to 2018. Exclusion of cases with polymicrobial or intrabdominal infection, who did not survive until AST results became available, or who did not receive inpatient antibiotics yielded a cohort of 9026 cases. Focusing on patients treated with β-lactam antibiotics of interest yielded a subset of 6541 cases (E. coli n = 4714; K. pneumoniae n = 1737) to which DOOR-MAT was applied. Even after these exclusions, characteristics of the initial cohort were conserved with a similar median age (>70 years) and Charlson comorbidity index but with lower all-cause 30-day mortality in the DOOR MAT subset (Table 1). Between 2009 and 2018, mortality remained stable in cases of E. coli but improved in cases of K. pneumoniae bloodstream infections (14% vs 9%; Figure 3A).

Antibiotic susceptibility trends in the last decade within the cohort for which inpatient antibiotic use and AST results were available (n = 9026) were notable for decreased susceptibility to fluoroquinolones from 70% to 64% in E. coli, while remaining stable at approximately 88% for K. pneumoniae. Susceptibility to ceftriaxone decreased from 94% to 84% for E. coli but did not substantially change for K. pneumoniae. Susceptibility to piperacillin-tazobactam (>95% for E. coli and >90% for K. pneumoniae) and to carbapenems (>99% and >97%, respectively) was stable (Figure 3B and Supplementary Table 1). The fraction of E. coli and K. pneumoniae classified as “difficult to treat” (ie, resistant to all β-lactams and fluoroquinolones) was 0.06% and 1.1%, respectively.

Regarding empiric therapy, piperacillin-tazobactam was the antibiotic most frequently used for both E. coli and K. pneumoniae bloodstream infections, followed by ceftriaxone and ciprofloxacin. Empiric use of piperacillin-tazobactam decreased from 48% of cases in 2014 to 38% in 2015 and was <40% in subsequent years. However, piperacillin-tazobactam remained the most frequent type of empiric overtreatment, and more than 85% of its use as definitive therapy represented overtreatment. Use of fluoroquinolones decreased during the study period, while use of ceftriaxone, carbapenems (Figure 3), and cefepime (not shown) increased.

Application of DOOR MAT

DOOR MAT was applied to a subset of 6451 cases, as described above. Undertreatment of E. coli and K. pneumoniae bloodstream infections during the empiric period occurred in 2% of cases and optimal treatment in 9% and 8% of cases, respectively. Throughout the past decade, there were reductions in the proportion of cases receiving overtreatment in the empiric period and even larger reductions in the definitive period (Figure 4). With the naive or equally spaced scoring (approach 1), overall mean DOOR MAT scores in the empiric period were 38.4 for E. coli and 35.7 for K. pneumoniae and increased in the definitive period to 49.5 and 45.8, respectively. Comparing 2018 to 2009, mean empiric DOOR MAT scores increased by 5.4 and 6.8, while mean definitive scores increased by 12.3 and 14.7 over time for E. coli and K. pneumoniae, respectively. Using a multilevel model to consider the effects of organism, treatment period (empiric vs definitive), time, and interaction of treatment period and time, we detected a significant interaction of treatment period and time with an estimated effect that implies increasing differences between empiric and definitive therapies with time (definitive vs empiric DOOR MAT score differences increasing by 0.6, 0.54, and 0.53 per year, all P < .001, for naive, strict, and lax scores, respectively). This model result is consistent with increased deescalation from empiric to definitive therapies in recent years (Supplementary Table 2). Similar effects of treatment period and time were observed across DOOR MAT scoring approaches, though with a downward shift in scores for the strict approach and an upward shift in scores for the lax approach. Such consistency demonstrates that, in this particular dataset, DOOR MAT trends are robust to changes in scoring system selection (Figure 4 B).

Clinical Validity of DOOR MAT

In an analysis aimed at clinical validation, DOOR MAT scores ranged from significantly associated with mortality with a protective effect to not significantly associated with mortality (Supplementary Table 3). From logistic regression models, the estimated odds ratios of a 10-point increase in DOOR MAT scores predicting all-cause 30-day mortality, after adjusting for age, organism, Charlson comorbidity index, and mAPACHE, suggested that more appropriate definitive treatment (higher DOOR MAT scores) was negatively associated with mortality under the naive and lax scoring approaches. We did not detect associations between empiric DOOR MAT scores and mortality. The odds ratios ranged from 0.89 (95% CI, .85–.94) for definitive lax approach scores to 1.00 (95% CI, .96–1.05) for empiric strict approach scores. When both empiric and definitive DOOR MAT scores were considered in the same model, the results were generally consistent with the separate models (Supplementary Table 4). Correlations among empiric DOOR MAT scores, definitive DOOR MAT scores, and additional covariates were modest; variance inflation factors fell below 2 for all variables in all estimated models.

Discussion

We applied DOOR MAT, an instrument that assesses the desirability of antibiotic treatment based on its activity against the specific isolate and overall spectrum, to a retrospective cohort of patients with E. coli or K. pneumoniae bloodstream infections from the VHA. Our analysis revealed improvements in DOOR MAT scores during the last decade that signify more desirable empiric and definitive antibiotic therapies and deescalation from broad empiric to narrower definitive therapy. This analysis serves to demonstrate DOOR MAT as a useful metric in antimicrobial stewardship that incorporates antibiotic susceptibility results from individual cases into the scoring method. To be clinically meaningful, DOOR MAT should reconcile treatment desirability with outcomes that are patient-centered and important to the healthcare system. In our analysis, we found that DOOR MAT scores are associated with improved 30-day survival, or at least not associated with increased mortality. Larger effect sizes and better fit statistics were observed for naive and lax scoring approaches than for a strict approach.

Our analysis of this large group of patients with advanced age and comorbid conditions demonstrated that empiric regimens were almost always effective, likely because we focused only on patients treated with parenteral β-lactams. In contrast, an analysis of electronic databases from US hospitals estimated that approximately 1 in 5 patients with bloodstream infections received discordant empirical antibiotic therapies (or to which isolates were not susceptible) [15]. In our study, the high proportion of patients who received overtreatment in the empiric period decreased over time, and deescalation to a more desirable treatment occurred in approximately 25% of cases. Reductions observed in the use of piperacillin-tazobactam coincide with the publication of observational studies suggesting that concomitant piperacillin-tazobactam and vancomycin administration is associated with kidney injury [16, 17]. Reductions in fluoroquinolone use reflect increasing rates of resistance and safety concerns, as well as efforts to curtail the use of this class of antibiotics within the VHA [18]. Increased use of carbapenems may derive from recent evidence that further supports their role in the treatment of bloodstream infections caused by E. coli and K. pneumoniae resistant to expanded-spectrum cephalosporins and the increasing frequency of such phenotypes [10]. It has not escaped our notice that in this cohort, the use of ceftriaxone increased during the last decade. We highlight the temporal trend toward improved all-cause mortality at 30 days across our population and, at the patient level, a neutral to favorable association between mortality and improved DOOR MAT scores. Although not explored here, a favorable impact in length of stay and incidence of C. difficile infection may also occur, as reported in association with antibiotic deescalation in patients with pneumonia at the VHA [19].

Overtreatment and absence of deescalation in a significant proportion of cases emphasize the need to improve the quality of the treatment of patients with E. coli and K. pneumoniae bloodstream infections. After AST results become available, additional clinical considerations may affect the decision to deescalate, such as the source of bloodstream infection (eg, respiratory, gastrointestinal, or genitourinary tracts), comorbidities, illness severity, availability, and cost of antibiotics. Consensus definitions of deescalation have not yet been formulated [20], but existing guidelines for the treatment of infections caused by gram-negative bacteria focus on approaches to empiric treatment selection and treatment of antibiotic-resistant organisms [21]. Development of standard definitions applied to electronic data may facilitate adoption of deescalation as a goal of stewardship programs [22].

The use of rapid diagnostic methods as a strategy to improve the quality of the treatment of bloodstream infection with gram-negative bacteria has yielded mixed results [23]. For instance, rapid AST performed directly in blood cultures that grow gram-negative bacteria (using a morpho-kinetic assay) reduced the time to deescalation (and optimal therapy) but did not lead to improvements in mortality, adverse events, or cost [24]. Inferring that AST results from rapid nucleic acid testing results is challenged by the complexity and varying prevalence of resistant genotypes in gram-negative bacteria and requires further clinical evaluation [25]. DOOR MAT can serve to assess the impact of these and other rapid diagnostic methods in deescalation in gram-negative bloodstream infections [9].

This study has several limitations. Patients were elderly and had a high burden of comorbidities, which may impact the decision to deescalate and limits the generalizability of these observations. We excluded polymicrobial and intrabdominal infections that often require broad-spectrum antibiotics but did not exclude cases of central nervous system infections or neutropenic fever where there are constraints to deescalation, nor did we assess severity of illness as a determinant of deescalation. Furthermore, we introduced survivorship bias by excluding deaths that occurred before definitive treatment. Although there is precedent using VHA’s microbiological, clinical, and pharmacological data to study gram-negative bloodstream infections [26], data may be miscoded or missing. Excluding cases with unavailable AST results likely introduced bias in the type of facilities represented. We did not address differences across facilities in AST protocols, including the adoption of new susceptibility breakpoints for fluoroquinolones, cephalosporins, and carbapenems, or the timing and process of communicating AST results to clinicians [27, 28]. We acknowledge that treatment selections, resistance profiles, and scores applied to this analysis did not capture some relevant clinical and stewardship considerations (eg, emergence of resistance, C. difficile infection). Furthermore, varying the configuration of DOOR MAT may yield different results; our alternative DOOR MAT scorings, however, did not modify overall trends.

In conclusion, in this cohort of patients with E. coli and K. pneumoniae bloodstream infections from the VHA, a high rate of effective empiric treatment was achieved, overtreatment reduced, and the desirability of empiric and definitive antibiotic regimens improved across a decade. Although not a direct assessment of their impact, these findings coincide with antimicrobial stewardship initiatives adopted at the VHA and increased awareness throughout the healthcare system of the harms of unnecessary antibiotic use. We hasten to emphasize that deescalation of empiric therapy on availability of AST results should be a focus of quality improvement efforts in antimicrobial stewardship. DOOR MAT may become a useful tool to inform such interventions, including rapid diagnostic methods, since it provides a framework to evaluate the desirability of antibiotic treatments that incorporates antibiotic efficacy and spectrum. Further work is required to refine DOOR MAT so that it reflects clinical and stewardship goals.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Disclaimer. The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of the Veterans Affairs, the National Institutes of Health (NIH), or the US government.

Financial support. This work was supported by the Veterans Integrated Service Network 10 (VISN 10) Geriatric Research Education and Clinical Center at the Louis Stokes Cleveland Veterans Administration Medical Center (B. M. W., R. A. B., R. L. P. J., F. P.) and by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health (R01AI100560, R01AI063517, R21AI114508, and R01AI072219 to R. A. B. and UM1AI104681 that funds the Antibacterial Resistance Leadership Group and supports R. A. B., R. P., S. R. E). Additional funds and facilities were provided by the Veterans Administration Northeast Ohio Healthcare System and the Veterans Health Administration Office of Research and Development (1I01BX001974 to R. A. B.).

Potential conflicts of interest. R. A. B. receives research grants from Merck, Allergan, Wockhardt, Achaogen, Shionogi, and GlaxoSmithKline. F. P. and R. L. P. J. receive research funding from Pfizer, Merck, and Accelerate; R. L. P. J. has also participated in advisory boards for Pfizer and Roche. S. R. E. reports personal fees from Takeda/Millennium, Pfizer, Roche, Novartis, Achaogen, the Huntington Study Group, University of Pennsylvania, Duke University, Clover, Rakuten, AbbVie, SAB Biopharm, Lung Biotech, Frontier Science Foundation, SVB Leerink, International Drug Development Institute, Council for International Organizations of Medical Sciences, Horizon, Nobel Pharma, NeoVasc, Roivant, Atricure, HI Clinical, ACTTION, Genentech, Amgen, GSK, American Statistical Association, the US Food and Drug Administration, Osaka University, National Cerebral and Cardiovascular Center of Japan, the NIH, Society for Clinical Trials, Statistical Communications in Infectious Diseases (DeGruyter), AstraZeneca, Teva, Austrian Breast and Colorectal Cancer Study Group/Breast International Group, the Alliance Foundation Trials, Zeiss, Dexcom, American Society for Microbiology, Taylor and Francis, Claret Medical, Vir, Arrevus, Five Prime, Shire, Alexicon, Gilead, Spark, Clinical Trials Transformation Initiative, Nuvelution, Tracon, Deming Conference, Antimicrobial Resistance and Stewardship Conference, World Antimicrobial Congress, WAVE, Advantagene, Braeburn, Cardinal Health, Lipocine, Microbiotix, and Stryker. R. P. reports grants from Merck, ContraFect, TenNor Therapeutics Limited, Hylomorph, and Shionogi. R. P. is a consultant to Curetis, Specific Technologies, Next Gen Diagnostics, PathoQuest, Selux Diagnostics, 1928 Diagnostics, PhAST, and Qvella; monies are paid to Mayo Clinic. R. P. is also a consultant to Netflix. In addition, R. P. has a patent on Bordetella pertussis/parapertussis polymerase chain reaction issued, a patent on a device/method for sonication with royalties paid by Samsung to Mayo Clinic, and a patent on an antibiofilm substance issued. R. P. receives an editor’s stipend from the Infectious Diseases Society of America and honoraria from the National Board of Medical Examiners, Up-to-Date, and the Infectious Diseases Board Review Course. All other authors report no potential conflicts.All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References