Population-Wide Peer Comparison Audit and Feedback to Reduce Antibiotic Initiation and Duration in Long-Term Care Facilities with Embedded Randomized Controlled Trial

Abstract

Background

Antibiotic overprescribing in long-term care settings is driven by prescriber preferences and is associated with preventable harms for residents. We aimed to determine whether peer comparison audit and feedback reporting for physicians reduces antibiotic overprescribing among residents.

Methods

We employed a province wide, difference-in-differences study of antibiotic prescribing audit and feedback, with an embedded pragmatic randomized controlled trial (RCT) across all long-term care facilities in Ontario, Canada, in 2019. The study year included 1238 physicians caring for 96 185 residents. In total, 895 (72%) physicians received no feedback; 343 (28%) were enrolled to receive audit and feedback and randomized 1:1 to static or dynamic reports. The primary outcomes were proportion of residents initiated on an antibiotic and proportion of antibiotics prolonged beyond 7 days per quarter.

Results

Among all residents, between the first quarter of 2018 and last quarter of 2019, there were temporal declines in antibiotic initiation (28.4% to 21.3%) and prolonged duration (34.4% to 29.0%). Difference-in-differences analysis confirmed that feedback was associated with a greater decline in prolonged antibiotics (adjusted difference −2.65%, 95% confidence interval [CI]: −4.93 to −.28%, P = .026), but there was no significant difference in antibiotic initiation. The reduction in antibiotic durations was associated with 335 912 fewer days of treatment. The embedded RCT detected no differences in outcomes between the dynamic and static reports.

Conclusions

Peer comparison audit and feedback is a pragmatic intervention that can generate small relative reductions in the use of antibiotics for prolonged durations that translate to large reductions in antibiotic days of treatment across populations.

Clinical Trials Registration. NCT03807466.

Antibiotic treatment is challenging in long-term care (LTC) facilities. On the one hand, LTC residents are at elevated risk of infection due to immune-senescence with aging, high prevalence of comorbidities, and close proximity to other vulnerable individuals [1]. On the other hand, residents are at elevated risk of antibiotic-related harms due to age-related physiologic changes in renal and hepatic drug clearance [2], polypharmacy [3], less diversity in microflora to protect against Clostridium difficile and antimicrobial-resistant pathogens [4–6].

Antibiotic prescribing in LTC facilities is further complicated by communication challenges in cognitively impaired patients, blunted febrile responses, difficulty distinguishing infection from symptoms of comorbidities, lack of on-site laboratory and imaging tests, and frequent use of phone-in prescriptions. An estimated 25–75% of antibiotic initiations in LTC are unnecessary or inappropriate [1, 6–9]. Even when antibiotic treatments are warranted, they are often continued for longer durations than necessary [10]. Shorter duration antibiotic treatment (≤7 days) has been shown in meta-analyses to be as effective as longer duration treatment for a range of infections, including the leading causes of bacterial infections in LTC residents: urinary tract [11], lung [12], and skin and soft tissue infection [13]. In the LTC population, longer course treatments offer no benefits in recurrence rates but present substantial increased risk of adverse events including C. difficile infection [14].

Antibiotic initiation and duration are highly variable across LTC facilities, and living in high-use homes is associated with greater antibiotic-related harms [15]. This variability is driven by prescriber preferences rather than resident characteristics [16]. Audit and feedback with peer comparisons has the potential to provide social influence towards optimal antibiotic decision-making, without compromising physician autonomy [17]. Therefore, we conducted a population-wide study to test whether peer comparison audit and feedback to LTC physicians could lead to a reduction in antibiotic initiation and duration among LTC residents. An embedded randomized controlled trial (RCT) was employed to test a novel dynamic feedback report against a standard static report. The overarching goal was to reduce antibiotic initiation and duration in LTC facilities, while advancing knowledge regarding audit and feedback methodology.

METHODS

General Study Design and Setting

We conducted a province-wide difference-in-differences study to determine whether peer comparison audit and feedback methods could lead to a reduction in antibiotic initiation and duration among residents of Ontario LTC facilities. For the physician recipients of the peer-comparison reports, we embedded a pragmatic RCT to compare a novel, dynamic feedback report to a static report (ClinicalTrials.gov Identifier: NCT03807466).

Resident and Physician Selection Criteria

Although our intervention was at the physician level, our analytic approach took advantage of patient-level information on antibiotic and clinical outcomes. The study included all Ontarians residing in a LTC facility for any period of time between 1 January 2018 and 31 December 2019, as identified in the Continuing Care Reporting System Long-term Care (CCRS-LTC) database. This record is created from the Resident Assessment Instrument Minimum Data Set 2.0 (RAI-MDS 2.0) and is a mandated clinical assessment that must be completed on all residents annually and at quarterly intervals. The RAI-MDS 2.0 has been well validated for research purposes [18]. During each quarter-year we identified all residents that had received at least one medication prescription of any class from at least one included physician. To be included, prescribing physicians had to care for at least 6 LTC residents; to minimize the risk of patient-re-identification, ICES safeguards prohibit disseminating data involving ≤ 5 patients. For the embedded RCT, we included all residents that had received prescriptions from the subset of physicians receiving the audit and feedback report.

Peer-Comparison Audit and Feedback Intervention

Ontario Health (an agency created by the Government of Ontario with a mandate to connect and coordinate the province’s healthcare) launched a voluntary audit and feedback report (MyPractice) in 2015 to provide LTC prescribers with quarterly information on their antipsychotic, benzodiazepine, and other neurotropic medication prescribing in relation to their peers [19, 20]. This report is considered a static report as it is a written document with graphs which is emailed to recipients. As part of this study, we received approval to add two antibiotic indicators to the report: (1) the percentage of residents initiated on an antibiotic, and (2) the percentage of antibiotic prescriptions with duration exceeding 7 days. The updated report was launched in January 2019. The 4 quarters of 2019 are the intervention period; the 4 quarters of 2018 serve as the preintervention period.

Embedded RCT: Dynamic Versus Static Report

In the embedded RCT, participating physicians were randomized 1:1 to receive the MyPractice report as a novel dynamic, online dashboard versus the usual static (PDF) email attachment (Supplementary Figure 1). Randomization was performed without stratification via random sequence generation centrally at ICES using encoded physician registration numbers. Allocation was concealed centrally and then revealed only to the Ontario Health team that disseminated the reports. Participating physicians were, by definition, aware of the type of report they had received, but the residents they cared for were unaware. The study team at ICES remained blinded to treatment allocation until completion of analyses.

Both versions of the report were developed with input from infectious diseases, implementation science, information technology, and quality improvement specialists, and then improved through an iterative, user-centered design process; both included the new antibiotic duration and initiation indicators; both offered the same change ideas; both offered details on data quality and caveats. However, the dynamic report was provided as a website link and enabled users to land immediately on the visual of the 2 antibiotic indicators and to toggle seamlessly between personal data, multiple peer comparisons, and linked change ideas. Although both types of reports have advantages, the dynamic report can incorporate more contextual data to aid clinicians’ understanding of their prescribing. These data are designed to overcome a key barrier to change in which the comparison of prescribing rates could be rationalized by the prescriber as being due to perceived differences in patient population. Screen shots of a sample dynamic report and static report are provided as Supplementary Material.

Primary Outcomes

The 2 primary outcomes for the peer comparison audit and feedback intervention were: (1) proportion of residents initiated on an antibiotic during the quarter, and (2) proportion of antibiotic treatments exceeding 7 days during the quarter. These antibiotic outcomes were derived from the Ontario Drug Benefit database, which includes all prescriptions dispensed to Ontarians over age 65 including all LTC residents and has an accuracy exceeding 99% compared to direct chart abstraction [21].

Secondary Outcomes

To measure the potential clinical benefit of reducing inappropriate antibiotic use, we captured resident emergency department (ED) visits and hospitalizations for potential antibiotic-related harm during each quarter, including: allergy, general medication adverse events, diarrhea, C. difficile infection, or infection with an antibiotic-resistant organism [15]. As a balancing measure, we examined ED visits and admissions for infection. We measured the net clinical impact of the intervention, by comparing all-cause ED visits, hospitalizations, and mortality. As a tracer outcome (not expected to change in the intervention year), we measured anti-psychotic and/or benzodiazepine prescriptions, medications that were included in the feedback report prior to (and during) the study period.

Data Sources

The drug, hospitalization, and emergency department databases were linked using unique encoded identifiers and analyzed at ICES. ICES is an independent, nonprofit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze healthcare and demographic data, without consent, for health system evaluation and improvement. The linked administrative databases at ICES which capture these data have been well validated in previous research [22, 23], including extensive studies of antibiotic use [24, 25], quality of LTC [26], and antibiotic use in LTC facilities [15, 16].

Statistical Analysis

Baseline characteristics for both residents and their physicians were calculated. Groups were stratified by enrollment status in the audit-and-feedback report and type of report (static vs dynamic) for those in the embedded RCT. Standardized differences between strata were calculated in order to evaluate whether subjects had differing baseline characteristics from the outset [27].

To evaluate the overall impact of the antibiotic indicators, we compared outcomes among residents treated by physicians enrolled in the audit and feedback report, to residents treated by other LTC physicians. In this difference-in-differences approach, we contrasted the change in outcomes in the 4 quarters of the intervention year to outcomes in the 4 quarters of the preintervention year [28]. A logistic difference-in-differences regression model included a dummy variable for the report group in the intervention period. In addition, the model included random intercepts to account for clustering of residents within physicians. The model also included resident characteristics from their most recent RAI-MDS 2.0 assessment including age, sex, 14 comorbidities, degree of functional dependence for each activity of daily living, bowel/bladder incontinence, hearing/visual impairment, and device use [15, 18].

We projected the total annual antibiotic days of treatment prevented in LTC facilities, by multiplying the adjusted risk difference between residents cared for by physicians receiving versus not receiving feedback, by the average number of excess days of treatment in longer courses, and the total number of these longer courses in the preintervention year.

The embedded trial of report format was evaluated using an intention to treat approach. Using logistic regression, we compared outcomes in residents of physicians randomized to the dynamic versus static report. In a secondary per-protocol analysis, we limited the analysis to residents treated by physicians who opened at least one of the emails containing a feedback report. The primary analyses in the embedded trial used the constrained baseline approach [29] and were unadjusted, but in secondary analyses we adjusted for resident characteristics (as above).

Assuming that standard audit and feedback would achieve an 11% improvement in antibiotic prescribing [30], we determined that to detect a further 3% incremental improvement with the dynamic report we would require 786, 352, or 174 physicians if standard deviation was 15%, 10%, or 7% (power 0.80, 2-tailed alpha 0.05). Statistical analyses were conducted using SAS Enterprise Guide 7.15.

RESULTS

Long-Term Care Physicians and Residents

After exclusions, there were up to 1140 physicians prescribing in Ontario LTC facilities during each quarter of 2018 and 2019 (Supplementary Figure 2). In total, 1238 unique physicians were practicing in LTC facilities for at least some period of the 2019 intervention year (Table 1). Ontario LTC physicians were predominantly male (789, 64%) and with extensive years of practice (median 28, interquartile range [IQR] 15–38 years).

There were up to 75 950 residents in Ontario LTC facilities during each quarter year of 2018 and 2019 (Supplementary Figure 3). In total, 96 185 unique residents were living in LTC for at least some period of the 2019 intervention year (Table 1). Ontario LTC residents were predominantly female (64 572, 67%), of advanced age (median 85, IQR 78–91 years old), and with a high prevalence of dementia, comorbidities, and assistance requirements (Table 1).

Impact of Peer Comparison Audit and Feedback Versus No Audit and Feedback

Of Ontario LTC physicians prescribing in 2019, 343 (28%) were enrolled to receive audit and feedback; the other 895 (72%) received no audit and feedback report (Table 1). Physicians receiving a report had longer median practice experience (31 vs 26 years) and lower likelihood of foreign medical training (15 vs 21%) but were otherwise similar to other Ontario LTC physicians (Table 1). Moreover, these physicians cared for similar residents as those not enrolled (Table 1).

There was a temporal decline in antibiotic initiation and duration between the first quarter of 2018 and the last quarter of 2019 (Figure 1). The proportion of residents initiated on an antibiotic decreased from 28.4% to 21.3%; the proportion of antibiotic treatments that exceeded 7 days decreased from 34.4% to 29.0%.

Differences-in-differences analysis indicated that audit and feedback reporting was not associated with a greater decline in antibiotic initiation (adjusted difference 0.10% (−0.51 to 0.67%, P = .73) (Table 2). However, audit and feedback reporting was associated with a greater decline in use of prolonged antibiotics (adjusted absolute difference −2.65%, 95% confidence interval [CI]: −4.93 to −.28%, P = .026) (Table 2). Secondary clinical outcomes were similar among residents treated by physicians enrolled (vs not enrolled) in the feedback program (Table 2).

Estimated Reduction in Antibiotic Days of Treatment in Long-Term Care Facilities

Given the 2.65% reduction in prolonged duration antibiotic use in residents cared for by feedback-recipients, the intervention was associated with an estimated total reduction of 335 912 days of antibiotic treatment in 2019. If the antibiotic feedback was administered to all Ontario LTC physicians, this would extrapolate to an annual reduction of 840 549 days of antibiotic treatment.

Impact of Dynamic Versus Static Feedback Report

Among the 343 physicians enrolled in audit and feedback, 171 were randomized to the static report and 172 to the dynamic report (Table 3); these physicians were responsible for the care of 17 456 and 18 738 long-term care residents (Table 3). Physician and resident characteristics were well balanced among the 2 groups. There was no discernible difference in antibiotic initiation or duration between recipients of these 2 types of reports (Figure 2, Supplementary Table 1), nor were there differences in clinical outcomes (data not shown).

Most physicians opened at least one of the quarterly emails containing the feedback report, including 154 (90%) in the static report and 155 (90%) in the dynamic report arm (Supplementary Tables 2, 3, and 4). A per-protocol analysis in these groups detected no statistically significant differences in antibiotic or clinical outcomes (Supplementary Figure 4, Supplementary Table 5).

Discussion

This province-wide study among more than 1200 physicians and 90 000 LTC residents revealed preexisting trends toward declining antibiotic initiation and duration in Ontario LTC facilities and observed that peer comparison audit and feedback was associated with a significantly greater decline in the use of long duration treatments. Audit and feedback was not associated with a detectable reduction in antibiotic initiation beyond existing trends, but effects on treatment duration were associated with 300 000 fewer days of antibiotic use in 2019. The embedded RCT of a dynamic versus static report detected no incremental differences in impact.

There are barriers to optimizing antibiotic use in LTC, and many residents receive treatments that are avoidable or unnecessarily prolonged [1, 6–9]. Although several types of interventions have been associated with reductions in antibiotic use in LTC, these often require labor-intensive approaches with questionable sustainability [31]. Peer-comparison audit and feedback offers a scalable and sustainable approach to improving prescribing [32]. The magnitude of reduction achieved by our intervention is in line with a prior systematic review, which indicates a median 4.3% absolute increase in healthcare professional compliance [32]. Although this is a small change for an individual prescriber, our study demonstrates how this can lead to large impacts on antibiotic use over a broad jurisdiction.

We observed temporal reductions in antibiotic initiation and duration that predated the intervention. These reductions are encouraging and may reflect an impact of presentation and publication of our prior province-wide studies, which had identified antibiotic overprescribing [33], an association with harms for residents [15], and that these variations were driven by prescriber habits [16]. Alternatively, these reductions may have been driven by other public health, Choosing Wisely, and stewardship interventions [34]. Even in the context of these coexisting efforts, the study intervention was associated with a further reduction in antibiotic duration beyond the preexisting trend. By systematically measuring professionals’ prescribing patterns, audit and feedback can overcome barriers to self-assessment, identify suboptimal practices, and motivate quality improvement [35]. Our intervention incorporated evidence-based behavior change techniques to target determinants of behavior, including cognitive and affective attitudes, normative beliefs, and self-efficacy [35]. Although the audit and feedback intervention successfully reduced antibiotic duration, it was not associated with detectable reductions in antibiotic initiation. Perhaps duration changes are more easily responsive to normative comparisons linked to evidence supporting short treatment durations. Antibiotic initiation decisions may be more complex, subject to more deeply ingrained prescriber behaviors, and noninitiation potentially associated with greater perceived harms. Alternatively, our initiation metric may have been too general and could have had more impact if it involved more indication-specific initiation comparisons, such as percent of patients with asymptomatic bacteriuria initiated on antibiotics.

Extensive prior studies have confirmed audit and feedback is effective, but there is a lack of rigorous science to improve upon the impact [30]. One possible explanation for the lack of differences in effects between the dynamic and static reports is that prescribers enrolled in the audit and feedback program were more familiar with the static report. The static report also went through recent revisions that may have made it difficult for the dynamic report to provide incremental benefit. Although the dynamic report was not superior, it was not inferior and may carry advantages for future use including easier tailoring to individual physician priorities. Although this particular audit and feedback innovation (dynamic dashboard) did not influence prescribing, we now have an opportunity to use the ongoing program as an implementation laboratory to serially test novel feedback methods [36].

Our study was vulnerable to bias related to voluntary physician participation, but we mitigated this through difference-in-differences analysis that accounted for temporal trends and potential differences in resident characteristics. Furthermore, we applied rigorous, blinded RCT methodology to test a novel feedback strategy. Our intervention period was 1 year, and we have not assessed long-term sustainability. However, lower rates of benzodiazepine and antipsychotic use in the audit and feedback group suggests this intervention will be sustainable, given that those quality indicators were implemented 2 years prior. Although administrative databases can sometimes be identified as a study weakness, we believe this was actually a strength, given that these databases are well validated, and the measurement of the primary outcomes utilized a provincial drug benefit database with near perfect accuracy in measuring antibiotic dispensation [21]. Moreover, routinely collected, accumulating data in administrative data sets offers a cost-effective opportunity to implement and evaluate antimicrobial stewardship interventions at scale across large populations.

This population-wide audit and feedback intervention was associated with a reduction in use of prolonged antibiotic treatments but not with measurable reductions in antibiotic initiation. The small percentage reduction in long duration use by individual prescribers was associated with large reductions in days of antibiotic treatment in the overall LTC population. Audit and feedback is a pragmatic, scalable intervention to improve antibiotic use, and when coupled with evaluation systems using administrative databases it could generate sustainable and large reductions in antibiotic use.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information (CIHI). The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. For the full trial protocol of the embedded RCT, please see online supplementary material.

Financial support. This work was supported by a collaboration across Public Health Ontario; Ontario Health and ICES. ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). This study also received funding from: the Canadian Institute for Health Research (CIHR) (grant number 378064 to N. D.).

Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References