Rapid Diagnostic Tests and Antimicrobial Stewardship Programs for the Management of Bloodstream Infection: What Is Their Relative Contribution to Improving Clinical Outcomes? A Systematic Review and Network Meta-analysis

Abstract

Background

Evidence about the clinical impact of rapid diagnostic tests (RDTs) for the diagnosis of bloodstream infections is limited, and whether RDT are superior to conventional blood cultures (BCs) embedded within antimicrobial stewardship programs (ASPs) is unknown.

Methods

We performed network meta-analyses using results from studies of patients with bloodstream infection with the aim of comparing the clinical impact of RDT (applied on positive BC broth or whole blood) to conventional BC, both assessed with and without ASP with respect to mortality, length of stay (LOS), and time to optimal therapy.

Results

Eighty-eight papers were selected, including 25 682 patient encounters. There was an appreciable amount of statistical heterogeneity within each meta-analysis. The network meta-analyses showed a significant reduction in mortality associated with the use of RDT + ASP versus BC alone (odds ratio [OR], 0.72; 95% confidence interval [CI], .59–.87) and with the use of RDT + ASP versus BC + ASP (OR, 0.78; 95% CI, .63–.96). No benefit in survival was found associated with the use of RDT alone nor with BC + ASP compared to BC alone. A reduction in LOS was associated with RDT + ASP versus BC alone (OR, 0.91; 95% CI, .84–.98) whereas no difference in LOS was shown between any other groups. A reduced time to optimal therapy was shown when RDT + ASP was compared to BC alone (−29 hours; 95% CI, −35 to −23), BC + ASP (−18 hours; 95% CI, −27 to −10), and to RDT alone (−12 hours; 95% CI, −20 to −3).

Conclusions

The use of RDT + ASP may lead to a survival benefit even when introduced in settings already adopting effective ASP in association with conventional BC.

Management of bloodstream infection (BSI) requires prompt initiation of effective treatment to improve patient outcomes [1–3]. The gold standard for the diagnosis of BSI is the blood culture (BC), which suffers from several limitations, including long turnaround times and limited sensitivity [4]. The implementation of matrix-assisted laser desorption ionization–time-of-flight mass spectrometry (MALDI-TOF MS) in routine microbiology in the past 2 decades has offered significant advance with respect to shortening time to results. Nonetheless, MALDI-TOF MS is still mainly a culture-dependent system, most frequently applied on isolates from cultures or early subcultures from positive BC bottles [5] and thus still has substantial limitations in terms of effective turnaround time, which should be taken into account when evaluating its clinical utility [6].

Several further novel diagnostic tests have emerged in the last years with the aim of shortening time to results and improving sensitivity of conventional BC methods [7]. These are mainly molecular tests, often based on the detection of microbial DNA from BC broth or directly from whole blood and are commonly referred to as rapid diagnostic tests (RDT) [8].

The diagnostic accuracy of RDT is mostly adequate to meet criteria for clinical implementation according to the main international regulatory agencies [8, 9]. However, data about the impact of RDT on clinical outcomes are scarce, and the best way to implement RDT in clinical practice is yet to be defined.

A meta-analysis published in 2017 showed a decreased mortality and length of stay (LOS) associated with the use of RDT when these were used in association with antimicrobial stewardship program (ASP), but not in their absence [10]. This meta-analysis represented an extremely valuable contribution to the literature and encouraged further research in this field with several new studies published since then. However, it left the unanswered question about whether the implementation of RDT in association to ASP is superior to the use of ASP alone for the implementation of conventional BC results, and if introducing RDT in a setting where effective ASPs are already in place in association with conventional systems may be beneficial. Furthermore, this meta-analysis classified MALDI-TOF MS as an RDT. Nonetheless, MALDI-TOF MS on isolated colonies growing on solid media is now a standard practice in high-income and many middle-income countries. In these same settings, RDTs may not yet be standard practice and their introduction requires further demonstration of benefits in patient outcome.

Network meta-analysis (NMA) allows to compare multiple interventions in a single analysis by combining both direct and indirect evidence across a network of studies. It yields more precise estimates than a direct comparison and estimates the ranking of interventions [11]. We performed NMA to compare the clinical impact of RDT and conventional BC, both assessed with and without ASP (4 interventions), in patients with BSI. The impact of the different strategies has been assessed with respect to (1) time to optimal treatment (TOT), (2) LOS, and (3) mortality. Our focus is on culture-independent tests; therefore, MALDI-TOF MS has been considered as an RDT when directly applied on BC broth/pellet and as a conventional test when applied on isolates growing on solid media.

METHODS

Our systematic review and meta-analyses are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Network Meta-Analyses [12] (Supplementary Table 1).

Search Strategy

The literature search, built by an experienced librarian, was performed in Medline, Embase, Web of Science, and the Cochrane Library in June 2022, and was updated in November 2023. The search terms included “bloodstream infection,” “rapid diagnostic,” “molecular diagnostic techniques,” “blood culture,” “length of stay,” “mortality,” “time to treatment,” and the brand names of several RDT. No restriction in publication date or language was introduced. Supplementary Table 2 reports the detailed search.

Inclusion Criteria

We included studies comparing any of the 4 following different strategies for the diagnosis of BSI with respect to TOT, LOS, and mortality: (1) BC (in the absence of ASP); (2) BC + ASP; (3) RDT (in the absence of ASP); and (4) RDT + ASP. RDT are defined as culture-independent tests applied on positive BC broth or whole blood for the identification of single or multiple pathogens (±antimicrobial resistance markers). MALDI-TOF MS was also eligible as RDT when applied directly on positive BC (broth/pellet). Conventional methods for the identification of bloodstream pathogens include biochemical BC-based techniques for phenotypic profiling (manual and automated) and the use of MALDI-TOF MS on isolates growing on solid media. Conventional methods for antimicrobial resistance included phenotypic testing, manual (eg, disk diffusion, Etest) or automated (eg, automated broth microdilution), as well as immunochromatographic lateral flow tests (eg, PBP2a detection in Staphylococcus aureus). ASP was defined as any antimicrobial stewardship intervention aimed at actively implementing the RDT or BC results, by means of recommendations from the ASP teams. Our definition of ASP specific to active implementation of RDT or BC results did not include broad stewardship interventions (ie, weekly audit and feedback or preauthorization strategies).

To reduce the risk of confounding by indication associated with observational studies, only randomized controlled trials (RCTs) and quasi-experimental studies were included [13]. Case-control studies were excluded. For studies comparing BC versus BC + ASP or RDT versus RDT + ASP, it was required that the BC methods or RDT used in the 2 groups be the same. For studies comparing BC + ASP versus RDT + ASP, it was required that the ASP methods in the 2 groups be the same.

Studies were excluded if rapid tests were culture dependent (ie, applied on colonies on solid media), if they reported theoretical outcomes, or were performed on non-human subjects. Studies including contaminated BC only were also excluded.

Screening and Selection

Papers identified during the main database search strategy were exported into Endnote [14] and subsequently in Rayyan [15]. Duplicates were deleted by both software. The screening of the papers was performed in Rayyan by 2 independent reviewers (A. M. P., W. L.) who resolved discrepancies upon discussion.

Data Extraction and Outcome Definitions

Once confirmed for inclusion, data were extracted from studies according to the 4 groups forming the network (RDT; RDT + ASP; BC; BC + ASP) (Figure 1). Data extracted included year of publication, the country where the study was performed, study design, patient population, type of RDT, and conventional tests used and outcomes.

TOT was defined as the time from either blood specimen collection or from positive Gram stain/BC positivity (in the case of studies evaluating RDT applied on positive BC broth) to initiation of the antimicrobial treatment targeted on the final isolate. LOS was defined as total hospital LOS or post-BSI LOS. For TOT, when means and standard deviation (SDs) were not available, but the quartiles were available, we estimated the mean by (q1 + median + q3)/3, and the SD by (q3-q1)/1.35. For LOS, medians were used as an estimate of the geometric mean, and geometric SDs [16] were estimated by GSD = exp[(ln(q3) – ln(q1))/1.35]. Otherwise, calculations assumed a lognormal distribution to convert arithmetic means and SDs to geometric means and GSDs. If different summary statistics were reported (eg, median 95% confidence interval [CI]) authors were contacted for further information. If available, infection related-mortality was extracted; otherwise, 30-day and in-hospital mortality were chosen next, respectively. If available, post-BSI LOS was extracted; otherwise, total LOS was used.

Quality Assessment

The latest Cochrane risk-of-bias tool was used to assess the risk of bias in RCTs [17]. The Risk of Bias in non-Randomized Studies of Interventions [18] was used for quasi-experimental studies.

Data Analysis

As well as performing NMA for each outcome, we also performed conventional random-effects meta-analyses to share the summary data we used and visualize the added value of the NMA. For TOT we report mean differences with 95% CIs; for LOS, ratios of geometric means with 95% CI; for mortality, odds ratios (ORs) with 95% CI.

The statistical heterogeneity parameter τ within each conventional meta-analysis and for the NMA was reported and 95% prediction intervals were calculated. A global test of consistency was performed for each NMA [19]. Publication bias was assessed using comparison-adjusted funnel plots [20]. Data analysis was performed using Stata 18 [21] and the following user-written packages: network [19], intervalplot, and netfunnel [22].

RESULTS

Yield of Strategy and Study Characteristics

A total of 9255 records were identified at the primary databases search, of which 227 were screened as full-text. The screening process is summarized in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart (Figure 2). In the analyses, 88 studies were included, representing 25 682 patients [23–110]. Forty-two studies (48%) reported information about TOT, 71 (81%) about LOS (of which 20/71, 28% reported post-BSI LOS), and 76 (86%) about mortality. Seventy-eight studies (89%) had a quasi-experimental design; 10 (11%) were RCTs. Five studies (6%) had more than 2 comparison groups. Twelve (14%) focused on children/newborns, 4 (5%) on intensive care unit patients. Thirty-two studies (36%) included Gram-positive BSI (of which 11 focused exclusively on S. aureus) and 15 (17%) Gram-negative BSI, 7 (8%) focused on candidemia and 34 (39%) included mixed infections. Among studies focusing on RDT (n = 75), 56 (75%) included RDT applied on positive BC broth, 12 (16%) MALDI-TOF MS applied on positive BC broth/pellet, 3 (4%) a combination of the former methods and 4 (5%) RDT applied on whole blood. Among studies including conventional methods (n = 83), 23 (28%) included MALDI-TOF MS applied on isolates grown on solid media. Characteristics of commercial RDT used by the included studies are summarized in Supplementary Table 3. Table 1 summarizes the study characteristics.

Network Characteristics

The most common comparisons assessed by the selected studies were between RDT, either alone or with ASP, and conventional BC (RDT vs BC, n = 28; RDT + ASP vs BC, n = 29), as well as between RDT and BC both embedded with ASP (RDT + ASP vs BC + ASP, n = 18). The comparisons BC + ASP versus BC (n = 14) and RDT + ASP versus RDT (n = 9) were less common. No study compared RDT alone to BC with ASP (Figure 1). Most studies included any hospitalized patients with BSI, ensuring balance in the distribution of the main effect modifiers (transitivity) as patient characteristics across the studies were not expected to vary based on the diagnostic methods used.

Quality Assessment

The study’s quality assessment is reported in Supplementary Tables 4a-b and Supplementary Figure 1. Approximately 30% of quasi-experimental studies had serious risk of bias due to confounding with respect to LOS and mortality, 20% with respect to TOT. All quasi-experimental studies had moderate risk of bias in the selection of reported results as a study protocol was never available. Similarly, most RCTs did not report adhering to a predefined protocol. In contrast, most of the other domains were scored as low risk of bias.

Time to Optimal Treatment

The NMA showed a significant reduction in TOT associated with the use of RDT. The difference was most pronounced when RDT + ASP was compared to BC alone (−29 hours; 95% CI, −35 to −23), whereas it was reduced to −18 hours (95% CI, −27 to −10) when RDT + ASP was compared to BC + ASP, and to −12 hours (95% CI, −20 to −3) when compared to RDT alone. A significant reduction of TOT was also observed when comparing BC + ASP to BC alone (−11 hour; 95% CI, −20 to −1) as well as when comparing RDT to BC in the absence of ASP (−17 hours; 95% CI, −24 to −11). Differently, no significant difference was found in TOT between the use of RDT alone and BC + ASP (−6 hours; 95% CI, −18 to 5). Pooled estimates from conventional and NMA for TOT are shown in Table 2 and Figure 3A and forest plots are shown in Supplementary Figures 2a–2f.

There was appreciable statistical heterogeneity (τ = 13 hours in NMA) with all prediction intervals including 0 except for the RDT + ASP versus BC comparison. There was no suggestion of publication bias (Supplementary Figure 3).

Length of Stay

The NMA showed a reduction in LOS when comparing the use of RDT + ASP to BC alone (0.91; 95% CI, .84–.98), whereas it failed to show any difference in LOS between any of the other groups (Table 2, Figure 3B, Supplementary Figure 4F). Supplementary Figures 4A–F show forest plots from conventional and NMA. Findings were similar for the 27 studies including Gram-positive BSI only (data not shown). No difference in LOS between any of the groups was found when considering studies including Gram-negative BSI only (n = 15) or post-BSI LOS only (n = 20) (data not shown). There was appreciable statistical heterogeneity (τ = 0.19 in NMA) with all prediction intervals including 1. There was no suggestion of publication bias (Supplementary Figure 5).

Mortality

The NMA estimated a decreased mortality when RDT are used with ASP compared to conventional systems alone (OR, 0.72; 95% CI, .59–.87). Interestingly, the NMA also showed a reduced mortality associated with the use of RDT versus conventional systems when both are embedded within ASP (RDT + ASP vs BC + ASP; OR, 0.78; 95% CI, .63–.96), whereas no difference in mortality was observed between RDT and conventional systems in the absence of ASP (RDT vs BC; OR, 0.91; 95% CI, .72–1.13). The NMA also failed to show any differences in mortality when comparing RDT versus BC + ASP (OR, 0.98; 95% CI, .72–1.34), nor when comparing BC + ASP versus BC (OR, 0.92; 95% CI, .73–1.16) or RDT + ASP versus RDT (OR, 0.79; 95% CI, .61–1.03). Pooled estimates from conventional and NMA for mortality are reported in Table 2 and Figure 3C and forest plots in Supplementary Figures 6A–6F.

There was appreciable statistical heterogeneity (τ = 0.33 in NMA) and no suggestion of publication bias (Supplementary Figure 7).

When including RCTs only (n = 10), NMA failed to show any statistically significant results for the 3 outcomes when comparing any of the groups. The estimates resulting from NMA restricted to the 10 RCTs were associated with a large amount of imprecision (data not shown).

DISCUSSION

Our NMA confirmed what was expected according to previous evidence showing a decreased mortality associated with the use of RDT in combination with ASP compared to BC alone [8]. Importantly, the NMA also showed a survival benefit associated with RDT versus conventional systems when both are embedded within ASP. This is a novel finding as the relative contribution of RDT and ASP on improved survival was unknown, not having been assessed by previous meta-analyses. Timbrook et al acknowledge that most of the RDT + ASP studies included in their meta-analysis did not state whether ASP was also present in the conventional BC comparator group, and they could not perform any analysis to compare these 2 specific interventions (RDT + ASP vs BC + ASP) [10]. In contrast, we were able to include 18 studies assessing this comparison, with the NMA adding further evidence based on indirect comparisons. This observation has important clinical implications because it suggests how even centers with efficient ASP in place to implement conventional BC results may benefit from the introduction of RDT, whereas the impact of ASP alone on survival seems more limited. Nonetheless, several challenges remain with respect to clinical implementation in real-life scenarios as the most efficient ASP and RDT have not yet been defined and may not necessarily be universal to all institutions.

Another novel finding of our work is the impact of RDT + ASP versus conventional BC on TOT, whereas the previous meta-analysis had focused on time to effective therapy instead [10]. Transitioning from a broad-spectrum early effective therapy to a narrow-spectrum optimal/targeted treatment is sometimes delayed in clinical practice, despite microbiological results being available. Although time to effective therapy has been more widely associated with increased survival compared to TOT [1–3], shortening TOT is advisable to reduce the exposure to broad-spectrum antibiotics, possibly reducing antimicrobial resistance, improve safety, and reduce costs [111–113]. Early deescalation might also positively affect LOS [114].

We showed an overall limited impact of RDT or ASP on LOS. This is relevant not only from a clinical point of view but also when assessing costs associated with the implementation of RDT and ASP because the increased expenses derived from establishing such interventions may not always be balanced by a shorter patient LOS.

Strengths of our work include NMA, which allowed us to evaluate the use of RDT and BC both with and without ASP, as well as the inclusion of studies with more than 2 comparison groups. Another strength is the high number of studies included, for a total of 25 682 patients—more than 4 times those of Timbrook et al [10].

Limitations include the high heterogeneity among studies, likely reflecting the use of different tests, different study settings (both in terms of epidemiology and hospital facilities), different provider attitudes toward implementation of tests results and antimicrobial prescription, and different types of ASP and laboratory practices in place. Different definitions of TOT and LOS among studies could have also increased heterogeneity. It is possible that the validity of our results could be restricted to some specific settings or type of RDT or ASP only. In a quasi-experimental study involving 130 hospitals in the United States, Britt et al have shown how some specific component of the ASP (such as infectious diseases consultation or the frequency of patient review at days 1–2) had the most significant impact on early antimicrobial deescalation coupled with BioFire FilmArray blood culture identification panel implementation [115]. Therefore, studies evaluating the impact of specific RDT/ASP in specific settings may be warranted to define and support local practices. A second limitation refers to our classification of MALDI-TOF MS as an RDT when applied to BC broth/pellet and as a conventional system when applied on isolates growing on solid media. We chose this classification to focus on the impact of culture independent testing. Yet, we acknowledge that the application of MALDI-TOF MS on early subcultures may have a turnaround time closer to molecular testing applied of positive BC broth rather than to conventional culture methods, and that overall heterogeneity was present in the conventional BC group because of the inclusion of both MS and traditional phenotypic testing. The validity of our results is also limited by the risk of confounding of some of the quasi-experimental studies, which is however intrinsically due to their lack of randomization.

In conclusion, our NMA suggests how the implementation of RDT + ASP may confer a survival benefit even in institutions already implementing conventional culture results through effective ASP, overall supporting the recommendation of the Infectious Diseases Society of America to use RDT within ASP for the management of BSI [116].

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Author Contributions. A. M. P. designed the study and drafted the original version of the manuscript, A. M. P. and W. L. performed the screening of the papers, M. D. C. performed statistical analysis, and all authors significantly contributed to revising the manuscript for important intellectual content and approved its last version.

Acknowledgments. The authors thank Lars Eriksson for the support with building the literature search.

Financial support. No funding was received for writing this paper. A. M. P. is receiving a scholarship from the University of Queensland in support of her PhD candidature.

References

Author notes