Clinical Impact of Metagenomic Next-Generation Sequencing of Plasma Cell-Free DNA for the Diagnosis of Infectious Diseases: A Multicenter Retrospective Cohort Study Free

Abstract

Background

Metagenomic next-generation sequencing (mNGS) of plasma cell-free DNA has emerged as an attractive diagnostic modality allowing broad-range pathogen detection, noninvasive sampling, and earlier diagnosis. However, little is known about its real-world clinical impact as used in routine practice.

Methods

We performed a retrospective cohort study of all patients for whom plasma mNGS (Karius test) was performed for all indications at 5 United States institutions over 1.5 years. Comprehensive records review was performed, and standardized assessment of clinical impact of the mNGS based on the treating team’s interpretation of Karius results and patient management was established.

Results

A total of 82 Karius tests were evaluated from 39 (47.6%) adults and 43 (52.4%) children and a total of 53 (64.6%) immunocompromised patients. Karius positivity rate was 50 of 82 (61.0%), with 25 (50.0%) showing 2 or more organisms (range, 2–8). The Karius test results led to positive impact in 6 (7.3%), negative impact in 3 (3.7%), and no impact in 71 (86.6%), and was indeterminate in 2 (2.4%). Cases with positive Karius result and clinical impact involved bacteria and/or fungi but not DNA viruses or parasites. In 10 patients who underwent 16 additional repeated tests, only 1 was associated with clinical impact.

Conclusions

The real-world impact of the Karius test as currently used in routine clinical practice is limited. Further studies are needed to identify high-yield patient populations, define the complementary role of mNGS to conventional microbiological methods, and discern how best to integrate mNGS into current testing algorithms.

(See the Editorial Commentary by Babady on pages 246–8.)

Metagenomic next-generation sequencing (mNGS) of pathogen nucleic acid in clinical samples has emerged as a promising 1-test approach for hypothesis-free diagnosis of potentially all infectious etiologies. mNGS is currently orderable in select reference laboratories from plasma cell-free DNA (Karius, Redwood City, California), from DNA and RNA in cerebrospinal fluid (CSF) (University of California, San Francisco), and from DNA and RNA in respiratory secretions (IDbyDNA, San Francisco, California) [1]. Of these, the Karius test has been commercially available the longest and is likely the most common mNGS send-out test in institutions in the United States (US). This assay is reported to detect and quantify pathogen cell-free DNA from 1250 bacteria, DNA viruses, fungi, and eukaryotic parasites [2], and has been employed for the diagnosis of invasive fungal disease [3, 4], community-acquired pneumonia [5], and infections in immunocompromised hosts [6, 7].

The availability of clinical metagenomic cell-free DNA sequencing has generated interest from clinical providers across disciplines given the noninvasive nature of testing, and the potential to pose an actionable diagnosis faster than through conventional microbiologic methods. However, little is known about the clinical impact of mNGS testing when used for routine patient care. A study assessing the clinical impact of mNGS from CSF in patients with suspected meningitis and encephalitis showed positive clinical impact in 8 of 204 (3.9%) patients tested [8]. In the present multicenter study, our aim was to comprehensively examine the real-world clinical impact of the Karius test ordered for all indications for the diagnosis of infectious diseases.

METHODS

Ethical Considerations

This study was approved by the institutional review boards (IRB) at Stanford University and Columbia University Medical Center (CUMC). IRB approval was waived for this project for University of California, Los Angeles (UCLA), the University of Utah (UT), and Children’s Hospital of Los Angeles (CHLA).

Karius Test

The Karius laboratory started to offer commercial testing in December 2016. Plasma from whole blood collected in a K2–ethylenediaminetetraacetic acid or BD Vacutainer PPT tube (Becton Dickinson, Franklin Lakes, New Jersey) and separated < 6 hours after draw was sent either fresh (within 4 days of blood draw) or frozen. All testing was performed by Karius per routine testing protocol as previously described [2].

Study Design

We performed a retrospective cohort study of patients for all consecutive plasma mNGS samples sent out for Karius test from 5 US institutions (CHLA, CUMC, Stanford Health Care [SHC], UCLA, and UT) from 1 August 2017 to 31 May 2019. Comprehensive records review was performed by each site investigator for respective patients, and clinical impact solely based on the treating team’s interpretation of Karius results and their subsequent management decisions was assessed according to predefined objective grading criteria (Table 1) as to whether Karius test result had a positive, negative, or no clinical impact, or if impact was indeterminate. This study intended to assess the real-world impact of Karius test and therefore only the treating team’s decisions and actions were considered for impact assessment, and no retrospective clinical consensus assessment of impact and adjudication by a panel of experts was performed. Given that Karius testing may be reordered in the same patient for different reasons including patient monitoring, the main analysis was performed on the first Karius test per patient. A separate analysis was performed on repeat tests.

Statistical Analysis

Statistical analysis was performed with Stata 15 (Stata Corp, College Station, Texas). Due to the descriptive nature of the review, a formal sample size calculation was not performed.

RESULTS

A total of 82 Karius tests were performed in unique patients cared for at UCLA (n = 50 tests), SHC (n = 21), CUMC (n = 5), UT (n = 3), and CHLA (n = 3) (Table 2). There were 39 (47.6%) adults, 43 (52.4%) children, 52 (63.4%) males, and 53 (64.6%) immunocompromised patients, most commonly solid organ transplant (17.1%) and primary immunodeficiency (13.4%) (Table 2). The mean turnaround time from sample collection to result was 3.1 days (standard deviation [SD], 1.2), and time from Karius receipt of sample to result was 1.5 days (SD, 0.9) for a subset of 20 samples. The most common ordering providers were infectious disease physicians (74.4%), followed by oncologists (11.0%). The main indication for testing was fever of unknown origin (23.2%), suspected respiratory infection (13.4%), sepsis (9.8%), suspected endocarditis (8.5%) and febrile neutropenia (7.3%). The Karius positivity rate was 50 of 82 (61.0%), with 25 (50.0%) of these tests resulted as monomicrobial (17 bacterial, 5 viral, and 3 fungal), and 25 (50.0%) showing 2 or more organisms (range, 2–8). There was no parasite detected in this cohort.

The Karius test results led to positive or negative clinical impact in 9 (11.0%) patients (Table 3). The impact was positive in 6 (7.3%) and negative in 3 (3.7%) (Table 3). Of these 9 Karius tests, 8 had organisms detected while 1 had no organism detected. Impact was none in 71 (86.6%) tests and indeterminate in 2 (2.4%). All 8 cases with positive Karius result and clinical impact involved bacteria and/or fungi; determination of the presence or absence of DNA viruses by plasma mNGS demonstrated no clinical impact in this study. Patients with a positive or negative Karius result where there was no (n = 71) or indeterminate (n = 2) clinical impact are presented in Supplementary Tables 1 and 2. Positive clinical impact was categorized as de-escalation of therapy (n = 1), earlier diagnosis than conventional methods and initiation of appropriate therapy (n = 2), new diagnosis not made by conventional methods and escalation of therapy (n = 1), and new diagnosis and de-escalation (n = 2) (Table 3). The 3 new diagnoses made by Karius test included presumptive diagnosis of an abdominal aorta mycotic aneurysm caused by Enterococcus faecalis and Prevotella melaninogenica, a diagnosis of Rhizopus oryzae confirming disseminated mucormycosis, and identification of mixed oral flora in a case of blood culture–negative endocarditis, facilitating de-escalation of therapy. The 3 cases associated with negative clinical impact resulted in additional, unnecessary diagnostic investigations (n = 2), and unnecessary treatment (n = 1). Among positive Karius test results, lack of clinical impact was most commonly due to identification of a new organism that was not acted upon (n = 20), confirmation of conventional microbiological result that was not acted upon (n = 14), or both (n = 6) (Supplementary Table 1).

All patients in this cohort underwent both conventional testing and Karius test send out in the order decided at the discretion of the treating team. Twenty-seven of 82 (32.9%) patients underwent Karius testing with a preestablished microbiological diagnosis through conventional testing, with relative proportions varying by institution from 0 to 60%. The most common conventional diagnostic method establishing a microbiological diagnosis in this group was blood culture (n = 9), tissue bacterial culture (n = 7), and viral blood-based polymerase chain reaction (n = 4). Of the 27 patients with a positive conventional diagnosis, 16 (59.3%) showed the same organism detected by the Karius test, with 7 of these showing other organisms in addition to the one that was reproducibly detected (Supplementary Tables 1 and 2). The remaining 11 cases where preceding conventional microbiological testing was positive but Karius testing was negative included bacteria (Staphylococcus aureus [n = 1], Streptococcus viridans [n = 2], Mycobacterium haemophilum [n = 1], Mycobacterium chelonae [n = 1], Mycobacterium bovis [n = 1]), DNA virus (adenovirus [n = 1]), and fungi (Candida parapsilosis [n = 1], Candida lusitaniae [n = 1], Coccidioides immitis/posadasii [n = 1], Mucorales [n = 1]).

A subset of 10 patients underwent repeat testing, for a total of 16 additional repeated tests from UCLA (n = 14), SHC (n = 1), and CUMC (n = 1). Review of the 16 repeat tests revealed 15 positive cases, of which 1 case was associated with a positive clinical impact, where the repeat Karius test was performed 3 months after the initial one. This 2-year-old boy with a history of heart transplant presented with positive blood and respiratory cultures for Stenotrophomonas maltophilia. Four days after these conventional tests were performed, Karius test was sent out and resulted 2 days later as positive for S. maltophilia. The absence of any other detected organism on the Karius test prompted the treating team to de-escalate therapy.

Discussion

Metagenomic next-generation sequencing has stimulated much interest among providers across medical subspecialties given its potential for broad-range pathogen detection. Furthermore, the noninvasive nature of plasma samples makes mNGS an attractive diagnostic modality for deep-seated infections that currently require invasive biopsy. However, the value of mNGS testing as a diagnostic tool is poorly understood. In this multicenter retrospective cohort study, we show that, as currently used in clinical practice, the real-world clinical impact of the Karius test is limited given that it resulted in no or negative impact in the majority (92.7%) of patients. This finding was consistent with that reported for mNGS on CSF where 196 of 204 (96.1%) tests did not have a clinical impact [8]. As it currently stands, in most academic and private health systems, microbiology and virology diagnostic laboratories offer a comprehensive menu of culture-based and molecular tests such that only few, typically complex, cases with infectious etiology will be left without a diagnosis. Important questions to consider now are whether mNGS has the diagnostic accuracy required to complement currently available diagnostics, and how best to integrate mNGS into current testing algorithms. Published studies indicate that CSF and plasma mNGS assays lack sensitivity compared to conventional methods and therefore cannot be used as stand-alone or rule-out tests [2, 8]. Thus, whether reserving mNGS as last resort for cases for which conventional microbiological testing has failed to provide an answer, or to include mNGS more proximally in testing in parallel to conventional tests, is currently an active subject of discussion in the microbiology field that needs to be determined empirically [9]. The decision to include mNGS testing should take into account the differential diagnosis, as well as the availability, accuracy, and turnaround time of conventional methods as well as cost-effectiveness of mNGS. The specific example of lack of clinical impact of positive Karius results for DNA viruses in this cohort provides a compelling case for routine highly available, highly accurate molecular virology assays providing sufficient and actionable diagnostic information in all cases.

Plasma cell-free DNA may originate from the site of infection or colonization. Although Karius reports quantitative units, there is no threshold that distinguishes colonization from infection, and interpretation of results may therefore be clinically challenging, especially in immunocompromised hosts and patients with disrupted intestinal barrier. Clinical validation data recently reported on the Karius test from patients presenting to the emergency department at a tertiary hospital with suspected sepsis have demonstrated a positive percentage agreement of 84.8% and negative percentage agreement of 48.2% compared to all microbiological reference testing [2]. The finding that only 6 of 50 (12.0%) positive test results were positively impactful in the current study is consistent with an assay specificity that is insufficient to enable clinical impact when organisms are detected under routine practice. Even in certain cases where care was tailored to the positive Karius result, it is unclear if the reported organisms were the underlying drivers of pathology. For example, the patient with abdominal aorta mycotic aneurysm (Table 3, study identifier SHC-11) in whom antibiotic therapy was de-escalated to chronic suppressive therapy with amoxicillin-clavulanic acid based on a positive Karius result with E. faecalis and P. melaninogenica presented with multiple clinical recurrences of back pain and eventually grew Mycobacterium avium complex from a vertebral tissue biopsy 9 months later.

Patient selection practices may also have contributed to low clinical impact as almost one-third (ranging from 0 to 60% by institution) of patients had already received a microbiological diagnosis prior to testing. The only case in this cohort to have demonstrated clinical impact in the setting of a preestablished microbiological diagnosis is that of the repeat Karius test identifying S. maltophilia only in a patient with known bacteremia with the same organism, leading to de-escalation of therapy. This highlights the important role of diagnostic stewardship in optimizing testing algorithms and preventing unnecessary testing in cases where a microbiological diagnosis has already been established. In this study, preapproval practices varied from one institution to the next with no center requiring infectious diseases (ID) preapproval at the beginning of the study. Policies changed over time with 2 centers (SHC and UCLA) requiring preapproval by microbiology or ID in the later phase of this study. In situations where positive conventional test results precede the mNGS result, patient management (ie, escalation or de-escalation of therapy) should be considered based on conventional microbiological diagnostics rather than waiting for confirmatory mNGS result. Microbiology laboratory or ID preapproval of mNGS send-out requests represents an important intervention component to reduce unnecessary testing in such cases.

The finding that 6 of 82 (7.3%) of Karius test results in this study led to positive clinical impact is in contrast to that recently reported in a single pediatric center in Chicago where 56 of 100 (56.0%) test results were considered clinically actionable [10]. This discrepancy may have been driven by differences in patient selection, in addition to differences in outcome assessment methodology. For instance, in our study, clinical impact was solely based on the treating team’s interpretation of Karius results and their subsequent management decisions; in contrast, in the above study, clinical adjudication was performed where “clinical relevance” was unclear from the medical record. Inherent interprovider interpretation differences of whether Karius results represent a clinically significant pathogen or colonization also likely contributed to findings, including the fact that not all results were reviewed by ID. Other differences between studies included variability in testing indication, proportion of ordering ID providers, and proportion of patients with a known microbiological diagnosis pre–Karius testing. The low clinical impact proportion in the present study was closer to that reported for mNGS on CSF where 8 of 204 (3.9%) of tests led to positive clinical impact [8].

This study was strengthened by its multicenter design including 5 adult and pediatric sites, and by its assessment of real-world practice in presenting data from provider-initiated testing. However, there are limitations. First, although we made significant effort to standardize the definition and application of clinical impact criteria, some of the cases included were very complex medically and interpretation of clinical impact may have been challenging to ascertain. The definition of impact was based on how providing physicians interpreted and reacted to test results. This may have over- or underestimated the real impact of the Karius test result. Standardized definitions of impact across studies, in addition to more comprehensive and objective measures of impact (eg, including antibiotic use, hospital length of stay, laboratory testing utilization) are needed to accurately quantify and compare clinical impact. Second, approval criteria for send-out testing varied by institution and time period such that patient populations may have varied in their pretest probability, which may have influenced the assessment of clinical impact. Finally, given that this was a convenience sample, not all patient populations are represented equally. For example, only 1 person living with human immunodeficiency virus type 1 infection was included, which limits generalizability to this patient population.

In conclusion, plasma mNGS represents an attractive diagnostic modality due to its noninvasive nature and potential to provide an early and actionable diagnosis. However, this study has shown that the real-world impact of the Karius test as currently used in clinical practice is limited given that this testing resulted in negative impact or no impact in addition to conventional results in the majority (92.7%) of patients. Further studies are needed to better understand which patient populations may potentially benefit, define the complementary role of mNGS to conventional microbiological methods, and improve diagnostic interpretation algorithms. Furthermore, institutions should adopt an active diagnostic stewardship role to ensure optimal use of mNGS.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Financial support. This work was supported in part by the Canadian Institutes of Health Research (grant number 201711HIV-398347-295229 to C. A. H.).

Potential conflicts of interest. N. B. has received research funding from Karius during 2015–2016. All other authors report no potential conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References