Impact of Changes of the 2020 Consensus Definitions of Invasive Aspergillosis on Clinical Trial Design: Unintended Consequences for Prevention Trials?

Abstract

Background

Consensus definitions for the diagnosis of invasive fungal diseases (IFDs) were updated in 2020 to increase the certainty of IFD for inclusion in clinical trials, for instance by increasing biomarker cutoff limits to define positivity. To date, there is a paucity of data as to the impact of the revised definitions on clinical trials.

Methods

In this study, we sought to determine the impact of the new definitions on classifying invasive aspergillosis (IA), the most common invasive mold disease in immunocompromised patients. We reclassified 226 proven and probable IA cases plus 139 possible IFD cases in the Aspergillus Technology Consortium (AsTeC) and in an antifungal prophylaxis trial (BMT CTN 0101) using the new criteria.

Results

Fewer cases met the more stringent diagnostic 2020 criteria after applying the reclassification criteria to define probable IA. Of 188 evaluable probable cases, 41 (22%) were reclassified to 40 possible IA and 1 probable IFD. Reclassification to possible IFD occurred in 22% of hematologic malignancy (HM) patients, 29% of hematopoietic cell transplant (HCT) patients, and in no lung transplant (LT) patients. Date of diagnosis was established a median (range) of 3 (1–105) days later in 15% of probable IA cases using the new criteria. Applying the new definitions to the BMT CTN 0101 trial, the power to detect the same odds ratio decreased substantially.

Conclusions

The updated IA consensus definitions may impact future trial designs, especially for antifungal prophylaxis studies.

Invasive fungal diseases (IFDs), especially invasive aspergillosis (IA), are among the most serious infections confronting immunocompromised patients, with high rates of morbidity and mortality. The diagnosis of IA remains challenging. Historically, invasive procedures to establish tissue diagnosis have been necessary, but biopsies of infected sites are fraught with the risk of serious complications and are insensitive [1]. Today, fungal biomarker blood assays are commonly used to establish the diagnosis [2].

Clinical trials to evaluate antifungal therapy have been plagued by uncertainties as to whether participants truly have the infection being treated. Such uncertainty has hampered the ability to establish guidelines for the best therapy to optimize treatment outcomes. For more than 2 decades, the European Organization for Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) has established standardized criteria that can guide clinical research trials [3–5]. With scientific advances, diagnostic criteria have evolved over time. The 2002 EORTC/MSG diagnostic criteria were revised in 2008 and more recently in 2020 [4, 5].

There are 2 substantive changes in the mycologic criteria for IA in the 2020 EORTC/MSG update: Higher thresholds of galactomannan (GM) positivity are required to establish probable IA, and, for the first time, Aspergillus polymerase chain reaction (PCR) assays are accepted as a mycologic criterion. The GM index threshold increased from the regulatory cleared positivity of 0.5–1.0 for a single serum or BAL specimen or a serum index of 0.7 plus a BAL sample index of ≥0.8. The GM assay is widely used worldwide, while some centers, particularly in Europe, rely on PCR assays. PCR assays are not widely used in the United States and other countries. This study was undertaken to determine the impact of the change in GM positivity of the new definitions on classifying IA.

METHODS

Objectives

We set out to determine the impact of the revised definitions on (1) the reclassification of cases, (2) effect on different key patient groups, (3) the date of diagnosis and its effect on survival rates, and (4) the magnitude of the effect on IA rates in antifungal prophylaxis trials.

Ethical Approvals and Patient Consent

This study describes secondary analyses of earlier studies that were conducted under the auspices of the University of Florida Institutional Review Board (IRB) and in accordance with the ethical standards of the Helsinki Declaration. The study was approved by local ethical committees at participating sites. All patients (or their designee) signed written informed consents, and patient information has been anonymized.

IA Cases

Cases were drawn from 2 data sets of patients treated in the United States: the National Institute of Allergy and Infectious Disease (NIAID)–funded Aspergillus Technology Consortium (AsTeC) and the National Heart, Lung, and Blood Institute and National Cancer Institute (NHLBI/NCI)–funded Blood and Marrow Transplant Clinical Trials Network (BMT CTN) 0101 clinical trial.

AsTeC Biorepository

Patients were scored as having no, possible, probable, or proven IFD using the EORTC/MSG 2008 diagnostic criteria [4]. A total of 872 patients were enrolled (282 HM, 432 HCT, 132 LT, and 26 other diseases) between 2008 and 2013. A total of 188 patients developed IA (27 proven and 161 probable) (Supplementary Table 1), 89 patients developed other types of IFD (65 proven and 24 probable), and 89 patients developed possible IFD. Illustrative publications [6–13] describe the study strategy, patient selection, data collection, sample procedures, and several uses of these specimens.

BMT CTN 0101Trial

This was a randomized, double-blind, multicenter study comparing fluconazole with voriconazole for the prevention of IFD in allogeneic HCT recipients enrolled at 35 US centers participating in the BMT CTN 0101 trial (NCT00075803) [14]. A total of 600 patients were randomized (voriconazole, n = 305; fluconazole, n = 295) from 2003 to 2006. Infections were scored using the 2002 MSG/EORTC criteria, with an additional criterion of “presumptive” IFD, which was defined as meeting the host and clinical criteria (including radiographic abnormalities) but lacking mycologic confirmation and with no alternative etiologies identified by extensive evaluation including bronchoscopy [3]. At 1 year post-HCT, 79 patients had developed IFDs (28 proven, 33 probable, and 18 presumptive IFDs). Aspergillus spp. accounted for 38 (48%) IFDs at 365 days. In addition, there were 75 patients with possible IFDs (Supplementary Table 1). For this study, the 18 presumptive IFD cases were placed in the possible IFD category as per EORTC/MSG criteria.

Definitions

The day of diagnosis was defined as the first day that all components of the diagnostic criteria were met. Survival from IFD was calculated at 6 and 12 weeks after diagnosis. In the BMT CTN 0101 trial, a GM optical density index value ≥0.5 in serum or BAL was considered positive for the GM index. Although the 2002 criteria stipulated the need for GM to be positive in ≥2 blood samples to eliminate false-positive results, subsequent studies have shown that false positivity could be eliminated by repeating the assay on the same blood sample (Wheat, personal communication); accordingly, samples that were repeat positive were used to satisfy this GM criterion for all AsTeC and BMT CTN 0101 samples rather than requiring collection of a second sample: 15 of 51 cases in AsTeC and 8 of 19 cases in BMT CTN 0101 met the criterion on the basis of repeat positive testing of the same sample rather than 2 separate samples. The 2002 definitions [3] allowed host factors plus either mycologic or clinical factors to satisfy possible IFD, but in the 2008 definitions [4], possible IFD required both host and clinical factors to be present in the absence of mycologic criteria. Thus, the number of possible IFD cases in the BMT CTN 0101 was reduced from 75 to 50 using the 2008 definitions for this study. Supplementary Table 3 summarizes the substantive changes in host, clinical, and mycologic criteria in IA definitions between 2002, 2008, and 2020.

Study Procedures

The cases were scored for IFDs using the 2008 and 2020 sets of MSG/EORTC definitions [4, 5]. In the BMT CTN 0101 trial, 8 cases were reported as having a positive GM (≥0.5) without quantification (7 serum and 1 BAL). We retested 4 available samples drawn within 3 days of the original sample for reclassification, but 4 cases were not able to be reclassified and were excluded from the analysis.

BMT CTN 0101 Analysis

The original 0101 trial targeted at least 80% power to detect an increase in fungal-free survival (FFS) at 6 months from 50% to 62% (a 12% absolute increase or delta, or equivalently an odds ratio [OR] of 1.632). The trial results (2002 definition) indicated FFS at 6 months, which was much higher (77% overall). The revised 2020 definitions indicated an FFS rate at 6 months of 79% overall. We recomputed power to detect a range of delta (absolute differences in 6-month FFS) for 3 scenarios of different baseline 6-month FFS rates: original trial assumption (50%), 2002 definition (77%), and 2020 definition (79%).

RESULTS

Our study population consisted of 226 proven and probable IA cases in the AsTeC and BMT CTN 0101 trial (Supplementary Table 1). Most of the cases were in the probable IA category. Supplementary Table 2 provides the characteristics of the patients with these infections.

Tables 1 and 2 demonstrate that fewer cases meeting the EORTC/MSG 2008 criteria met the more stringent 2020 diagnostic criteria. As shown in Figure 1, no cases with proven IA changed. However, 41 of the 188 (22%) evaluable probable cases were reclassified (40 possible IA, 1 probable IFD). Using 2008 criteria, 108 of 161 (67%) cases in AsTeC and 22 of 31 (71%) cases in the BMT CTN 0101 trial met probable IA classification based on serum and/or BAL GM EIA tests that were positive at the ≥0.5 index. However, under the revised higher limits in the new 2020 definitions, 32 of the 108 (30%) AsTeC cases and 9 of the 31 (29%) BMT CTN 0101 cases no longer met probable IA criteria. Of the 32 reclassified AsTeC cases, 3 possible IFD cases also had concomitant proven IFDs (2 candidiasis and 1 mucormycosis). Those 3 cases were included within the “reclassified to possible IFD” group for this analysis due to the focus on IA. Among BMT CTN 0101 cases, 9 of 27 (33%) evaluable probable IA cases were reclassified; 8 cases were reclassified to possible IFD, and 1 case was reclassified to probable IFD. As previously noted, 4 cases were not able to be evaluated for reclassification.

The effect of reclassification was different in various patient groups: the percentage of probable IA cases reclassified to possible IFD was greatest in HM and HCT patients (22 and 29%, respectively), and there was no effect in LT patients (Tables 1 and 2). The type of antifungal prophylaxis was examined to determine if it influenced the likelihood of reclassification of cases. In the BMT CTN 0101 trial, at 6 months, 4 of 14 (28%) probable IA cases in the fluconazole arm compared with 3 of 5 (60%) cases in the voriconazole arm (P = NS) required reclassification (Tables 3 and 4). For the entire 1-year follow-up (with most patients having completed antifungal prophylaxis at day 100), the difference was comparable with 5 of 16 (31%) in the fluconazole arm vs 4 of 11 (36%) in the voriconazole arm (P = NS) being reclassified.

The date the EORTC/MSG diagnostic criteria were satisfied for probable IA using the 2020 definitions was delayed compared with using 2008 definitions (median delay [range], 3 [1–105] days) in 22 of 147 (15%) evaluable patients (Supplementary Figure 1). Combining these 22 cases in which the diagnosis was delayed with the 40 cases (Tables 1 and 2) that would have been treated for IFD using regulatory cleared biomarker positivity but which never met the research end point, 62 of 188 (33%) probable IA patients based on 2008 criteria would have had antifungal treatment delayed or not started if the 2020 criteria had been used to initiate antifungal treatment.

The survival rates of probable IA were 66% and 65% at 6 weeks and 42% and 43% at 12 weeks, respectively, for cases scored using the 2008 criteria and retained as probable IA using the 2020 criteria (Supplementary Table 4). Survival rates of probable IA nonreclassified cases did not significantly differ from those reclassified to possible IFD at 6 weeks (65% vs 75%; P = .82) or 12 weeks (43% vs 46%; P = NS).

The original 0101 trial targeted at least 80% power to detect an increase in FFS at 6 months from 50% to 62% (a 12% absolute increase or delta, or equivalently an OR of 1.632). The trial results (2002 definition) indicated FFS rates at 6 months that were much higher (77% overall). The revised 2020 definitions indicated FFS at 6 months of 79% overall. We recomputed power to detect a range of delta (absolute differences in 6-month FFS) for 3 scenarios of different baseline 6-month FFS: original trial assumption (50%), 2002 definition (77%), and 2020 definition (79%) (Table 5). Note that power increases with baseline FFS further away from 50%. However, this is power to detect the same absolute difference in FFS. This comes with a caveat that targeting a particular absolute difference in FFS when FFS is 50% is much different than targeting the same absolute difference in FFS when FFS is 77%. The OR for 6-month FFS was 1.632 for the original 0101 trial design (corresponding to an FFS of 62% vs 50%). Using the same OR, the power to detect the same OR when the baseline FFS shifts to 77% or 79% decreases to 65% and 62%, respectively.

DISCUSSION

Defining IFD is difficult and continually evolving, prompting definitions to be established and updated by consensus groups, such as the EORTC/MSG. These definitions are intended principally for antifungal treatment trials to ensure that individuals enrolled in clinical trials truly have IFD and to standardize reporting and enable cross-trial comparisons of outcomes. The new definitions will have important implications for antifungal trials, as setting goalposts in mycologic criteria more conservatively leads to people with higher fungal burden. Also, prophylaxis trials stress sensitivity/specificity characteristics of testing differently than testing to confirm a diagnosis.

One implication is that, as expected, fewer patients with probable IA meet the more stringent EORTC/MSG 2020 criteria. The 22% and 29% reduction in probable IA cases in HM and HCT patients, respectively, who meet the new definitions will necessitate screening more candidates for antifungal treatment trials in HM and HCT patients to achieve comparable absolute risk reductions of IFDs. In contrast, there was no change in the LT cohort as all subjects (n = 15) recovered Aspergillus spp. from culture. The reasons for this are likely related to the lack of routine use of the GM assay in solid organ transplant patients owing to its reduced sensitivity in serum and specificity in BAL compared with its performance in HM populations [15].

A second implication is the divergence between the GM assay’s regulatory cleared threshold for GM positivity and the new research criteria, which is especially problematic for antifungal prophylaxis trials. Most clinicians will not delay start of antifungal therapy for suspected IFD in order to wait for the more stringent 2020 GM (index = 1.0) diagnostic criteria to be met because accepted practice uses the GM assay’s licensed threshold (index = 0.5). The frequent use of “empiric” antifungal therapy for “possible” IA will also need to be taken into consideration in the trial design. As noted, 62 of 188 (33%) patients with probable IFD based on the regulatory approved GM index criterion would have had treatment delayed or not given if the research criteria had been used to initiate antifungal treatment.

A third implication is how to evaluate possible IFD cases. Possible IA cases are typically not included in antifungal treatment trials, due to diagnostic uncertainty. In the past, most antifungal prophylaxis trials excluded possible IFD cases in the primary end point. This will need to be rethought as many possible IA cases that meet the regulatory GM criteria but not the 2020 criteria will be treated with empiric antifungal therapy, and such cases treated empirically will confound the study results.

There are several ways to address this quandary. One would be to embrace inclusion of possible IFD cases into the study primary end point. This, of course, has the downside of increasing the uncertainty of how many true IFD cases one is actually preventing. A second would be to adopt the more stringent criteria but to increase sample size to accommodate the lower event rate for probable or proven IFDs. This would still not address the confounding effect of all the empiric antifungal therapy courses given by clinicians for positive GM tests using the regulatory criteria of positivity in accordance with accepted clinical practice and necessitate a substantially higher number of patient enrollments (and increase the cost of the trial). A third would be to retain the 2008 criteria (which align with the regularly cleared threshold of the GM assay) for prophylaxis trials. A fourth solution is to incorporate the use of empiric antifungal therapy into the end point. This has been done in the past, although it was also criticized due to the variability of clinician decision-making.

Aspergillus PCR assays were not used in the diagnostic assessment of these cases; it is possible that some of the cases reclassified from the 2008 definitions might have continued to meet the mycologic criteria for probable IA if PCR assays had been used. Enormous strides in identifying components of the assay that affect the performance and standardization of these components have improved reliability [16–19]; still, numerous centers continue to use in-house assays. Unfortunately, at present no commercial Aspergillus PCR has been approved in the United States, and usage is patchy worldwide. If the PCR assay is used in a clinical trial, consistency of use across all participating centers would be highly desirable.

A fourth implication is that it is possible, perhaps likely, that cases meeting the more stringent criteria are either more aggressive infections or more extensive infections that might be less responsive to antifungal therapy. Survival of patients with IA is generally assessed at 6 or 12 weeks in antifungal trials [20]. Although we did not find a survival difference between patients with probable IA scored by the 2008 criteria and meeting the more stringent 2020 criteria in survival at 6 or 12 weeks in this analysis, a difference could have been obscured as all patients were treated using the 2008 criteria. A difference, if there is one, could have implications for response estimates in trials that include possible IFD with probable and proven IFD.

A limitation of this analysis is that these cases were reclassified >10 years ago. It is possible that changes in transplant practice or other factors may have had impacts on fungal-free survival in substantive ways other than the reclassification system, which could also affect sample size collection.

In conclusion, the more stringent 2020 EORTC/MSG criteria provide greater certainty of diagnosis, an important consideration for treatment trials; however, these new criteria also pose new challenges for clinical trial design for IA, particularly for prevention studies, where end points should reasonably approximate clinical practice. The findings in this study suggest that several important considerations are affected by the new definitions: the intent of the trial (treatment vs prevention), the type of patient group being studied, the type of mycologic biomarker used in diagnostic assessment (GM vs PCR), divergences between licensed and research criteria for biomarker assays, and the estimates of the expected rates of infection and response and survival rates for power sample size calculations. Until the impact of the new criteria on IFD rates in antifungal prophylactic trials has been determined, we suggest classifying IFD by both the older and the new consensus criteria.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Acknowledgments

The authors would like to thank G. Marshall Lyon, Emory University School of Medicine, for his participation and expertise in the AsTeC consortium.

Financial support. This work was supported by National Institute of Allergy and Infectious Diseases at the National Institutes of Health (contract number HHSN266200700023C [N01-AI-70023]) to J.R.W. for the AsTeC consortium. For the BMT CTN 0101 trial, support for this study was provided by the National Heart, Lung, and Blood Institute and the National Cancer Institute of the National Institutes of Health (grant numbers 5UG1HL069301-21 to J.R.W. and 2U10HL069294-11 to the Blood and Marrow Transplant Clinical Trials Network), along with contributions by Pfizer Inc. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Potential conflicts of interest. J.R.W. serves on the data safety committee of a clinical trial for Ansun, Celgene, Shire, Merck, Janssen, ReViral, and Cidara. B.D.A. reports grants from Astellas, Leadiant, Cidara, and Scynexis. L.R.B. is involved in clinical trials conducted in collaboration with the NIH, HIV Vaccine Trials Network (HVTN), COVID Vaccine Prevention Network (CoVPN), International AIDS Vaccine Initiative (IAVI), Crucell/Janssen, Moderna, Military HIV Research Program (MHRP), Gates Foundation, and the Ragon Institute. A.M.C. serves on the data safety committee for Roche Molecular, Quidel, ChromaCode, Visby, Day Zero, and First LightDanaher and receives grants from ArcBio and Hologic. D.W.D. and family hold Founder shares in F2G Ltd., a University of Manchester spin-out antifungal discovery company, and acts or has recently acted as a consultant to Pulmatrix, Pulmocide, Zambon, iCo Therapeutics, Mayne Pharma, Biosergen, Bright Angel Therapeutics, Cipla, and Metis. D.W.D. serves on the data safety and monitoring board for a SARS-CoV-2 vaccine trial and has been compensated on behalf of Dynamiker, Hikma, Gilead, Merck, Mylan, and Pfizer. F.M.M. receives grants, personal fees, and nonfinancial support from Amplyx, F2G, and Merck, grants and nonfinancial support from Cidara, and grants from Scynexis and WHISCON. L.J.W. is president and Medical Director of MiraVista diagnostics and receives no funding from sales of the Aspergillus antigen test. K.A.M. receives personal fees from Cidara, Merck, Sfunga, and Pulmocide and other compensation from MycoMed Technologies. K.A.M. reports personal fees from Cidara, personal fees from Merck, personal fees from Sfunga, personal fees from Pulmocide, and other from MycoMed Technologies. All other authors declare no competing financial interests. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Author contributions. Dr. Wingard drafted and revised the manuscript and approved the final version. All authors contributed to the revisions of the manuscript and approved the manuscript. The AsTeC investigators responsible for enrolling patients and collecting data and samples were Drs. Wingard, Alexander, Baden, Leather, and Sugrue. Drs. Caliendo, Clancy, Denning, Nguyen, and Wheat contributed to the design of AsTeC study and this analysis. Dr. Wheat retested several of the patient samples for the GM index. The BMT CTN investigators who contributed patients and samples were Drs. Wingard, Baden, Marty, and Marr. The BMT CTN investigators who calculated the effect of the reclassification of the cases on the power estimates of the prophylactic trial were Drs. Chen, Logan, and Horowitz.

References