https://orcid.org/0000-0003-1106-3747
Sir,
We applaud van Hoogenhuijze and colleagues for the individual participant data (IPD) meta-analysis, which concluded that endometrial scratching may improve live birth rates in women undergoing IVF/ICSI, regardless of age, number of previous failed embryo transfers, treatment type, and infertility cause (Van Hoogenhuijze et al. 2023). This is intriguing given the absence of a plausible biological rationale, and we think it requires more consideration. We would like to present a different opinion than Vitagliano et al. (2024).
Van Hoogenhuijze and colleagues (2023) did a meticulous work by analyzing and presenting their result in a multitude of ways including both adjusted and unadjusted effect estimates. We would like to commend the authors for their insightful interpretation of pooled analyses results, without focusing on statistical significance, which is merely described by the confidence intervals including ‘the One’ or not, rather than biological facts. While the majority of the adjusted analyses resulted in ‘statistically non-significant’ differences, consistency of point estimates across multiple one-stage, two-stage, intention-to-treat (ITT), and as-treated analyses may increase confidence in them, and it can be tempting to conclude that endometrial scratching may improve IVF/ICSI outcome. However, we need to consider what drives the point estimates. When we examine the forest plot of trials reporting live births, what we see is a complete or near complete diminution of the intervention effect in two of the largest trials (Lensen et al. 2019; Metwally et al. 2021). The third largest trial by van Hoogenhuijze et al. (2021) still shows a substantial effect, but the fact remains that it is the smaller studies that are the main drivers of the positive effect shown in the aggregate summary. A one-stage unadjusted summary of the included studies demonstrates an interaction between study arm size and effect estimate (Fig. 1). In our opinion, this should be quite alarming for the possibility of publication bias and/or small study effects. Applying modified Egger’s test for funnel plot asymmetry to studies reporting live births, we observe a clear indication of publication bias and/or small study effect, as suggested by Van Hoogenhuijze and colleagues themselves (Fig. 2). However, given that the aggregate summary is driven primarily by trials with small sample, we feel that the relevance of this finding needs to be strongly emphasized. Adding the strong possibility of publication bias to the inconsistency and imprecision of the aggregate effect, the result is a summary effect of very low certainty according to the GRADE evaluation (Guyatt et al., 2008). Moreover, the consistency of point estimates in both ITT and as treated with embryo transfer (AT-ET) analyses can be regarded to support ineffectiveness of endometrial scratching, since the effect of this intervention would be expected to be apparent in women undergoing embryo transfer. Hence, had the intervention been truly effective, point estimate should have been perceivably higher in the AT-ET population than in the ITT population.
Predicted livebirth probability between interventions according to trial arm size of included studies.
Funnel plot of studies reporting on live birth (Lee et al. excluded due to outlier status and negligible effect of aggregate summary).
Statistics is an indispensable tool for making sense of observations in an objective manner. However, it is the science of measuring uncertainty, and a fuller understanding of the results by content experts, which may include subjective judgements, is not only inevitable but indeed essential. We cannot set aside the principles of causality, directness, precision, dose-response, and biological plausibility and shout ‘Eureka’ every time we happen to observe an association. A healthy dose of skepticism is always required in science and clinical practice, and well-justified criticism is not necessarily a reflection of ‘intellectual ego’, as suggested by Vitagliano et al. (2024). Subjecting all individuals contemplating an embryo transfer to an invasive procedure of questionable benefit (which is likely driven by publication bias) but with certain costs, discomfort and possible complications seems unjustified; embarking on studies to understand how endometrial scratching benefits IVF/ICSI outcomes before proving its efficacy beyond reasonable doubt may lead to wasted time and effort in pursuit of a potential benefit that may not even exist.
Conflict of interest
The authors have no conflict of interest associated with this letter.
