Table 2
Open in new tab

Performance of the AIPW Software Package in Estimating the Average Treatment Effect (Risk Difference) in a Simulated Observational Study Based on the EAGeR Triala

Method and Software PackageBias (SE)MSEMean 95% CI  
Width		95% CI  
Coverage (SE), %b		Mean Run Time,  
seconds
True model: GLM + no cross-fitting
 G-computation−0.002 (0.002)0.0050.27194.8 (0.5)1.82
 IPW−0.002 (0.002)0.0050.28095.8 (0.4)0.01
 AIPW−0.002 (0.002)0.0050.26894.8 (0.5)0.36
 CausalGAM−0.003 (0.002)0.0050.26794.8 (0.5)0.07
 npcausal−0.002 (0.002)0.0050.26794.6 (0.5)0.24
 tmle−0.002 (0.002)0.0050.26194.4 (0.5)0.29
 tmle3−0.002 (0.002)0.0050.26894.8 (0.5)0.31
GAMs + no cross-fitting
 AIPW−0.002 (0.002)0.0050.26193.8 (0.5)1.16
 CausalGAM−0.004 (0.002)0.0050.26692.7 (0.6)0.19
 npcausal−0.002 (0.002)0.0050.26093.9 (0.5)0.98
 tmle−0.002 (0.002)0.0050.25794.0 (0.5)0.86
 tmle3−0.002 (0.002)0.0050.26193.9 (0.5)4.54
GAMs + k = 10 cross-fitting
 AIPW−0.002 (0.002)0.0050.31096.6 (0.4)7.92
 npcausal−0.002 (0.002)0.0060.31996.5 (0.4)3.55
 tmlec−0.002 (0.002)0.0050.27295.6 (0.5)5.15
 tmle3−0.002 (0.002)0.0050.30896.5 (0.4)7.51
SuperLearnerd + no cross-fitting
 AIPW−0.009 (0.002)0.0050.24693.0 (0.6)14.65
 npcausal−0.005 (0.002)0.0050.23290.3 (0.7)21.71
 tmle−0.009 (0.002)0.0050.25193.8 (0.5)13.44
 tmle3−0.005 (0.002)0.0050.24692.2 (0.6)36.76
SuperLearnerd + k = 10 no cross-fitting
 AIPW−0.002 (0.002)0.0050.28195.6 (0.5)128.48
 npcausal−0.004 (0.002)0.0050.28595.5 (0.5)183.54
 tmlec−0.006 (0.002)0.0050.26694.5 (0.5)43.38
 tmle3−0.004 (0.002)0.0050.27295.2 (0.5)48.52
Method and Software PackageBias (SE)MSEMean 95% CI  
Width		95% CI  
Coverage (SE), %b		Mean Run Time,  
seconds
True model: GLM + no cross-fitting
 G-computation−0.002 (0.002)0.0050.27194.8 (0.5)1.82
 IPW−0.002 (0.002)0.0050.28095.8 (0.4)0.01
 AIPW−0.002 (0.002)0.0050.26894.8 (0.5)0.36
 CausalGAM−0.003 (0.002)0.0050.26794.8 (0.5)0.07
 npcausal−0.002 (0.002)0.0050.26794.6 (0.5)0.24
 tmle−0.002 (0.002)0.0050.26194.4 (0.5)0.29
 tmle3−0.002 (0.002)0.0050.26894.8 (0.5)0.31
GAMs + no cross-fitting
 AIPW−0.002 (0.002)0.0050.26193.8 (0.5)1.16
 CausalGAM−0.004 (0.002)0.0050.26692.7 (0.6)0.19
 npcausal−0.002 (0.002)0.0050.26093.9 (0.5)0.98
 tmle−0.002 (0.002)0.0050.25794.0 (0.5)0.86
 tmle3−0.002 (0.002)0.0050.26193.9 (0.5)4.54
GAMs + k = 10 cross-fitting
 AIPW−0.002 (0.002)0.0050.31096.6 (0.4)7.92
 npcausal−0.002 (0.002)0.0060.31996.5 (0.4)3.55
 tmlec−0.002 (0.002)0.0050.27295.6 (0.5)5.15
 tmle3−0.002 (0.002)0.0050.30896.5 (0.4)7.51
SuperLearnerd + no cross-fitting
 AIPW−0.009 (0.002)0.0050.24693.0 (0.6)14.65
 npcausal−0.005 (0.002)0.0050.23290.3 (0.7)21.71
 tmle−0.009 (0.002)0.0050.25193.8 (0.5)13.44
 tmle3−0.005 (0.002)0.0050.24692.2 (0.6)36.76
SuperLearnerd + k = 10 no cross-fitting
 AIPW−0.002 (0.002)0.0050.28195.6 (0.5)128.48
 npcausal−0.004 (0.002)0.0050.28595.5 (0.5)183.54
 tmlec−0.006 (0.002)0.0050.26694.5 (0.5)43.38
 tmle3−0.004 (0.002)0.0050.27295.2 (0.5)48.52

Abbreviations: AIPW, augmented inverse probability weighting; CI, confidence interval; EAGeR, Effects of Aspirin in Gestation and Reproduction; GAM, generalized additive model; GLM, generalized linear model; IPW, inverse probability weighting; MSE, mean squared error; SE, standard error.

a Simulations were conducted with a sample size of 200 and 2,000 Monte Carlos simulations; the true risk difference was 0.128. Numbers in parentheses show Monte Carlo SEs for the performance indicator estimates.

b Asymptotic SEs were used for CI calculation in AIPW, CausalGAM, tmle, and tmle3. The CIs for G-computation and IPW were obtained via 200 bootstraps and sandwich estimators, respectively.

c Cross-fitting was conducted in the outcome model only because of its implementation.

d SuperLearner was used for tmle and AIPW, and sl3 was used for tmle3. Algorithms included gam, earth, ranger, and XGBoost.

Table 2
Open in new tab

Performance of the AIPW Software Package in Estimating the Average Treatment Effect (Risk Difference) in a Simulated Observational Study Based on the EAGeR Triala

Method and Software PackageBias (SE)MSEMean 95% CI  
Width		95% CI  
Coverage (SE), %b		Mean Run Time,  
seconds
True model: GLM + no cross-fitting
 G-computation−0.002 (0.002)0.0050.27194.8 (0.5)1.82
 IPW−0.002 (0.002)0.0050.28095.8 (0.4)0.01
 AIPW−0.002 (0.002)0.0050.26894.8 (0.5)0.36
 CausalGAM−0.003 (0.002)0.0050.26794.8 (0.5)0.07
 npcausal−0.002 (0.002)0.0050.26794.6 (0.5)0.24
 tmle−0.002 (0.002)0.0050.26194.4 (0.5)0.29
 tmle3−0.002 (0.002)0.0050.26894.8 (0.5)0.31
GAMs + no cross-fitting
 AIPW−0.002 (0.002)0.0050.26193.8 (0.5)1.16
 CausalGAM−0.004 (0.002)0.0050.26692.7 (0.6)0.19
 npcausal−0.002 (0.002)0.0050.26093.9 (0.5)0.98
 tmle−0.002 (0.002)0.0050.25794.0 (0.5)0.86
 tmle3−0.002 (0.002)0.0050.26193.9 (0.5)4.54
GAMs + k = 10 cross-fitting
 AIPW−0.002 (0.002)0.0050.31096.6 (0.4)7.92
 npcausal−0.002 (0.002)0.0060.31996.5 (0.4)3.55
 tmlec−0.002 (0.002)0.0050.27295.6 (0.5)5.15
 tmle3−0.002 (0.002)0.0050.30896.5 (0.4)7.51
SuperLearnerd + no cross-fitting
 AIPW−0.009 (0.002)0.0050.24693.0 (0.6)14.65
 npcausal−0.005 (0.002)0.0050.23290.3 (0.7)21.71
 tmle−0.009 (0.002)0.0050.25193.8 (0.5)13.44
 tmle3−0.005 (0.002)0.0050.24692.2 (0.6)36.76
SuperLearnerd + k = 10 no cross-fitting
 AIPW−0.002 (0.002)0.0050.28195.6 (0.5)128.48
 npcausal−0.004 (0.002)0.0050.28595.5 (0.5)183.54
 tmlec−0.006 (0.002)0.0050.26694.5 (0.5)43.38
 tmle3−0.004 (0.002)0.0050.27295.2 (0.5)48.52
Method and Software PackageBias (SE)MSEMean 95% CI  
Width		95% CI  
Coverage (SE), %b		Mean Run Time,  
seconds
True model: GLM + no cross-fitting
 G-computation−0.002 (0.002)0.0050.27194.8 (0.5)1.82
 IPW−0.002 (0.002)0.0050.28095.8 (0.4)0.01
 AIPW−0.002 (0.002)0.0050.26894.8 (0.5)0.36
 CausalGAM−0.003 (0.002)0.0050.26794.8 (0.5)0.07
 npcausal−0.002 (0.002)0.0050.26794.6 (0.5)0.24
 tmle−0.002 (0.002)0.0050.26194.4 (0.5)0.29
 tmle3−0.002 (0.002)0.0050.26894.8 (0.5)0.31
GAMs + no cross-fitting
 AIPW−0.002 (0.002)0.0050.26193.8 (0.5)1.16
 CausalGAM−0.004 (0.002)0.0050.26692.7 (0.6)0.19
 npcausal−0.002 (0.002)0.0050.26093.9 (0.5)0.98
 tmle−0.002 (0.002)0.0050.25794.0 (0.5)0.86
 tmle3−0.002 (0.002)0.0050.26193.9 (0.5)4.54
GAMs + k = 10 cross-fitting
 AIPW−0.002 (0.002)0.0050.31096.6 (0.4)7.92
 npcausal−0.002 (0.002)0.0060.31996.5 (0.4)3.55
 tmlec−0.002 (0.002)0.0050.27295.6 (0.5)5.15
 tmle3−0.002 (0.002)0.0050.30896.5 (0.4)7.51
SuperLearnerd + no cross-fitting
 AIPW−0.009 (0.002)0.0050.24693.0 (0.6)14.65
 npcausal−0.005 (0.002)0.0050.23290.3 (0.7)21.71
 tmle−0.009 (0.002)0.0050.25193.8 (0.5)13.44
 tmle3−0.005 (0.002)0.0050.24692.2 (0.6)36.76
SuperLearnerd + k = 10 no cross-fitting
 AIPW−0.002 (0.002)0.0050.28195.6 (0.5)128.48
 npcausal−0.004 (0.002)0.0050.28595.5 (0.5)183.54
 tmlec−0.006 (0.002)0.0050.26694.5 (0.5)43.38
 tmle3−0.004 (0.002)0.0050.27295.2 (0.5)48.52

Abbreviations: AIPW, augmented inverse probability weighting; CI, confidence interval; EAGeR, Effects of Aspirin in Gestation and Reproduction; GAM, generalized additive model; GLM, generalized linear model; IPW, inverse probability weighting; MSE, mean squared error; SE, standard error.

a Simulations were conducted with a sample size of 200 and 2,000 Monte Carlos simulations; the true risk difference was 0.128. Numbers in parentheses show Monte Carlo SEs for the performance indicator estimates.

b Asymptotic SEs were used for CI calculation in AIPW, CausalGAM, tmle, and tmle3. The CIs for G-computation and IPW were obtained via 200 bootstraps and sandwich estimators, respectively.

c Cross-fitting was conducted in the outcome model only because of its implementation.

d SuperLearner was used for tmle and AIPW, and sl3 was used for tmle3. Algorithms included gam, earth, ranger, and XGBoost.

Close
This Feature Is Available To Subscribers Only

Sign In or Create an Account

Close

This PDF is available to Subscribers Only

View Article Abstract & Purchase Options

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Close