Enhancing robustness in acute cardiovascular observational studies: evaluating covariate adjustment

In clinical trials, the primary objective often involves studying the associations between several variables. In randomized clinical trials (RCTs), the focus typically lies on the association between clinical outcomes and two or several treatment options. Conversely, in observational studies, interest extends beyond treatments and may include associations with various patient characteristics (e.g. demographic and disease-specific factors), frequently for predictive or prognostic purposes.

These associations, however, may be subject to influence by external factors, commonly known as confounding factors, which may introduce biases in the conclusions. To mitigate the impact of these confounding factors, appropriate measures should be taken to avoid imbalances. One strategy to eliminate potential bias is to balance subjects across these factors through randomization and stratification. Another approach is to adjust the statistical analyses for these covariates. Although covariate adjustment is not strictly necessary in RCTs, it may enhance efficiency.¹

Covariate adjustment is well studied in RCTs, with available regulatory guidance,^2,3 although the quality of implementation varies and may be improved.¹ There is also a need to study covariate adjustment in observational studies and to develop guidance documents,⁴ as the requirement to adjust is higher in non-randomized studies. Moreover, the variability in implementation of covariate adjustments is equally, if not more, present in observational studies. The latter is demonstrated by a review of publications in the European Heart Journal: Acute Cardiovascular Care over the past 5 years, which included 55 non-randomized studies, representing 62 covariate-adjusted analyses.

In the observational studies, the most commonly used covariate adjustments were multiple or multivariable regression models (n = 49; 79%), similar to those in RCTs. These included Cox proportional hazards (n = 23; 47%), logistic (n = 22; 45%), and linear (n = 4; 8%) regression models. Due to the non-randomized nature of observational studies, often more covariates are imbalanced compared with randomized trials and need to be included in these models. Adding numerous covariates in regression models, however, can be cumbersome and may lead to over-fitting, i.e. tailoring the model too much to the available data, thereby reducing generalizability. It is important to note that multiple/multivariable regression models should not be confused with multivariate models, which aim at modelling multiple clinical outcomes in a multilevel or joint approach.

An alternative method for covariate adjustment, aimed at correcting potential bias rather than examining covariate–outcome associations, is covariate adjustment through propensity scores⁵ (n = 13; 21%). These scores, calculated through logistic models, represent the probability of patients belonging to a subgroup given a set of covariates, summarizing all patient characteristics into a single value. Propensity scores reduce the potential for over-fitting and can be used for covariate adjustment through matching, stratification, inverse probability weighting, or as a covariate in a regression model.⁵ Although there is no clear superior method,⁵ matched propensity scores were most commonly used (n = 8; 62%), followed by modelling as a covariate (n = 3; 23%) and inverse treatment weighting (n = 2; 15%). Propensity score matching has the potential disadvantage of excluding unmatched observations. When interpreting treatment effects, it is important to realize that inverse probability weighting and matching estimate marginal treatment effects, whereas multivariable regression and stratification estimate conditional effects.⁵

The methods for selecting covariates for adjustment are another source of variability. Most studies pre-selected covariates based on prior knowledge (n = 33; 61%), while some used automated selection procedures (n = 11; 20%) or included all covariates with a P-value below a pre-specified threshold in a univariate analysis (n = 8; 15%). A few studies selected covariates based on observed imbalances between groups (n = 2, 4%). Pre-selecting covariates risk missing important confounders, whereas automated selection procedures may lead to over-fitting. Regardless of the method, the selection procedure should be clearly presented, as in some cases involving automated procedures, it was insufficiently detailed whether a forward or backward selection was used.

Additionally, variation existed in evaluating the validity and robustness of conclusions. Robust conclusions require assessing the validity of the models’ implicit assumptions, such as proportional hazards and the form of the association (n = 4; 7%). The reliability of conclusions also depends on the absence of the influence of extreme observations (n = 2; 4%), which may cause bias, and the absence of multi-collinearity (n = 6; 11%), which can increase variability and decrease efficiency. Confirming results through sensitivity analyses (n = 14; 26%), by varying covariate selection procedures^6,7 or adjustment methods,^6,8 also enhances robustness. Notice that when multiple analyses are performed in an exploratory observational trial, handling multiplicity is crucial, even though conclusions from observational studies are generally hypothesis generating, especially when based on single-centre or single-country databases.

Finally, variation was observed in handling missing data (n = 7; 13%). Missing data may lead to biased conclusions, and the mechanism of missingness may be complex for observational studies. Approaches applied to deal with missing data included multiple imputations (n = 6; 11%) and worst-case/best-case scenarios (n = 1; 2%).

In the context of acute cardiovascular trials, observational studies are conducted frequently, which require covariate adjustments for unbiased results. However, current practice of covariate adjustment varies. There is a need for further methodological investigation and the development of guidance on covariate adjustment for non-randomized studies (Table 1).

Funding

None declared.

Data availability

There are no new data associated with this article.

References

Author notes