Lead Investigator: Johnathan Bartlett, London School of Hygiene & Tropical Medicine
Title of Proposal Research: Hypothetical estimand in clinical trials: an application of causal inference and missing data methods
Vivli Data Request: 6764
Funding Source: This research is funded by the UK Medical Research Council, grant MR/T023953/1.
Potential Conflicts of Interest: Dr. Bartlett has received consultancy fees from Bayer for advice on statistical methods in clinical trials. The University of Bath has received consultancy fees for Bartlett’s advice on statistical issues in clinical trials from Bayer, AstraZeneca, Novartis and Roche.
Summary of the Proposed Research:
To compare the benefits of two or more treatments for a given condition, we conduct experiments in a group of subjects with the condition, assign one of the treatments at random and follow them up to assess their outcome. These experiments are known as randomised trials. The analysis and interpretation of randomised trials is often complicated by the occurrence of non-planned events such as patients dropping out of the trial, patients requiring additional medication (i.e. rescue medication), or dying before their outcome is measured.
There are different statistical methods to handle these non-planned events. The chosen method for each non-planned event should be pre-specified in the statistical analysis plan when a trial is designed because different methods result in different estimates of the effectiveness and/or safety on the drug. This target effect to be measured in a trial is known as the estimand.
One of the possible estimands is the hypothetical estimand, where the treatment effect is estimated under the hypothetical scenario in which these non-planned events were prevented from occurring. For instance, the envisaged scenario could be one where patients continue taking their assigned drug as indicated and do not require additional medication.
In this project, we focus on different ways to estimate the hypothetical estimand. We consider statistical methods from causal inference, which are methods developed to evaluate the effects of treatments in settings where the treatment is not assigned at random (i.e. outside of a randomised trial). We will also apply missing data methods that were developed to deal with missing values in a given data set and have been more often used in the context of randomised trials. Establishing links between ‘causal inference estimators’ and ‘missing data estimators’ may help those familiar with one set of methods but not the other.
To compare the performance of these different statistical methods, we will apply them to the present diabetes clinical trial where rescue medication was allowed to be given in addition to the patient’s randomised treatment in case of inadequate blood sugar control with the study. Different rates of rescue medication used between treatment groups can potentially dilute or mask the treatment effect. By targeting the hypothetical estimand, the treatment effect can be adjusted for rescue medication use and other non-planned events including treatment discontinuation. We hope that by illustrating how the different statistical methods can be applied to a real clinical trial dataset we will demonstrate to other researchers how these methods can be usefully applied to their trials.
Statistical Analysis Plan:
We plan to estimate the treatment effect targeting the hypothetical estimand, where we consider the treatment effect in the hypothetical situation where no patient would have been given insulin as rescue medication or discontinued from their randomised treatment. For the analysis, we will include all patients randomized to the three arms (n=939). As in the original trial, we will compare the absolute difference in glycated hemoglobin (HbA1c) and fasting plasma glucose (FPG) from baseline to 52 weeks after follow-up between dapaglifozin plus metformin and glimeperide plus metformin and between dapaglifozin plus saxagliptin plus metformin and glimepiride plus metformin. First, we will estimate the treatment effect as the difference in mean outcome between the randomised treatment arms, among those who did not receive rescue medication and stayed on their randomised treatment. We will consider this the ‘naïve’ approach.
Then, we will apply statistical methods from the causal inference and missing data literature. The analyses will be adjusted for the baseline and time-varying covariates available and listed earlier through four different estimation approaches: g-formula, inverse probability weighting (IPW), multiple imputation and mixed models. We will discuss different implementations of these approaches and show that the choice of a particular implementation is a trade-off between precision and robustness.
Below we provide details of the different implementations
• G-formula using all data (GF1): a sequence of conditional multivariate models will be fitted to the measurements made at each follow-up visit using all available data. Each of these models will include main effects of the randomised treatment, baseline covariates and time-varying covariates until that visit, including past rescue medication use and whether the patient had discontinued randomised treatment. These models will then be used to predict the values of the time-varying variables at each visit for each patient, and consequently the value of HbA1c at week 52 under the hypothetical scenario of interest, namely no rescue medication used at any time point during follow-up and no discontinuation of randomised treatment. The treatment effect estimate is the difference in means of the predicted HbA1c for the final visit between treatment groups under this scenario.
• G-formula among those who did not receive rescue medication or discontinue randomised treatment (GF2): similar GF1 but the models at each visit are fitted including only subjects who did not use rescue medication or discontinue randomised treatment before the corresponding visit.
• G-formula separately by treatment arm (GF3): similar to GF1 but the models are fitted separately by randomised treatment groups.
• G-formula among those who did not receive rescue medication or discontinue randomised treatment separately by treatment arm (GF4): this combines the variations introduced in GF2 and GF3 – the models are fitted separately by randomised treatment groups and among only those who at that visit did not receive rescue medication or discontinued randomised treatment before the corresponding visit.
• IPW using all data (IPW1): a logistic regression is fitted to estimate the probability of receiving rescue medication at each time point among patients who have not yet discontinued randomised treatment, adjusting for randomised treatment, observed rescue medication history, and observed covariate history. A separate logistic regression model is fitted to estimate the probability of discontinuing randomised treatment at each time point, among those who have not yet discontinued, with the same covariates. From these models, we calculate the weight of each patient as the inverse of the probability of them not receiving rescue medication at any time and also not discontinuing randomised treatment. The treatment effect estimate is then the difference in weighted mean HbA1c measured at the last visit between treatment groups, where the weighted means are calculated using data from those patients who did not receive rescue nor discontinue randomised treatment.
• IPW among those who did not receive rescue medication or discontinue (IPW2): this is similar to the IPW1 method with the difference that the first logistic regression model at each visit is fitted including only those among those who did not previously receive rescue medication (or discontinue randomised treatment).
• Separate IPW per arm treatment (IPW3): similar to the IPW1 method but fitting weight models separately by treatment arm.
• Separate IPW per arm treatment among those who did not receive rescue medication (IPW4): this method combines IPW2 and IPW3, with weight models fitted separately by treatment arm but among only those who did not previously receive rescue medication (or discontinue randomised treatment).
• Multiple imputation with treatment as covariate (MI1): first, all the observations that occur after receiving rescue medication for the first time or discontinuation of randomised treatment are deleted. Sequential imputation of the variables measured (including HbA1c and FPG) at follow-up each visit is performed using the mice package in R, adjusting for main effects of treatment and past covariate history. The treatment effect estimate is the difference in mean of Hb1c between treatment groups across the 1000 imputed datasets.
• Multiple imputation separately by treatment arm (MI2): similar to MI1 but the imputation models are constructed separately for each treatment arm.
• Linear mixed models for repeated measurements (MIX1): as with multiple imputation, all the observations that occur after receiving rescue medication for the first time or after discontinuation of randomised treatment are deleted. Then the linear mixed model is fitted using the nlme package (gls function) to the repeated measurements of HbA1c and other variables measured at each visit. To allow for within-patient correlation, an unstructured residual covariance matrix will be specified. The mean of each variable will be allowed to depend on baseline covariates, including randomised treatment. The treatment effect estimate is the model estimated difference in mean Hb1c at week 52 between treatment groups.
• Mixed models for repeated measurements separately by treatment arm (MIX2): similar to MIX1 but the models are fitted separately for each treatment arm, and the contrast in treatment group means will be manually calculated.
For each of the preceding analyses we will also repeat all of them, using only HbA1c measurements at follow-up, i.e. without utilising the other variables measured at follow-up visits. This is because this approach matches (the mixed model approach MIX1 specifically) the analysis method used in the original analysis of this trial.
The standard error and confidence intervals for each method will be derived using sandwich estimators and bootstrap. The assumptions of each approach will be assessed using methods appropriate to each, e.g. model fit diagnostics for the fit of the logistic regression models used for modelling whether patients received rescue medication at each follow-up visit. We will then compare the estimates obtained by the different approaches and attempt to explain differences with reference to the established theoretical properties of the different estimators. We will finally consider any potential advantages and drawbacks of the different approaches in terms of their precision, computation efficiency, and strength of assumptions.
Requested Studies:
A 52-Week, Multi-Centre, Randomised, Parallel-Group, Double-Blind, Active Controlled, Phase IV Study to Evaluate the Safety and Efficacy of Dapagliflozin or Dapagliflozin Plus Saxagliptin Compared With Sulphonylurea All Given as Add-on Therapy to Metformin in Adult Patients With Type 2 Diabetes Who Have Inadequate Glycaemic Control on Metformin Monotherapy
Data Contributor: AstraZeneca
Study ID: NCT02471404
Sponsor ID: D1689C00014
Public Disclosures:
Olarte Parra, C., Daniel, R.M., Wright, D. and Bartlett, J.W., 2025. Estimating hypothetical estimands with causal inference and missing data estimators in a diabetes trial case study. Biometrics, 81(1), p.ujae167. Doi: 10.1093/biomtc/ujae167