Lead Investigator: Nigam Shah, Stanford University
Title of Proposal Research: Applying machine learning tools to personalize dabigatran treatment decisions
Vivli Data Request: 6716
Funding Source: NHLBI Award, R01HL144555
Potential Conflicts of Interest: None
Summary of the Proposed Research:
Atrial fibrillation—a condition in which the top of the heart beats irregularly—affects 35.5 million people worldwide. People with atrial fibrillation experience a heightened risk of stroke and are therefore often prescribed “blood thinners”, or anti-clot medications. Unfortunately, the most commonly-used drug called warfarin presents risks and burdens to patients, including the need for laboratory monitoring, and interactions with many other medications.
The newer medication dabigatran does not pose these limitations. In the RELY trial, dabigatran 150 mg twice daily was superior to warfarin in preventing a stroke or major clot, while dabigatran 110 mg twice daily was similar to warfarin for these outcomes. Both doses greatly reduced strokes caused by bleeding in the brain, and dabigatran 110 mg twice daily significantly reduced major bleeding anywhere in the body in comparison with warfarin.
The investigators of RELY commented that because the risk of clot was lower with the 150-mg dose, but the risk of major bleeding was lower with the 100mg dose, it may be possible to tailor the choice of dosage to an individual patient, based on their unique risk of clotting or bleeding. There were variations in the absolute risk reduction in clotting and absolute risk increase in bleeding within each group of participants on dabigatran in RELY and the subsequent RELY-ABLE trial. Traditional statistical analyses (called univariate subgroup analyses) have low power for detecting what features contribute to greater benefit or greater harm from a medication, but newer “machine learning” methods have greater power for accomplishing this aim.
Our objective is to create a personalized dabigatran treatment decision tool, which is a calculator that estimates the personalized estimated reduction in risk in stroke or major clot, and increase in risk of major bleeding, from either dose of dabigatran. The tool would be developed using a novel machine learning method called gradient forest analysis, which analyzes participant data from trials to assess how multiple combinations of patient features predict benefit or harm from a drug. Our research will help improve patient care by assisting clinicians to determine the potential risks and benefits of dabigatran, and optimal dose of dabigatran, before prescribing it. We have chosen the gradient forest method because it has been shown to have benefits over previous methods, particularly reducing the rate of wrong conclusions (“false positive” findings), accounting for complex interactions between multiple factors (e.g., a patient who is both lower weight and of South Asian ethnicity may have more benefit than a patient with just one or the other feature), and producing correct (“unbiased”) estimates of the change in the probability of an event on the drug.
The findings will be interpreted using a pre-specified protocol for assessing significance of the results, and communicated through a peer-reviewed journal.
Statistical Analysis Plan:
Two effect measures will be used: (i) Absolute risk reduction in stroke or systemic embolism on each dose of dabigatran versus warfarin; and (ii) absolute risk increase in major hemorrhage on each dose of dabigatran versus warfarin. In the sensitivity analysis, the secondary outcome will be assessed as absolute difference in net clinical benefit on each dose of dabigatran versus warfarin.
The covariates to be included in the analysis are baseline values for: age, sex, long-term vitamin K antagonist therapy, body mass index, weight, ethnic group, creatinine clearance, CHADS2 (congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, stroke [double weight]) score, symptomatic heart failure, systolic and diastolic blood pressure, diabetes status, prior history of stroke or transient ischemic attack, region, aspirin use, amiodarone use, and proton pump inhibitor use.
The gradient forest method constructs one multi-variate decision tree to predict a personalized estimated of the effect measures listed above, among subgroups defined by combinations of the above covariates. To control for bias, it proceeds in four steps that utilize covariate adjustment through a frequentist statistical approach.
A visualization of the four steps in the approach is available here:
https://stanfordmedicine.box.com/s/of6el1eozq8nqsffqlshr1qxmbdcq65n
First, the derivation dataset (RELY participants not in RELY-ABLE) will be divided in half randomly, with an equal number of intensive and standard glycemic control arm participants in each of the two data subsets.
Second, variables will be chosen from the pre-defined set of covariates listed above, by randomly sampling subsets of covariates to construct a decision-tree made of those predictors that could split the first of the two subsamples of data into subgroups with higher and lower treatment effect. Treatment effect is defined as above, with the effect measure being the difference in event rates between each dabigatran therapy arm (separately analyzed) and the warfarin arm. Subgroups will be required to be >5% of the overall study sample.
Third, once the initial decision tree is constructed from the first subsample of data, the values of each predictor that define branches in the decision tree are refined using the second subsample of data, so that the final subgroups at the bottom of the tree (“leaves” of the tree) have maximum between-group differences and minimum within-group differences in treatment effect. Refinement in the second data subset reduces the influence of outliers, and helps produce unbiased heterogeneity of treatment effects (HTE) estimates. The overall approach will be repeated 4,000 times from the first step, to produce a “forest” of trees by repeated random resampling of the data (cross-validation to reduce the risk of false-positives). No change in estimated variable importance is typically observed beyond 4,000 trees for a trial of up to 20,000 participants. Variable importance is defined as the frequency with which a given variable was incorporated into a tree at the first, second, and further split points (i.e., a variable can change positions between trees, but variable selection for each position is tracked to monitor its importance).
After the forest is constructed and cross-validated, a summary (average) decision tree is selected that separated participants into the sub-groups that were most consistent across all trees in the forest.
Once the summary decision tree is constructed, Kaplan-Meier curves for each outcome for the derivation dataset are drawn for each subgroup defined by the final leaves of the tree, with a stratified log-rank test to assess differences in the in Kaplan-Meier event rates between study arms among the subgroups, with P value adjusted using the Q-value estimation approach that empirically adjusts for multiple hypothesis tests.
Following derivation of each decision tree for each effect measure on the derivation population, each decision tree will then be validated by plotting Kaplan-Meier curves for each outcome for the validation dataset (participants who were in both RELY and RELY-ABLE) for each subgroup defined by the final leaves of the tree, with a stratified log-rank test to assess differences in the in Kaplan-Meier event rates between study arms among the subgroups, including both the RELY and RELY-ABLE durations of follow-up to assess robustness.
Although traditional statistical power calculation methods have not been defined for the gradient forest method, we performed a simulation study in which we simulated the published hazard ratio differences between each dabigatran arm and the warfarin arm. We observed that if there were genuine heterogeneity in treatment effects defined by an interaction of at least two predictor variables among those listed above within the RELY trial population, of magnitude equal to the main effect, there would be 23% power to detect it in the standard univariate subgroup analysis published with the RELY trial but 83% power to detect it in the gradient forest analysis.
Missing data will not be imputed in the main analysis. In sensitivity analyses, we will impute missing data with chained equations.
Requested Studies:
Randomized Evaluation of Long Term Anticoagulant Therapy (RE-LY) Comparing the Efficacy and Safety of Two Blinded Doses of Dabigatran Etexilate With Open Label Warfarin for the Prevention of Stroke and Systemic Embolism in Patients With Non-valvular Atrial Fibrillation: Prospective, Multi-centre, Parallel-group, Non-inferiority Trial (RE-LY Study)
Data Contributor: Boehringer Ingelheim
Study ID: NCT00262600
Sponsor ID: 1160.26
RELY-ABLE Long Term Multi-center Extension of Dabigatran Treatment in Patients With Atrial Fibrillation Who Completed the RE-LY Trial and a Cluster Randomised Trial to Assess the Effect of a Knowledge Translation Intervention on Patient Outcomes
Data Contributor: Boehringer Ingelheim
Study ID: NCT00808067
Sponsor ID: 1160.71
Public Disclosures:
Xu, Y., Bechler, K., Callahan, A., & Shah, N. (2023). Principled estimation and evaluation of treatment effect heterogeneity: A case study application to dabigatran for patients with atrial fibrillation. Journal of Biomedical Informatics, 104420. doi:10.1016/j.jbi.2023.104420