A novel statistical approach to detect the efficacy of drugs that have heterogeneous treatment effects

Lead Investigator: Aziz Mezlini, Massachusetts General Hospital
Title of Proposal Research: A novel statistical approach to detect the efficacy of drugs that have heterogeneous treatment effects
Vivli Data Request: 5940
Funding Source: None
Potential Conflicts of Interest: None

Summary of the Proposed Research:

There is often a lot of variability among patients with the same diseases, this includes variability in disease stage, variability in the underlying disease causes and pathways, and variability in drug metabolism profiles. For all these reasons, a given drug might work in some patients and show no effect on many others. This is called a heterogeneous treatment effect.

Clinical trials are often conducted with the aim of proving the drug’s efficacy. However, the statistical methods frequently used in these trials are looking for an average effect and are not specifically designed to detect a heterogeneous treatment effect scenario. Therefore these methods can sometimes be underpowered to detect drug efficacy.

We designed a novel statistical test for this scenario and showed that it has superior power compared to widely used statistical tests: https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2

Better power means potentially the discovery of more beneficial drugs and/or the need for fewer participants and lower costs for future trials. We aim to show the usefulness of our test in real data of clinical trials.

The novel statistical test can be useful for any setting with heterogeneous effects. In the future, it can be applied in other clinical trials as well as other research fields such a biomarker discovery.

Statistical Analysis Plan:

We went through the list of trials on clinicaltrial.gov and curated trials that fit the following criteria: A large enough sample size, a continuous main outcome, and the potential for heterogeneous treatment effects (by looking at the summary statistics).

The reason for this selection is that we want trials where our test is applicable and where we can hope to have enough power to detect drug efficacy. We requested the clinical trials at the intersection of our list and the list of trials available on Vivli.

We will apply our novel statistical test on the main outcomes of each trial we are granted access to.

Here is a reference to our test with more details: https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2

In each study, we will correct for the same covariates and interactions (same model) as was done in the original analysis of the trial. We will handle missing data (imputation or exclusion) in the same way and using the same methods as the original analysis whenever possible.

We do not plan on doing an analysis of subgroups based on covariates.

We are following a case-control design. In scenarios with more than two groups (such as multiple drug dosages), we will consider the same groupings and we will do the same pairwise comparisons as the original analysis.

Our goal is to detect efficacy as a significant statistical difference in distribution between cases and controls. Our main result will be a p-value per outcome, corrected for multiple hypothesis testing.

Requested Studies:

A 24-week, Double-blind, Double-dummy, Randomized, Parallel-group Study to Investigate the Effects of Rosiglitazone (Extended Release Tablets), Donepezil, and Placebo as Monotherapy on Cognition and Overall Clinical Response in APOE ε4-stratified Subjects With Mild to Moderate Alzheimer’s Disease. (REFLECT-1)
Sponsor: GlaxoSmithKline
Study ID: NCT00428090
Sponsor ID: 105640

A Multinational, Randomized, Double-Blind, Placebo-Controlled, Dose Ranging Study to Assess the Efficacy and Safety of Lu AA21004 in Patients With Major Depressive Disorder
Sponsor: Takeda
Study ID: NCT01255787
Sponsor ID: LuAA21004/CCT-002

A Prospective Multi-Centre Randomised, Double-Blind, Active Comparator-Controlled, Parallel-Groups Study Comparing the Fully Human Monoclonal Anti-TNFα Antibody Adalimumab Given Every Second Week With Methotrexate Given Weekly and the Combination of Adalimumab and Methotrexate Administered Over 2 Years in Patients With Early Rheumatoid Arthritis (PREMIER).
Sponsor: AbbVie
Study ID: NCT00195663
Sponsor ID: DE013

A Phase 3 Study to Evaluate the Efficacy and Safety of Dexlansoprazole MR (30 mg QD and 60 mg QD) Compared to Placebo on Symptom Relief in Subjects With Symptomatic Nonerosive Gastroesophageal Reflux Disease (GERD)
Sponsor: Takeda
Study ID: NCT00321984
Sponsor ID: T-GD05-137

A Randomised, Double-blind, Parallel-group, Placebo-controlled, Duloxetine-referenced, Fixed-dose Study Evaluating the Efficacy and Safety of Three Dosages of [Vortioxetine] Lu AA21004, in Acute Treatment of Major Depressive Disorder
Sponsor: Lundbeck
Study ID: NCT00635219
Sponsor ID: 11984A

A Randomized, Double-blind, Parallel-group, Placebo-controlled, Fixed-dose Study Comparing the Efficacy and Safety of Lu AA21004 Versus Placebo in Acute Treatment of Adults With Major Depressive Disorder
Sponsor: Takeda
Study ID: NCT00672958
Sponsor ID: LuAA21004_303

A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Phase III Study to Evaluate the Efficacy and Safety of Once Daily Oral Lu AA21004 in Patients With Major Depressive Disorder
Sponsor: Takeda
Study ID: NCT02389816
Sponsor ID: LuAA21004/CCT-004

A Clinical Outcomes Study to Compare the Effect of Fluticasone Furoate/Vilanterol Inhalation Powder 100/25mcg With Placebo on Survival in Subjects With Moderate Chronic Obstructive Pulmonary Disease (COPD) and a History of or at Increased Risk for Cardiovascular Disease
Sponsor: GlaxoSmithKline
Study ID: NCT01313676
Sponsor ID: HZC113782

A Randomized, Double-blind, Placebo-controlled, Parallel Group Trial Assessing the Rate of Decline of Lung Function With Tiotropium 18 mcg Inhalation Capsule Once Daily in Patients With Chronic Obstructive Pulmonary Disease (COPD).
Sponsor: Boehringer Ingelheim
Study ID: NCT00144339
Sponsor ID: 205.235

A Randomized, Phase 3, Controlled, Double-Blind, Parallel-Group, Multicenter Study to Evaluate the Safety and Efficacy of Rituximab in Combination With Methotrexate (MTX) Compared to MTX Alone, in Methotrexate-Naive Patients With Active Rheumatoid Arthritis
Sponsor: Roche
Study ID: NCT00299104
Sponsor ID: U3373g

A Randomized, Double-blind, Parallel-group Study to Evaluate the Safety and Efficacy of Tocilizumab (TCZ) Versus Placebo in Combination With Disease Modifying Antirheumatic Drugs (DMARDs) in Patients With Moderate to Severe Active Rheumatoid Arthritis (RA)
Sponsor: Roche
Study ID: NCT00531817
Sponsor ID: ML21136

CONSTATRE: Risperdal Consta Trial Of Relapse Prevention And Effectiveness
Sponsor: Johnson & Johnson
Study ID: NCT00216476
Sponsor ID: CR002269

A Phase 3, Randomized, Double-blind, Placebo-controlled, Multicenter Study of the Efficacy and Safety of Four 12-week Treatment Cycles (48 Weeks Total) of Epratuzumab in Systemic Lupus Erythematosus Subjects With Moderate to Severe Disease
Sponsor: UCB
Study ID: NCT01262365
Sponsor ID: SL0009

A Phase 3, Randomized, Double-blind, Placebo-controlled, Multicenter Study of the Efficacy and Safety of Four 12-week Treatment Cycles (48 Weeks Total) of Epratuzumab in Systemic Lupus Erythematosus Subjects With Moderate to Severe Disease
Sponsor: UCB
Study ID: NCT01261793
Sponsor ID: SL0010

A Randomized, Double-Blind, Placebo-controlled Trial of Long-term (2-year) Treatment of Galantamine in Mild to Moderately-Severe Alzheimer’s Disease
Sponsor: Johnson & Johnson
Study ID: NCT00679627
Sponsor ID: CR012463

Summary of Results:

The aim of the project was to look for evidence of heterogeneous treatment effects using a new statistical test better powered to detect them (R package aziztest). We investigated 15 clinical trials datasets obtained via Vivli. Unfortunately, our analysis did not find significant evidence of heterogeneous treatment effect in the15 trials studied. In the trials where the drugs had proved their efficacy, our p-values were comparable to those obtained using traditional methods in the original analysis. Therefore, we could not show significantly improved power (by design, this would only be possible in a heterogeneous treatment effect setting). In the trials where the drugs/treatments previously failed to show efficacy, using our test also resulted in non-significant p-values especially after correcting for multiple tests (nominal significance in only one Alzheimer’s disease trial). We were hoping that some drugs previously failed to show efficacy because of a lack of statistical power due to the methods used being inadequate to detect heterogeneous efficacy. This does not seem to be the case for the datasets we considered. Through this project we learned that heterogeneous effects, where a drug only shows efficacy on a proportion of participants, are likely not a very common occurrence. In the future, we aim to better select trials where there is reason to believe heterogeneous efficacy is likely in order to showcase our statistical test.