Lead Investigator: Klaus Gottlieb, Eli Lilly USA
Title of Proposal Research: Mapping of components of the complete blood count and comprehensive metabolic profile to disease-specific clinical and endoscopy endpoints in phase 2 and 3 clinical trials of ulcerative colitis and Crohn’s disease
Vivli Data Request: 3328
Funding Source: Commercial Funding: The work will be conducted by full-time employees of Eli Lilly and company, the applicant.
Potential Conflicts of Interest: Eli Lilly is involved in research and development of drugs for autoimmune diseases, including inflammatory bowel disease. Research findings will be made public.
Summary of the Proposed Research
If successful, this machine learning research project will lead to a better understanding of how routinely obtained reported laboratory values can be used to predict endoscopic outcomes of patients with ulcerative colitis and Crohn’s disease. Endoscopic improvement is a prerequisite of healing and getting better. Endoscopic improvement and related outcomes are measured by endoscopic evaluations and multi-component clinical endpoints, such as Crohn’s Disease Activity Index (CDAI) and Mayo score. Given an invasive nature of endoscopic evaluations and the complex nature of clinical trial endpoints, an ability to evaluate patient outcomes using alternative non-invasive and commonly performed tests would be greatly beneficial to patients in both clinical practice and clinical drug development.
Statistical Analysis Plan
The SAP departs from the usual clinical trial SAP in that no inferences will be made regarding the interventions tested in the requested studies. This is an exploratory study using machine learning approaches aiming to evaluate associations between laboratory tests and disease-specific clinical outcomes. Analyses will be performed separately on data from studies in ulcerative colitis and Crohn’s disease. There will be three main analytical objectives.
1. Unsupervised learning will be used in an effort to uncover distinct patient phenotypes (clusters) based on routine laboratory tests at baseline and patterns of their changes post-baseline. Patterns of lab values over time in the identified clusters will be summarized; association of these patterns with disease-specific outcomes will be explored.
2. Several feature selection strategies will be used in order to identify lab tests having high predictive power with respect to endoscopic and disease-specific outcomes. Methods such as penalized regression, recursive elimination algorithm, principal component analysis, and random forest variable importance will be considered, and results from different algorithms will be compared for consistency. Presence of important interactions between individual lab tests will also be explored, e.g., using penalized regression and gradient boosting machines. We will explore predictive power of test results concurrent with the modelled clinical and endoscopic outcomes as well as temporal lagged associations. The results of the feature selection analysis will be interpreted together with insights obtained from unsupervised learning.
1.3. Several supervised machine learning approaches will be used to train predictive models of clinical and endoscopic outcomes based on laboratory values as predictors. Methods such as penalized logistic regression, support vector machines, random forest, and gradient boosting machines will be considered. Model selection, including feature selection described in (2), will be validated using appropriate techniques, such as cross-validation. Performance of the final models will be tested on a hold-out test dataset not used for training and model selection. The final model performance will be characterized by metrics appropriate for the prediction task at hand, such as root mean squared error, adjusted R2 (for regression), specificity, sensitivity, and AUC (for classification).
Response to request for information (2/19/2019):
1.Please provide the reasoning behind/criteria used for selecting a specific study (ie. search criteria)
We were interested in recent global multicenter studies with a large population of phase 3 and phase 2 patients in Crohn’s disease and ulcerative colitis. It was important to us that the studies, where appropriate, used central reading for the determination of endoscopy related endpoints. The vedolizumab trials in ulcerative colitis fulfilled these criteria and provide data across multiple geographic regions, which would allow us to assess variability. We understand that earlier Phase 2 studies in ulcerative colitis that we requested may not have used central endoscopy reading; they will be used to assess the variability with and without central reading services within the same clinical program.
Requested studies in Crohn’s disease represent the most recent program in this therapeutic area available on Vivli with sample sizes that would be appropriate for machine learning applications.
In both therapeutic areas, we are requesting both Phase 3 and Phase 2 studies to have a sufficiently large dataset in each therapeutic area to divide it into a subset used for training the models and a subset used for testing the models.
There are other more recent trials fulfilling these criteria, however, they have not been deposited on Vivli or elsewhere.
We have also refined our selection and removed the study NCT01177228 from our request as it does not have endoscopic endpoints of interest.
2. What is your approach for handling missing values?
We are interested in the correlation of commonly obtained laboratory data with endoscopic and patient reported outcomes. Inference whether the active drug is better than placebo is not our objective. Consequently, we will not attempt to impute data at time points where all or most parameters of interest are missing.
However, at time points where only some parameters are missing while others have been collected, we will use appropriate imputation techniques and perform sensitivity analyses. Information about treatment status at the time points where missing values occur maybe used to select clinically plausible imputation strategies.
3.How will differences in outcome measures used across studies be handled?
Different outcome measures are for our research project not a hindrance but an opportunity. Our aim is to find out how high dimensional laboratory data correlate with different outcome instruments, endoscopic, patient reported, in Crohn’s disease and in ulcerative colitis. We intend to work not only with binary remission and response endpoints (which might have been defined using different criteria), but also with the underlying numerical scores, e.g., CDAI index and Mayo score, as we all their individual components. Therefore, we are requesting data at a granular level (including component measures of composite endpoints) and not only at the level of remission and responder endpoints.
4.How will the differences in study designs be handled?
The team is very well aware of the different study designs, please see appendix listing the different study designs, links to relevant literature, clinical trials.gov, and FDA reviews, where appropriate. We see different study designs, for example, the differences between the Japanese trial NCT02039505 and the trials conducted elsewhere as opportunity rather than a liability. As indicated in response to question 2, our primary focus is on association between laboratory data with other clinical outcomes, and the differences in schedules of assessments across different studies does not present a problem for this objective. See also question 3 regarding differences in outcome measures.
Requested Studies
Sponsor: Takeda
Study ID: NCT00783692
Sponsor ID: C13007
Sponsor: Takeda
Study ID: NCT01224171
Sponsor ID: C13011
Sponsor: Takeda
Study ID: NCT00783718
Sponsor ID: C13006
Sponsor: Takeda
Study ID: NCT02039505
Sponsor ID: MLN0002/CCT-101
Sponsor: Takeda
Study ID: NCT00655135
Sponsor ID: L299-016
Sponsor: Takeda
Study ID: N/A
Sponsor ID: M200-022
Summary of results
Vedolizumab Deep Dive Overall Report:
Overall, the correlations between fecal calprotectin and the mayo scores grow over time. At baseline patients must have a mayo score with the range of the trial to quality. This could be the reason for low correlation at baseline. As the trial progresses the change in mayo score correlates with the change in fecal calprotectin.