News & Events

Vivli Researcher Spotlight: Dr. Yizhe Xu on analyzing clinical trial data to inform development of machine learning tools

Yizhe Xu is a Postdoctoral Researcher at Stanford Center for Biomedical Informatics Research, Stanford University. Dr. Xu’s team submitted a research proposal to access Vivli to conduct analysis relevant to their topic, “Applying machine learning tools to personalize dabigatran treatment decisions”. The team’s completed research has been presented to the research community at conferences and in publications including the Journal of Biomedical Informatics. She sat down with Vivli to tell us more about accessing individual participant data to advance her research, and the potential for machine learning tools to support more accurate estimation and evaluation of heterogeneous treatment effects.

Tell us more about your research. What is the current state of management in your public disclosure topic?
Our final paper has been published in the Journal of Biomedical Informatics as of July 2023.

What led you to want to research this topic?
First of all, treatment effect heterogeneity is an important question that informs clinical decision making given the fact that treatment effects often vary across patients. Thus, accurate estimation of individual treatment effects helps to tailor treatment to patient characteristics and maximizes their benefits. However, it has been realized by a wide group of researchers that estimating treatment heterogeneity is challenging, so we summarized the best practices and advanced methodologies and showed a case study on how to carefully estimate heterogeneous treatment effects using the RE-LY and RELY-ABLE trials.

What difference do you hope your research might make, either in the field or for patients? How has it moved forward the treatment of patients?
We hope our case study provides clear instructions and serves as a concrete example for clinical researchers, and that by following our suggestions, they will be able to avoid possible false discoveries of treatment heterogeneity and prevent misleading findings. The improvement of research quality will directly benefit everyday clinical care in the sense that patients will truly benefit from personalized treatment selection if there is treatment heterogeneity and can be estimated reasonably well. On the other hand, we can save the clinicians’ time and efforts on considering personalized treatment when the treatment effect is essentially uniform across patients.

How could your findings be used in future clinical trials in your disease area?
The statistical methods we have summarized and the guidance we provide on how to select a method and evaluate the model performance can be applied to clinical trials in any disease area. However, for observational studies, practitioners need to consider adjusting for confounding, for instance, using methods such as propensity score matching or weighting.

How did the data you accessed through Vivli help you in answering your research question?
Very well. The RE-LY trial enables a case study for us to demonstrate the principled approach we proposed for estimating heterogeneous treatment effect in a real study. The RE-LY study has a large data size, and the fact that it is an RCT helps to simplify the task of treatment effect estimation.

What was your experience like in the process of requesting data using the Vivli platform?
It was an okay experience – we had some difficulties in resolving issues related to the DUA, which made us wait for quite a while, but we were able to get the access eventually.

Would you use the Vivli platform again? Would you recommend Vivli to others? What improvements would you recommend?
Yes, especially if I think some of the unique data sources on the Vivli platform will help to answer particular research questions of my interest. I will also recommend Vivli to others for the same reason. I would recommend simplifying the data requesting process to shorten the waiting time, as well as expanding user autonomy – particularly that users do not need to make a request every time they want to export results.

What advice would you give to other researchers about doing this kind of analysis?
We encourage researchers to understand their data first, then select the most suitable statistical approaches based on that knowledge. After that, we suggest interpreting findings based on the results from multiple estimators that are weighted by their performance, which is evaluated using several different metrics. Please also see our paper for our detailed recommendations.

 

Interested in finding out more about how access to Vivli’s data repository can help advance your research? Find out more about how to search and request data.

Vivli Researcher Spotlight: Dr. Elena Myasoedova on accessing clinical trial data to advance research and the possibilities for using machine learning as a tool in clinical support for people with rheumatoid arthritis

Elena Myasoedova, M.D., Ph.D., is a clinical rheumatologist with specialty interest in inflammatory arthritis. She is an associate professor of medicine and epidemiology and clinical practice, and leads research in rheumatology and specifically inflammatory arthritis at the Mayo Clinic. Dr. Myasoedova’s team submitted a research proposal to access Vivli to conduct analysis relevant to their topic, “Individualized Prediction of Treatment Response to Methotrexate in Patients with Rheumatoid Arthritis: A Machine Learning Approach”. Their completed research has been presented to the rheumatology research community at conferences and in publications including Annals of the Rheumatic Diseases. She sat down with Vivli to tell us more about accessing clinical trial data to advance her research, and the possibilities for using machine learning as a tool in clinical support for people with rheumatoid arthritis.

Please tell us a little bit more about your research – what led you to want to investigate this particular topic?
In patients with rheumatoid arthritis, methotrexate is a medicine that is used very commonly, and frequently is the first medicine and the most common medicine used in combination later on during the disease course. The challenge that we are facing in rheumatology in general is that we do not have individualized prediction of response to treatments. This means that we use a trial and error approach to treatments; we start patients on medications that are generally effective and safe, and then if a patient does not respond, we upgrade to a different medicine. Because most of the medicines that we use take weeks and months to show their effects, it’s important to understand early on if a person is likely to be a responder. That would help to save time and potentially save some unwanted side effects for the patient, and also help us to be more proactive and helpful.

For this specific study, we looked at clinical markers or clinical predictors of response to methotrexate. We found more than 1400 patients from 13 randomized trials. A total of 775 patients from 4 RCTs were included in the study, and we monitored their response across a six-month timeframe. We further evaluated whether people who did not respond to methotrexate and had some moderate to high disease activity at 12 weeks – who out of this population would respond by follow-up at 24 weeks. We also found a couple of markers for that: specifically if there was a drop in DAS28 sat rate from baseline to 12 weeks of at least 1 point, then it was predictive of then achieving remission or low disease activity at 24 weeks. Otherwise the chances were slim.

We developed this algorithm, and externally validated it using two independent trials with good results. I think that these findings advance the understanding of who are the most likely responders, and how we should discuss with patients their likelihood of response at the very start of their treatment.

Are you hoping that this is going to change the clinical approach? Has it already had an impact in clinical approaches to working with people who have arthritis?
This particular study, and similar studies would probably change the way we discuss this treatment with a patient; changing the treatment approach is a very complex task that probably has to come through the association guidelines.

Is there anything specific that you’d like to say about what working with the dataset in Vivli enabled you to do that you might not have been able to do otherwise?
It was actually a very good experience for us to work with Vivli datasets. It provided longitudinal data on patients who were users of methotrexate but not other medicines, and there were hundreds of these patients available from the trials. So it’s a fairly good dataset to work with, and it had multiple data points longitudinally. Also, at each point, there were multiple clinical points collected, so it was fairly comprehensive.

Do you have any advice that you would give to other researchers who might be interested in accessing this type of data using a platform like Vivli?
The most important advice is to put together a comprehensive proposal with a plan right off the bat, to make sure that the timeline is feasible and that the question that they want to address is feasible with the available data – just to make sure that they are not over-expecting or under-planning. I think it’s most important to ensure that the study question matches the data set.

Have you had feedback about your research and findings?
This research has been presented at several conferences, and the comments have mostly been positive, acknowledging the need for developing such algorithms.

Webinars, Publications & Presentations about Data Sharing

VIVLI WEBINARS

View Vivli’s webinars at your convenience by clicking on the links below:

DateWebinarSpeakers
April 2023Recent Developments in Data Privacy Laws and their Impact on Data SharingRebecca Li, Vivli
David Peloquin, Ropes & Gray
Katherine Wang, Ropes & Gray
February 2023Submitting Your NIH Data Management and Sharing Plan using VivliAnne Seymour, Johns Hopkins University
Amy Nurnberger, Massachusetts Institute of Technology
John Borghi, Stanford University
Rebecca Li and Julie Wood, Vivli
November 2022Applying the SAFE Data Standard to Securely Share Clinical Trial DataLuk Arbuckle, Chief Methodologist, Privacy Analytics (an IQVIA company)
Stephen Bamford, Head of Clinical Data Standards & Transparency, Janssen Research and Development
Aaron Mann, CEO, Clinical Research Data Sharing Alliance

Moderated by:
Marcia Levenstein, Senior Advisor, Vivli
October 2022Mind the Data Sharing Gap: Navigating sponsor policies and data protection methodologiesLuk Arbuckle, Chief Methodologist, Privacy Analytics (an IQVIA company)
Liz Roberts, Senior Director, Data Policy and Privacy Lead, UCB

Moderated by:
Aaron Mann, CRDSA CEO
June 2022How Vivli Promotes Discoverability of Partner Platforms and RepositoriesIshwar Chandramouliswaran, NIH
Dawei Lin, ImmPORT
Ida Sim, UCSF and Vivli
June 2021Sharing Academic Clinical Research DataIda Sim, UCSF and Vivli
Dan Ford, Johns Hopkins University
Susanna Naggie, Duke University
Kim Serpico, Harvard T.H. Chan School of Public Health
Ara Tahmassian, Harvard University
March 2021Clinical Trial Data Sharing and Machine Learning ResearchIda Sim, UCSF and Vivli
Abigail Gregor, Ropes & Gray
Daniel Freshman, Ropes & Gray
September 2020Synthetic Data: How to preserve privacy of participants when sharing clinical dataPatrick Cullinan, Bluebird Bio
Khaled El Emam, Replica Analytics
Michael Lesh, Syntegra
July 2020Accelerating Science in the Age of COVID-19: Three Key Data InitiativesRebecca Li, Vivli
Elizabeth L. Ogburn, Dr. Barbara Bierer, COVID Collaboration Platform
Janice Chang, Cara Rinaldi, TransCelerate BioPharma, Inc
April 2020 Data Sharing and Data Anonymization: the Vivli-Privacy Analytics Partnership Rebecca Li
Niamh McGuinness
February 2020Future Directions: Real World Data, Real World Evidence and Clinical TrialsDr. Ida Sim
Marcia Levenstein
Dr. Gregory Pappas
Dr. Jack Mardekian
November 2019Credit for Data SharingDr. Barbara Bierer,
Ms. Heather Pierce
October 2019Why Data Sharing and Data Standardization MattersMr. David Bobbitt MSc, MBA
Dr. Ida Sim, MD, PhD
Dr. Rebecca Li
May 2019Top five questions small and mid-size companies should ask before embarking on a data sharing programDr. Rebecca Li
March 2019 Keys to Submitting a Quality Research Proposal to a Data Sharing PlatformMs. Cynthia Holas,
Dr. Sonali Kochhar,
Dr. Georgina Humphreys,
Dr. Joe Ross,
Ms. Ginger Gamble, MPH.

February 2019 Informed Consent and Data Sharing Dr. Barbara Bierer ,
Dr. Rebecca Li ,
Mr. David Peloquin,
Dr. Stephen Rosenfeld
January 2019How to Share and Request Data on VivliDr. Ida Sim, MD, PhD
November 2018 IPD Meta-Analysis Webinar Dr. Sarah Nevitt

PUBLICATIONS ABOUT DATA SHARING

Data Sharing Goals for Nonprofit Funders of Clinical Trials

Coetzee T, Ball M, Boutin M, Bronson A, Dexter DT, English RA, Furlong P, Goodman AD, Grossman C, Hernandez AF, Hinners JE, Hudson L, Kennedy A, Marchisotto MJ, Myers E, Nowell WB, Nosek BA, Sherer T, Shore C, Sim I, Smolensky L, Williams C, Wood J, Terry SF, Matrisian L
Data Sharing Goals for Nonprofit Funders of Clinical Trials J Particip Med 2021;13(1):e23011 doi: 10.2196/23011 PMID: 33779573
COVID-19 trials: declarations of data sharing intentions at trial registration and at publication

Li, R., von Isenburg, M., Levenstein, M. et al. COVID-19 trials: declarations of data sharing intentions at trial registration and at publication. Trials 22, 153 (2021). https://doi.org/10.1186/s13063-021-05104-z
Timely access to trial data in the context of a pandemic: the time is now.

Li R, Wood J, Baskaran A, et al. Timely access to trial data in the context of a pandemic: the time is now. BMJ Open 2020; 10:e039326. doi: 10.1136/bmjopen-2020-039326
Move clinical trial data sharing from an option to an imperative

New NonProfit Aims To Bring Data Transparency To Researchers

Sharing Health Data: The Why, the Will, and the Way Forward A Special Publication from the National Academy of Medicine

Whicher, D., M. Ahmed, S. Siddiqi, I. Adams, M. Zirkle, C. Grossmann, and K. L. Carman, Editors. 2021. Health Data Sharing to Support Better Outcomes: Building a Foundation of Stakeholder Trust. NAM Special Publication.Washington, DC: National Academy of Medicine.
COVID-19 interventional trials: Analysis of data sharing intentions during a time of pandemic

Kristina Larson, Ida Sim, Megan von Isenburg, Marcia Levenstein, Frank Rockhold, Stan Neumann, Catherine D'Arcy, Elizabeth Graham, David Zuckerman, Rebecca Li. COVID-19 interventional trials: Analysis of data sharing intentions during a time of pandemic. Contemporary Clinical Trials. 2022.
Health Data Sharing Platforms: Serving Researchers through Provision of Access to High-Quality Data for Reuse

Rebecca Li, Nina Hill, Catherine D’Arcy, Amrutha Baskaran, Patricia Bradford, "Health Data Sharing Platforms: Serving Researchers through Provision of Access to High-Quality Data for Reuse", Health Data Science, vol. 2022, Article ID 9768384, 3 pages, 2022

PRESENTATIONS

May 10-11, 2021"Considerations for Sponsors when using a Data Sharing Platform" and panel, "Balancing Sharing Results and Data Whilst Protecting Participants Data Privacy" at InformaConnect Clinical Data Disclosure, Transparency & Plain Language SummariesJulie Wood
Rebecca Li, PhD
May 5, 2021"Data Sharing in a Time of Pandemic", a webinar with Front Line GenomicsRebecca Li, PhD
April 12, 2021"Implementation of Data Sharing platforms and How Researchers are Utilizing these Platforms to Further their Research.", a webinar with the HRB-Trials Methodology Research NetworkRebecca Li, PhD
March 26, 2021"Acceleration of Research and Implications for Research Transparency." Center for Biomedical Research Transparency Summit SeriesIda Sim, MD, PhD
January 19, 2021"Data Sharing of Rare Disease Data – Challenges and Rewards." PHUSE Data Transparency Winter MeetingRebecca Li, PhD
December 11, 2020 "Vivli: A Global Clinical Trials Data Sharing Platform." dkNET webinarIda Sim, MD, PhD
November 6, 2020"Responsible Data-Sharing to Improve Research Integrity." European Medical Writers Association 2020 Virtual Symposium Julie Wood
October 16, 2020"Privacy Incoherence in Medicines: The Confluence of Corporate Trade Secret, Government Non-Transparency, and AI Mediated Erosion of Patient Privacy." Northeastern University's Center for Law, Innovation and Creativity Annual ConferenceRebecca Li, PhD
August 18, 2020"Platform Analytics Tools to Support Rare Disease Drug Development." FDA convening hosted by the Duke-Margolis Center for Health PolicyRebecca Li, PhD
April 24, 2020"Sharing, Discovering and Citing COVID-19 Data and Code." National Library of Medicine (NLM) at the National Institutes of Health (NIH)
Ida Sim, MD, PhD
February 11, 2020"Vivli Clinical Research Data Sharing: Share. Discover. Innovate." NIH Workshop on the Role of Generalist Repositories to Enhance Data Discoverability and Reuse, Bethesda MD
Ida Sim, MD, PhD
October 25, 2019 “Finding New Solutions to Problems and Concerns in Clinical Data Sharing – Outcomes from Datathon."DIA Future of Evidence WorkshopRebecca Li, PhD
October 10, 2019 "Clinical Trial Data Sharing and Reuse: A New Reality for Researchers." Clinical Data Interchange Standards Consortium WebinarIda Sim, MD, PhD
Rebecca Li, PhD
September 27 2019 "Preparing for clinical trial data sharing and re-use: the new reality for researchers." NIH Collaboratory Grand RoundsRebecca Li, PhD
Frank Rockhold, PhD
September 19, 2019“The Vivli Experience in Sharing Clinical Trial Data Globally.”OHRP Exploratory Workshop: Privacy & Health Research in a Data-Driven WorldRebecca Li, PhD
June 2019“Symposium SY13: Data sharing and responsible conduct of research: sharing industry experiences as part of the research transparency environment."World Conference in Research Integrity, Hong KongRebecca Li, PhD
March 27, 2019“Vivli: A Global Secure Data-Sharing Platform for Participant-Level Clinical Trial Data." Medical Informatics Association Summit, San Francisco CAIda Sim, MD, PhD
January 30, 2019“The Disclosure and Data Sharing Lifecycle.” CBI Clinical Data Disclosure and Transparency Conference, Philadelphia PAJulie Wood
Thomas Wicks (TrialScope)
October 25, 2018 "Sharing and Using Deidentified Individual Participant Data (IPD)." DIA Global Clinical Trials Transparency Conference, LondonRebecca Li, PhD
March 13, 2018“Vivli: A Global Secure Data Sharing Platform for Participant-level Clinical Trial Data.” American Medical Informatics Association Summit, San Francisco CAIda Sim, MD, PhD
November 17, 2017 “Acceleration of Innovation to Overcome Intractable Diseases." Academic Research Organization’s 2nd Global Network Workshop, Austin TXIda Sim, MD, PhD