Evaluating predictive modeling algorithms to assess patient eligibility for clinical trials from routine data

Language
en
Document Type
Article
Issue Date
2014-02-12
Issue Year
2013
Authors
Köpcke, Felix
Lubgan, Dorota
Fietkau, Rainer
Scholler, Axel
Nau, Carla
Stürzl, Michael
Croner, Roland
Prokosch, Hans-Ulrich
Toddenroth, Dennis
Editor
Abstract

Background The necessity to translate eligibility criteria from free text into decision rules that are compatible with data from the electronic health record (EHR) constitutes the main challenge when developing and deploying clinical trial recruitment support systems. Recruitment decisions based on case-based reasoning, i.e. using past cases rather than explicit rules, could dispense with the need for translating eligibility criteria and could also be implemented largely independently from the terminology of the EHR’s database. We evaluated the feasibility of predictive modeling to assess the eligibility of patients for clinical trials and report on a prototype’s performance for different system configurations.

Methods The prototype worked by using existing basic patient data of manually assessed eligible and ineligible patients to induce prediction models. Performance was measured retrospectively for three clinical trials by plotting receiver operating characteristic curves and comparing the area under the curve (ROC-AUC) for different prediction algorithms, different sizes of the learning set and different numbers and aggregation levels of the patient attributes.

Results Random forests were generally among the best performing models with a maximum ROC-AUC of 0.81 (CI: 0.72-0.88) for trial A, 0.96 (CI: 0.95-0.97) for trial B and 0.99 (CI: 0.98-0.99) for trial C. The full potential of this algorithm was reached after learning from approximately 200 manually screened patients (eligible and ineligible). Neither block- nor category-level aggregation of diagnosis and procedure codes influenced the algorithms’ performance substantially.

Conclusions Our results indicate that predictive modeling is a feasible approach to support patient recruitment into clinical trials. Its major advantages over the commonly applied rule-based systems are its independency from the concrete representation of eligibility criteria and EHR data and its potential for automation.

Journal Title
BMC Medical Informatics and Decision Making
Volume
13
Issue
134
Citation

BMC Medical Informatics and Decision Making 13.134 (2013): 10.02.2014 http://www.biomedcentral.com/1472-6947/13/134

DOI
Zugehörige ORCIDs