Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Citations Over TimeTop 1% of 2020 papers

Abstract

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long-standing concern when applying trial results to real-world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real-world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.

Related Papers

• → Generalizing Generalizability in Information Systems Research(2003)1,544 cited
• → External Validity, Generalizability, and Knowledge Utilization(2004)251 cited
• → Applying Generalizability Theory using EduG(2011)108 cited
• → Reliability of observers' subjective impressions of families: A generalizability theory approach(2012)20 cited
• → Using Generalizability Theory for the Estimation of Reliability of a Patient Classification System(1994)1 cited