University:
Boston University
Course:
SAR HP 151 | Introduction to Health Professions
Academic year:

2023
Views:

138

Pages:

19
Author:

Barnaboneoicj

Design considerations for pilot studies in clinical and translation research University Slideshow Title Goes Here • What are “pilot studies”? • Pilot studies have been defined as small-scale tests of methods and procedures” to be subsequently used “on a larger scale” Boston University Slideshow Title Goes Here 2 Example • In leading to a randomized clinical trial (RCT), a pilot study can be used to evaluate • Feasibility of study procedures including recruitment, randomization, retention, assessment, novel methods, and/or implementation of the novel method. Boston University Slideshow Title Goes Here 3 • Data from a pilot study should NOT be used to test hypotheses!! • The trial subsequent to a pilot uses sample data to make inference about efficacy or effectiveness of a treatment or intervention IN THE POPULATION, and does so with a focus on minimizing bias by employing randomization, blinding, and the use of a control group or condition. Boston University Slideshow Title Goes Here 4 • Why should a pilot study be conducted? • To examine feasibility, acceptability, or safety of a treatment or intervention, or a proof of concept. • Feasibility • Evaluate study procedures and intervention design in a “test” phase, altering the intervention if needed, in advance of the conduct of the subsequent trial. Boston University Slideshow Title Goes Here 5 Should a pilot study include a control group? University Slideshow Titlewill Goes Here • Boston If no inference be made with regard to the population treatment effect, why include one? • The use of a control group allows for a more full evaluation of study processes than would be the case • in a single arm study. In particular, in behavioral research, the use of a control group allows for proper evaluation of feasibility, consistency, and acceptability of the intervention. 6 • Pilot studies also allows for the development of good clinical practices in the conduct of research, for example, by providing opportunities for staff training. Boston University Slideshow Title Goes Here 7 • How should one plan the sample size for a pilot study? • Without hypothesis testing, one can still base sample size on pragmatics of recruitment (for example, patient flow, budgetary constraints) and the evaluation of feasibility. Boston University Slideshow Title Goes Here 8 • Thus, preliminary endpoint data need not be available for the planning of sample size for a pilot study (often encountered in bench science studies of new measures), though a theoretically based rationale is required to support the aims of a pilot study and with awareness of the goals of the subsequent study. Boston University Slideshow Title Goes Here 9 • Although there may be temptation to do so, pilot data should never be included in the sample analyzed in the subsequent confirmatory study in which hypotheses are tested!! Changes in the protocol around an intervention may add a source of unaccounted for variation. Boston University Slideshow Title Goes Here 10 Why should hypothesis tests NOT be conducted using data from a pilot study? Boston University Slideshow Title Goes Here • The study methods, including the intervention itself, will typically not be fully developed at the outset of the pilot and • The sample size of the pilot, if based on considerations of patient flow and budget, is likely to be too small for appropriate hypothesis testing. One may base the sample size for a pilot on relevant measures in order to result in confidence intervals sufficiently narrow to provide an acceptable degree of precision. 11 Boston University Slideshow Title Goes Here • Pilot studies can, however, provide important information about the safety of a treatment or intervention. If the pilot study is focused on safety, adverse event rates should be reported using accepted methods for phase I and phase II RCTs that are consistent with CONSORT guidelines from 2010 regarding the reporting of results from pilot studies. 12 Boston University Slideshow Title Goes Here • The size of the effect of treatment estimated from a pilot study should NOT be used to plan future confirmatory studies (Kraemer). Why? • The pilot study will have reduced precision because of the inherently small sample size AND because the differences in outcomes between the intervention and control conditions could be inconsistent with effects in the subsequent study. 13 Boston University Slideshow Title Goes Here • The effect size from a pilot study could be smaller than in the future study because the intervention was developed during the conduct of the study. It could also be larger than in the future study because focus may be placed on the most optimistic result in favor of the intervention. 14 Boston University Slideshow Title Goes Here • Some have argued that some use of hypothesis testing of treatment effects in pilot studies is justified as long as the lack of precision is acknowledged and tests should be conducted with higher Type I error thresholds than in the subsequent confirmatory study (Schoenfeld proposed using an alpha of 0.20). Some caution, however, in this practice is recognized given that the 95% confidence interval for the true population effect size when based on pilot data will typically include both positive and negative effects of intervention. 15 Boston University Slideshow Title Goes Here • What data from a pilot study should be used in planning the sample size of a future study? Kraemer suggests using estimates only from the control group or condition in the pilot study. These estimates then provide a base on which clinically meaningful effects of the intervention can be hypothesized and guide the computation of sample size and power. 16 Boston University Slideshow Title Goes Here • This practice should be conducted with caution given that estimates of the standard deviation for the outcome of interest may be too small in practice when based on the small sample size of a pilot study, notwithstanding that SDs are in theory invariant to sample size. 17 Boston University Slideshow Title Goes Here • The implications are clear for the use of estimated effect sizes for treatment from pilot studies: • if the true effect size is underestimated, the subsequent study will enroll more subject than is needed to detect statistically significant effects; or if the true effect size is overestimated, the sample size of the subsequent study will be too small and the study is likely to be stopped for futility. Both of these situations are unethical. 18 Summary • “Keep the next study in mind” • In summary, the aims and methods of a pilot study need to be properly aligned with the goals of the research program and the plan of the subsequent study. Boston University Slideshow Title Goes Here 19

Design Considerations for Pilot Studies In Clinical And Translation Research

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock