University:
Boston University
Course:
CAS MA 511 | Introduction to Analysis I
Academic year:

2024
Views:

189

Pages:

23
Author:

Annabel Combs

Power Calculations Introduction • A statistician’s role in a study begins well before data are ready for analysis • Statistician assists in the “design” of the study prior to study start – Major design component: Determining appropriate sample size for the study •i.e., determine sample size large enough to represent population with reasonable confidence; •while not having a sample size be cumbersome from cost and/or time perspective. Superiority • Suppose have two treatment groups that we wish to compare on a dichotomous outcome in a clinical trial: Ho: pE ≥ pC (“Null” hypothesis) H1: pE < pC (“Alternative” hypothesis – what we hope to prove) where pE and pC are the TRUE (but unknown) event rates of a negative outcome (e.g., Major Adverse Event) for experimental and control groups IN THE POPULATION. Introduction • We hope the null hypothesis is false in the population • We base our decision about whether the null hypothesis is false (whether to reject H0) on a sample from the population. • We want a sample large enough to represent the population with reasonable confidence (there’s always a chance we’ll obtain a sample not representative of the population, but we want to minimize this chance). Introduction “TRUTH” (Unknown): Ho True Ho False Decision: Accept Ho Reject Ho CORRECT DECISION Type II ERROR Type I ERROR CORRECT DECISION Introduction “TRUTH” (Unknown): Ho True Ho False β= P(Type II Error) Decision: Accept Ho Reject Ho CORRECT DECISION Type II ERROR Type I ERROR CORRECT DECISION α= P(Type I Error) (controlled by significance level; usually set to one-sided 0.025 for pivotal trials). =P(accept H0|H0 False) Power = 1 - β =P(reject H0|H0 False) (set to at least 0.80 for Confirmatory studies) β and Power • As α decreases, β increases (and power decreases) • As sample size (n) per group increases, power increases (if the null hypothesis is truly false) – For a confirmatory study: choose desired power; determine n per group • Power also depends on the true treatment effect Example Ho: pE ≥ pC (“Null” hypothesis) H1: pE < pC (“Alternative” hypothesis – what we hope to prove) Sample Size • How do we come up with our assumptions of the “true” mean effect (i.e., true mean difference)? – Past Studies – Literature review (maybe someone else did similar work and published it) – Best guess? – What we “hope” is the effect (even though we may not have scientific evidence yet) •Assumption should be “clinically” meaningful Underpowering • We do not want to “underpower” our study – I.e., we don’t want a sample size that is too small. • This can happen if, when powering the study, we assume the effect size (e.g., treatment difference) is larger than it truly is Underpowering • E.g., if we powered previous example assuming an experimental treatment 10% MAE rate and a control treatment 20% MAE rate, we need 199 per treatment group • If, in reality (and unbeknownst to us) the true MAE rates are 10% and 17.5%, then in reality 199 per group yields only 58% power. – But at least we still have some power Evaluable vs. All Randomized • The above sample size calculations yield number of evaluable subjects – i.e., subjects who will not prematurely withdraw and hence be available for follow-up – Add a certain % of patients to number of evaluable to obtain number of patients who should be enrolled – e.g., if 10% dropout rate is anticipated, enroll 199/0.90 = 222 patients/group. Continuous Outcome • A study is conducted to assess the effect of a new stem cell treatment on LVEF improvement at 4 months following acute MI • Control group is a placebo (sham control procedure) • H0: µT ≤ µPL vs. H1: µT > µPL where µT and µPL are mean improvement in LVEF % from baseline to 4 months for experimental treatment and placebo Continuous Outcome • Assumptions made in powering study: – True µT - µPL = 5% – Standard deviation of change in LVEF = 8% in each group – Desire power of 80% – One-sided significance level of 0.025 • Note: We don’t need assumption of the true means; just the difference in means • Note: We need an assumption of the variability (the more variability, the larger the sample size needed) Assumptions Needed for Common Statistical Tests Objective Show superiority of one treatment over another on binary outcome rate Methodology Assumptions Needed for Power Chi-square test Event rate in each treatment (or Fisher's Exact test) Show superiority of one treatment over another on mean of outcome Show that increase in a variable X increases/decreases the odds of the outcome event Two-sample ttest Univariate Logistic Regression Difference between means; sd per treatment Odds Ratio, binary outcome rate when X=0 (if X is binary) or at mean of X (if X is continuous) Show that increase in a variable X increases/decreases the odds of the outcome event, after adjusting or other coariates Adjusted Logistic Regression Odds Ratio, binary outcome rate when X=0 (if X is binary) or at mean of X (if X is continuous), Rsquared of X with other covariates Show that increase in a variable X Simple Linear increases/decreases the average outcome Regression of Y Slope (average change in Y per 1 unit increase of X), sd of X, sd of Y or correlation of X and Y Show that increase in a variable X Multiple Linear Slope (average change in Y per 1 increases/decreases the average outcome Regression unit increase of X), sd of X, sd of Y of Y, after adjusting for other covariates or correlation of X and Y, Rsquared of X with other covariates Multiple Primary Analyses • Suppose there are two primary hypotheses; e.g.: H0: pE,SAF≥pC,SAF H0: pE,EFF ≤ pC,EFF H1: pe,SAF pC,EFF (SAFETY) (EFFICACY) • Suppose we need to reject BOTH nulls for the study to be considered successful – Each can be tested at a 0.05 level of significance – Power each endpoint at 80% could yield only 64% power to reject BOTH null hypotheses – We need to power each null with at least 90% power to have an 80% chance of rejecting BOTH null hypotheses Multiple Primary Analyses • Suppose there are two primary hypotheses; e.g.: H0: pE,SAF≥pC,SAF H0: pE,EFF ≤ pC,EFF H1: pe,SAF pC,EFF (SAFETY) (EFFICACY) • Suppose study is a success if we reject AT LEAST ONE null hypothesis – Powering each hypothesis at 80% at alpha = 0.025 (=0.05/2) yields 80% power to reject at least one null hypothesis