Hypothesis Testing Cheat Sheet

Hypothesis Testing 1. Assumptions: Data comes from randomization Shape of population distribution is normal Sample size large enough 2. Hypothesis: H0: null hypothesis H1: parameter falls in alternative range of values 3. Test statistic: How far point estimate falls from the parameter value given in null hypothesis Measured by number of standard errors between point estimate and parameter value in H0 4. Interpreting test statistic: Presume H0 is true But if sample test statistic falls out in the tail of sampling distribution, value is unlikely P-value is the probability of “what we saw or something more extreme” given H0 is true Significance level: a number such that we reject H0 if p-value is less than or equal to that number if we reject H0, the results are statistically different Hypothesis Testing for proportions 1. Assumptions: Variable collected is categorical Data comes from randomization - 2. 3. Sample size n is sufficiently large that the sampling distribution of sample proportion ^ p approximately normal if np0(1-p0) Hypothesis H 0: p = p 0 H 1: p ≠ ≥ 2. 3. is 5 - 4. ) when H0 is true - ,Z Dependent sample test -Testing if μ1 = μ2 1.Construct new variable -DI=XI-YI 2.Hypothesis: -H0: μD =0 3.Test statistic: - T= ´X−μ0 s / √n with n-1 degrees of freedom 4.Pairing up reduced standard error in D́ ANOVA: Compare more than 2 Group means 1. Assumptions: Population distribution for the response for k groups are normal Largest group sd less twice of smallest sd Data randomized N observations in total: n observations in k group, N=nk 2. Hypothesis: H0 : μ1= μ2=μ k 3. H1: At least 2 means are not equal Test statistic: Involves within grp & √ H0 : μ=μ 0 Test statistic: Distance between sample - p0 p 0(1− p0 ) n Z= sided t-test Hypothesis: - Test statistic: ^p N( p0 , Hypothesis testing for mean (One sample t-test) 1. Assumptions: Variable collected is quantitative Data comes from randomization Population distribution is approximately normal n is small one mean X́ and H0 value of population mean Measured in terms of standard errors ´X−μ0 T= s / √n with n-1 degrees of freedom Conclusion: If H0 is true Mean is not significantly different from p − p0 ^ Normality assumption 1. Histogram with a Normal pdf overlaid Symmetric, unimodal 2. Q-Q plot Standardized sample quantiles against theoretical quantiles of N(0,1) distribution If fall on a straight line, the data is normal Right tail below straight line: fatter Left tail below straight line: thinner Errors in conclusions 1. Type 1 error: occurs if we reject H0 when in fact it is true when H0 is assumed to be true 2. Type 2 error: occurs if we do not reject H0 when it is false When H1 is assumed to be true 3. sample size type 2 4. significance level, type 1 but type 2 2 independent sample t-test, Equal variance 1. Assumptions: Independent samples with randomized experiment Sample standard deviations not twice… Population distribution is approximately normal and n is small 2. Hypothesis: 3. - p-value ¿ a m 2.Tukey: - replacing use of tdistribution for each pairwise distribution with a multiplier from another μ1=μ2 - - Estimate of common s 2p = 2 ( n1−1 ) s 1 +(n2−1 n1+ n2−2 q 0.5=0 3. Test statistic: Sum of the ranks of the difference that were T= ( X́− Ý)−0 ( X́− Ý) −0 - se se= sp √ 4. 1 1 + n1 n 2 Simple linear Regression -F-test equals t-test 1. Assumptions: Random data R/S between X and Y is linear ε N(0,σ 2 ) , normally distributed subpopulation for each explanatory variable and same variance Y N(β 0+ β1 X t-test Hypothesis: √ 2 2 s 1 s2 + n1 n2 Conclusion: If do not reject H0, no statistically difference between means Point estimates ^ β0 (constant) and - ^ β 1 are point estimates - ^ Y is a point estimate of the mean of response at a particular X ^ Y =β 0 + β 1 X 1+ β2 value -when X when reference=female increases by 1 (0) &male (1) unit, mean response 1. Hypothesis: -Use adjusted R2 to compare models Indicator variables - 2. se= Multiple linear Regression -more than one explanatory variable -t-test for individual coefficient(quantitative) -F-test for overall significance (categorical) - se with complicated number of degrees of freedom (df) - - μ1 = μ2 X́ − Ý X́ − Ý T= Kruskal-Wallis -If more than 2 groups of independent samples -Not normal and unequal variance 1. Assume: -Independent observations from k groups where k ¿ 2 -Random data 2.Hypothesis: -H0: The distributions of all groups are the same H0 : Test statistic: Point estimate of difference between the population means variance: μ0 2-sample Wilcoxon rank sum (Mann-Whitney U) 1. Assumptions: Independent observations X1, X2…Xn1 and Y1,Y2…Y n2 from 2 groups Random data Hypothesis: H0: 2 samples are from same distribution 2. Test statistic: Minimum of Rx and RY, RX is the sum of ranks of X- Hypothesis: - 3. with n1+n2-2 degrees of freedom Wilcoxon signed rank For matched pairs 1. Assumptions: 2 groups are independent Data random Difference of observations can be ranked Population distribution of difference is symmetric 2. Hypothesis: H 0: 2. Test statistic: Point estimate of difference between the population means p 0 (1− p0 ) n (0,1) Multiple comparison test -To control Experiment error rate a: the probability of making at least 1 Type 1 error when all m hypothesis are true 1. Bonferroni correction: - Not doing ANOVA - perform each test at a/m significance level - for each CI at (1a/m)100% level H0 : 2 independent sample t-test, Unequal variance 1. Assumptions: Quantitative variable for 2 groups, independent, random Sample standard deviation twice… Approximately normal and n is small - H0: all coefficients=0 H1: at least 1 not zero ^ Y increases by ^ β1 unit Compare within 2 groups -H0: μi =μ j - Btwn grpvaria withingrp vari if F-test is significant - T= Ý i− Ý j se 4. where se= sp √ between grp variability F= 1 1 + ni n j with numerator k-1 and denominator N-k degrees of freedom p-value: always the areas to the right of it; larger F against H0 distribution (a studentized range distribution) - input: no. of k grps; output: all significant pairwise comparisons 3. Dunnett: -k groups with one control, conduct m=k-1 comparisons -same as Tukey but will provide shorter CI positive (W+) If W+ ≈ W-, do not reject H0 For one sample 1. Hypothesis: H 0: - - q 0.5=m0 2. sample while RY is the sum of ranks of Ysample H0 is correct if RX ≈ RY 3.Test stats: -Uses the variability of sample-mean ranks -Under H0, follows a 3. 2 X k−1 Test statistic: -sum of ranks ¿ m0 - - distribution (chi-square) known as H0 : β 1=0 2. and tdistribution follows n-2 degrees of freedom F-test Hypothesis: H0 : Estimating σ σ computed using residuals e i=Y i− -Variance is minimized at X0= X , smaller the better Scatterplot 1. Linearity violated: Add higher order terms in X eg. X2 2. Variance not constant: Transform the response by taking ln(Y), √Y or (1/Y) 1 unit increase in X, mean response ^ Y e by ^ Y ´ with N-k degrees of freedom - 2 -how variable the response measurements are around fitted line 2 - ^ is ^ Y =β 1 + β 1 X 1 + β 2 β 1=0 and F distribution with 1 and n-2 degrees of freedom (W+) R2 Adjusted =0.068 means only 6.8% of variation in the marks change is explained by regression model With interaction term increas ^ β e 1 times Residuals plot Detect nonnormality Check for nonconstant variance and the need to transform Y Check for the need to add higher order terms in X Explanatory data analysis 1.Frequency table: -For categorical or quantitative variable -Modal category: highest frequency Plots to make 1. Plot ri’s against 2. 3. ^ Yi on Plot ri’s against XI Create histogram of ri 4. Create QQ plot using ri  Transform Y when there is funnel shape  Add higher order X term when there is a curve band  Non-normal when points are outside 3 and -3 *Note that ri are not independent Cook’s distance plot To check for influential point R2: The proportion of total variation of the response that is explained by the fitted regression. 4.Boxplot -Minimum, lower quartile, median, upper quartile and maximum -For comparing between quantitative vs categorical variable -Skewness of distributions (mention if obvious) Chi-square test for 2 × 2 with continuity correction 1. Assumptions: 2 categorical variables all expected cell count >5 Data random 2. Hypothesis: H0: 2 Variables are independent H1: 2 Variables are dependent 3. Test statistic: - χ 2=∑ 4. - ( | observedcoun ex P-value: Right tail probability of χ 2 Fisher’s exact 1. Assume: ≥ 20% expected cell count <5 2 binary categorical variables Data random 2. Hypothesis: H0: 2 variables are independent 3. Test stats: The first cell count Linear by linear -There is at least 1 ordinal variable distribution with 1 degree of freedom Pearson chi-square r rows and c columns that define 2 categorical random variables - use Strength of association -How different is P(disease|exposure) is from P(disease|no exposure) ^ p1 = - a a+b c ^ p2 = c+ d - × 2 ^ p2 is a distribution ^ p1 from 2 2 table Relative risk: - ^ p1 ^ p2 -odds= point estimate of P(disease|no exposure) Prospective study: -randomly assign the exposure variable to subjects or record their exposure variable status Retrospective study: -case control study and 2 Odds of success: point estimate of P(disease|exposure) -cannot obtain χ (r −1)(c−1) is Difference of proportion: - ^ p1 − ^ p p p−1 where p is the probability of success and p-1 is failure -odds ratio= a/ b ad = c / d bc -The odds of having disease in expose is n times the odds of having disease in unexposed Computing CI for OR -estimated se for ln(OR) is se(ln(OR))= √ 1 1 1 1 + + + a b c d -Formula for a (1- α )100% CI: e ad ln⁡( bc)± q1−α /2 √ 1 1 1+ + + a b c Logistic regression -Response variable Y is a binary random variable (categorical) 1.Assumptions: -Data is randomized -Observations are independent -Denote pi=P(Yi=1) (success probability) given a value of Xi 2.Odds of success: - P (Y i=1) pi = P( Y i=0) 1− pi pi - ln ( )= 1−p i β 0 + β1 X i pi - ln ( )= 1−p i β 0 + β1 X i 1 + β 2 X i 2 + 3.Distributional assumption: - Y i Bin (1, pi ) β +β X e pi = 1+ e β +β X 1 0 - 0 1 1 1 Maximum likelihood estimation: -Test fitness of model using R2 Model interpretation -For every increase in XI, the ln β1 odds increase by e β1 times Wald Hypothesis Test -Test if an individual term is significantly different from zero 1. Hypothesis: β 1=0 -H0: 2. Test statistic: - 1 ^ β¿ s .e(¿ ¿) ^ β1 ¿ ¿ 2 Probability Random variable: Binomial Distribution 2 χ1 distribution Confidence Interval of Wald Test - ^ β 1 ±q 1− ∝ 2 ×s.e β 1) (^ -odds ratio corresponding to β is ( L U e ,e ¿ -95% CI for odds ratio corresponding to 1 unit increase in Xi is ( L 6.pth-quantile with sampling distribution U e ,e ¿ 1.Mean times -For every increase in Xi, the odds of success will increase by Confidence interval -Look at proportion of modal category, compare it via the difference 2.Bar Plot: -To display single categorical variable -Mention group of categories with high/low proportions -Mention any apparent trend in proportions if there is ordering 3.Histogram -use bars to portray the frequency or relative frequency of the possible outcomes for quantitative variable -Overall pattern: Gap, outlier? -Skewed or symmetric? -Unimodal? Bimodal? -Spread of data -Median and IQR -No. of outliers -Outlier: 1.5 ×IQR below and above upper/lower quartile 5.Contingency table -To display 2 categorical variables 6.Scatterplot -For association between quantitative vs quantitative variable -correlation: ( Y i − ´Y sy X i − X́ ( ¿) sx ) n 1 r= ∑¿ n−1 i=1 -r is between 1 and -1 -Sum of observations divided by number of observations n - X́ = 1 ∑X n i=1 i 2.Median -Middle value of the observations when observations ordered from smallest to largest - ( n+1 ) 2 average of n ( +1) 2 X 0.5 th largest if odd, n ( ) 2 th and th if even *If date is highly skewed, report median and if symmetric, report mean 3.Range -Difference between largest and smallest observations -Measure of spread but sensitive to extreme observations 4.Variance -Average of the squared deviation from the mean - s 2= n 1 ∑ ( X i− ´X )2 n−1 i=1 5.Standard deviation -Represents average distance from mean -68% observations fall between 1 sd of mean, between X́ −s and X́ +s -95% fall between 2 sd of mean -All fall between 3 sd of mean -value that p percent of the observations fall below or at that value ( q 0.5=X 0.5 ¿ ^ 7.IQR -Distance between upper and lower quartile (q^ q 0.25) 0.75− ^ -50% of sample points fall within IQR -how spread the ‘middle’ data is Lurking variable -Variable that is usually unobserved that influence the association between variables of primary interest Confounding -2 explanatory variables are associated with a response variable but also associated with each other Poor sources of bias -Sampling bias -Non-response bias -Response bias 1.Mutually exclusive (disjoint) -A and B has no common outcome A∩B=∅ -If P(A) ≥ 0 , P(S)=1, P ( A ∪ B )=P ( A) 2.Not mutually exclusive - P ( A ∪ B )=P ( A) - -Numerical measurement of outcome of experiment via random sampling −20 - pz = e 20 z! z -n trials have 2 possible outcomes, independent -same probability of success X Bin ( n,p ) n x P ( X=x ) =C x p (1− 1−p E ( X)=np,Var ( X)= for barplot 1.Mean: -sum of probabilities multiplied by possibilities μ=∑ xpx ) x 2.Expected value of X, E(X): The mean of the probability distribution of a random variable X - P ( A| B )= P ( A ) 4.Conditional probability -Probability of A given B - P ( A| B )= P ( A∩B P(B) 5.Bayes Theorem 6.Sensitivity -Probability that the test is positive given that the person has disease P(A|B) 7.Specificity -Probability that the test is negative given no disease P(AC|BC) 8.Prevalance -Having Disease P(B) =mean of each random variable 3.Standard deviation -measures variability of a random variable from the mean Probability distribution P(X=1)=p, P(X=0)=1-p Sampling distribution: 2 2 distribution that σ = ( X−μ) p -Probability specifies the probabilities for x possible values that statistic 4.Variance can take -large standard deviation has larger variability Sampling proportion ^ p n for random sample size n: 1 σ 2 Mean=p, sd= Var X́ = 2 ∑ ( ) n ∑ i =1 Probability density function pdf -curve that determines probabilities of intervals Mean of continuous variable X μ=∫ xf (x ) dx Variance of continuous variable σ =∫ (x −μ ) f ( x ) 2 2 Pth quantile of cont variable P ( x≤q p )= p Combination: n! = C= k! ( n− k )! n k √ p (1− p ) , n 2 p ( 1− p ) p,( ) n p N¿ ^ √ if n is sufficiently large that p (1− ^ p ) ≥5 n^ p (1− ^ p )≥5 n^ When , ^ p ±q -independent observations -n ≥ 30 -X is approximately normal 1− √ ^ p (1− ^ p) n Width: higher the CI, smaller α and wider the width 2×q n= 1− ( × α 2 2×q √ 1− ^ p (1− ^ p) n α 2 D ) p(1− p *use p=0.5 (highest variability to obtain highest n) t-distribution: -probability on the degree of freedom(df) -thicker tail, more variability than N(0,1) -larger the df, closer to N(0,1) X́±t 2×t 2 σ X́ N( μ, ) n × α 2 99%=2.58,80%=1.28,95%=1.96 , the sampling distribution is approximately normal CLT: ^p Sample proportion: for cat Normal distribution -Symmetric, bell-shaped X N( μ, σ 2 ) X +..X n 3.Independent event n N (μ, X́ = 1 1 -2 events do not influence n ( ) X =μ = E X́ ∑ one another n i=1 i X−μ N ( 0,1) Z= *mean of variables X́ σ P ( A∩B ) =P ( A) P P ( A )= P (A∩B ) + *a long run interpretation Margin of error: -measures how accurate the point estimate is likely to be estimating a parameter sd Standard error: -an estimated deviation of a sampling distribution Population proportion: p α n−1,1− 2 α n−1,1 − 2 t n≥( n−1,1− × × s √n ×s α D s √n 2 ≤D 2 )