Statistics Lecture 10: Regression

Lecture 10: Regression What is Regression -​ A statistical technique that allows the researcher to make a prediction about a future outcome -​ Each prediction (Y) is closer to its mean than the independent variable (X) was to its mean; it regresses towards the mean -​ It estimates the relationship between variables: -​ How variations in the independent variable affect the dependent variable -​ What form the relationship takes -​ What confounders influence the outcome 3 Different Types of Regression 1.​ Linear Regression: relationship between a single independent variable and single (interval/ratio) dependent variable, allowing us to make a prediction about the future outcomes 2.​ Logistic Regression: relationship between multiple independent variables and a single dependent variable when the outcome is binary (only 2 categories, eg. yes or no) 3.​ Multiple Regression: method used to look at the relationship between a dependent variable and multiple independent variables to develop a prediction equation based on the data Linear Regression -​ Assumptions: -​ Variables are interval or ratio -​ Normally distributed -​ Relationship between variables is linear -​ The variance for the dependent variable is uniform (homoscedastic) -​ Simple linear regression: -​ One continuous or categorical predictor (independent variable) -​ One continuous outcome (dependent variable) -​ Has 2 components: -​ X and Y (independent and dependent variable) -​ Slope of the line (line of regression) -​ Data points -​ The stronger the relationship (correlation) the more accurate the prediction will be to the actual score -​ Linear regression formula: -​ Y = B0 + B1(X1) + E -​ Y = dependent variable -​ X = independent variable -​ B1 = regression coefficient, gives us the slope and direction of relationship -​ -​ B0 = Y-intercept -​ E = random error or residual Residual value: the deviation of a particular point from the regression line Estimating Error -​ Looking for the line of best possible fit -​ This is the total sum of squares -​ Analyzing Relationships -​ Y is the variability of Y -​ X is the variability of X -​ The blue value is the predictable part of the variability of Y by knowing X = R^2 -​ R^2 is reported as a percentage and it represents the % of the variation in the outcome that can be explained by the model -​ R on its own is a simple correlation between Y and X and can range from 0-1 Analyzing Regression is like a Puzzle??? -​ Beta value: estimate of change in dependent variable for every 1 point or category change (slope) -​ P-value: which IV is a predictor of DV change (whether it is significant) -​ R^2: how much explained (the bigger the beta, the higher R^2) -​ 95% CI: range of beta between which 95% confident is a true value (where the true population might lie) Interpreting Linear Regression Coefficients -​ A positive coefficient (beta) means that as value of the independent variable (Y) increases so does the dependent variable (X) -​ A negative coefficient means that as the independent variable increases the dependent variable tends to decrease Output from a Linear Regression: Notes are here: -​ Example of Multiple Linear Regression: -​ Btw formula is too complicated for us to know -​ Output from a Multiple Linear Regression: -​ Regression Notes Logistic Regression vs. Linear Regression: Binary Outcomes -​ The line is rounded in a logistic regression -​ Logistic Regression -​ Multiple independent variables and a dependent variable that is categorical -​ independent variables can be continuous or categorical and used to predict dichotomous (2 category) dependent variable -​ Relationship does not have to be linear -​ Types of Logistic Regression: -​ Binary logistic regression: used when dependent variable is dichotomous -​ Multinomial logistic regression: used when dependent variable has more than 2 categories -​ Formula is complicated How Results are Reported -​ Code the dependent variable as either 1 or 0 -​ 1 = the event happened -​ 0 = the event didn’t happen -​ The results of a logistic regression are reported as an odds ratio (instead of B0) -​ Formula: Odds (A) = P(A) / 1-P(A) -​ Tells us how more or less likely an event is to happen -​ Confidence intervals of odds ratios: -​ We want CI to be narrow -​ Reported like: OR 2.3 (95% CI 2.0-2.6) → 2.3x more likely to happen, 95% chance that true odds ratio in the population falls within this range -​ If the CI contains 1 it is not significant Analyzing Logistic Regression is like a Puzzle??? -​ Odds ratio: estimate of change in odds dependent variable for every 1 point or category change -​ P-value: Which IV is a predictor of DV change -​ R^2: how much is explained. Also bigger the beta higher R^2 -​ 95% CI: range of ODDS between which 95% confident the true value lies Interpretation of Logistic Regression -​ identify the dependent variable (should be dichotomous) -​ check odds ratios for independent variables -​ -​ if more than 1, increased odd of developing DV given exposure -​ if less than 1, decreased odds of developing DV given exposure Check confidence interval -​ if it includes 1.0 it is not significant Output for Logistic Regression -​ Review of Logistic Regression: -​ independent variable can be measured at any level -​ dependent variable can be measured at any level (NOIR) -​ dependent variable MUST be dichotomous