Regression Notes

Lecture 10: Regression 3 Different Types of Regression 1.​ Linear Regression: relationship between a single independent variable and single (interval/ratio) dependent variable, allowing us to make a prediction about the future outcomes 2.​ Logistic Regression: relationship between multiple independent variables and a single dependent variable when the outcome is binary (only 2 categories, eg. yes or no) 3.​ Multiple Regression: method used to look at the relationship between a dependent variable and multiple independent variables to develop a prediction equation based on the data Linear Regression -​ -​ Single independent variable Single interval/ratio dependent variable Logical Regression -​ -​ -​ Assumptions: -​ Variables are interval/ratio -​ Normally distributed -​ Linear relationship -​ Variation is equal among groups (homoscedasticity) Multiple independent variables Single dependent variable that is categorical Outcome is binary Assumptions: -​ Independent variable can be any level (NOIR) -​ Dependent variable MUST be dichotomous (2 choices) -​ Relationship does not have to be linear Multiple Regression -​ -​ Multiple independent variables Single dependent variable Assumptions: -​ Same as linear regression, but just has more independent variables Formula: -​ The formula changes Two Types of Logical Regression: based on the amount of 1.​ Binary: used when dependent independent variables, variable is dichotomous but is similar to linear (We focus on this one^) regression 2.​ Multinomial: used when -​ ex) this is the formula if dependent variable has more than you have 2 independent 2 categories variables: Y = B0 + B1(X1) + B2(X2) + E Formula: -​ B1 and X1 is for the first -​ It uses logarithms (out of our scope) independent variable, and but it does make the regression line B2 and X2 is for the Additional Notes: curvy second -​ Residual is the deviation of a particular point from the Odds Ratios: regression line -​ The result of a logistic regression -​ The regression line is the creates an odds ratio instead of line of best fit for the plotted B0, and this tells us how likely an event is going to happen points. In order to have the -​ The dependent variables are coded best fit you want low SSE at 1 (event happened) and 0 (event Formula: -​ Y = B0 + B1(X1) + E -​ Y = dependent variable -​ X = independent variable -​ B1 = regression coefficient, (gives us the slope and direction of relationship) -​ B0 = Y-intercept -​ E = random error or residual -​ -​ (sum of squared error) and a high r-squared value which will give you the best fit and least amount of variance If you square the correlation coefficient you get r-squared which tells you the % of variance in the outcome that can be explained (ex. 52% of the variance in postpartum depression was explained by maternal age) Beta value: estimate of change in dependent variable for every 1 point or category change (slope) -​ -​ -​ -​ didn’t happen) Confidence intervals are used with odds ratios, and this tells us the range of values likely to contain the true odds ratio. We want CI to be narrow ex) OR 2.3 (95% 2.0-2.6) → we interpret this as 2.3x more likely to happen If the CI contains 1, the results are not significant The line is rounded in a logistic regression -​ Analyzing Linear Regression There are 4 things you should look at when interpreting linear regressions: sum of squared errors (SSE), the F-value, the R-squared value, and the P-value. 1.​ Is it Significant? You look at the P-value in green and you can see that the P < 0.05 so therefore the study is significant. This should always be the first step, because if the p-value is not significant, neither is anything else. 2.​ Does the Model Fit? You look at the SSE value circled in blue, and the F-value in red. If the F-value is significant, that means that the regression line fits the model 3.​ How much Variance is Explained? You look at the R-squared value in purple, and you can see that 47.3% of the variance is explained by this model Analyzing Multiple Linear Regression This example compares 4 independent variables on the effects of VO2 max. For this regression we look at the SSE, F-value, R-squared value, p-value, and coefficient values 1.​ Is it Significant? You look at the P-value in green and you can see that the P < 0.05 so therefore the study is significant, and all the variables are significant 2.​ Does the Model Fit? You look at the SSE value circled in blue, and the F-value in red. If the F-value is significant, that means that the regression line fits the model 3.​ How much Variance is Explained? You look at the R-squared value in purple, and you can see that 57% of the variance is explained by this model 4.​ How do the Variables affect the Data? You can see in yellow all the variables have numbers beside them. These numbers indicate how they affect the dependent variable (VO2 max) → such as for every increase in age, VO2 max decreases by 0.165 Analyzing Logistic Regression This example looks at how different ethnicities, sec, and gender and its result on the dependent variable. For logistic regression we look at P-values, odds ratios, coefficient values, and CI 1.​ Is it Significant? You look at the P-values for all the sub-variables in the significance column. In this example, not all of the variables are significant. You may also notice that the CI for the insignificant variables are also insignificant (they contain 1) 2.​ Are the Variables Chosen Significant? In red, you can see the constant values for the entire variable section. These indicate whether the variable itself is significant. In this case, both ethnic and sec are significant to consider. (P<0.05) 3.​ What are the Odds? You look at the odds ratios in the purple Exp(B) column, this explains the odds of one of the variables being more likely to cause (whatever the dependent variable is). For example sec(1) is 11x more likely. 4.​ How do the Variables affect the Data? You can see in the yellow B column all the variables have numbers beside them. These numbers indicate how they affect the dependent variable. (We don’t know what the dependent variable is in this case)