Chapter 12: Regression Analysis

Chapter 12: Regression Analysis New Vocabulary: Regression: a stats test that allows you to make a prediction about a future outcome Linear Regression: technique for analyzing the relationship between a single independent variable and a single interval/ratio level dependent variable to make a prediction for a future outcome based on the data Logistic Regression: method for analyzing the relationship between multiple independent variables and a single dependent variable when the outcome is binary Multiple Regression: method used to look at the relationship between a dependent variable and multiple independent variables to develop a prediction equation based on the research data Odds Ratio: the odds or probability of the outcome occurring divided by the odds or outcome of the event not occuring Regression Coefficient: the b value, which tells you the rate of change in the outcome or dependent variable with a one-unit increase in the corresponding independent variable R-squared Value: the percentage of the variance in the dependent variable that is explained Residual: the amount of prediction error in a regression equation Quantifying an Association -​ Correlations measure the strength of associations, but regression will quantify that association -​ Linear regression looks for a relationship between a single independent variable and a single interval/ratio level dependent variable to make an inference or prediction about the future value of the dependent variable at any given level of the independent variable. -​ For example, you can look at how many daily cigarettes (independent variable) it takes to cause shifts in fetal weight (dependent variable) when pregnant → see image -​ Y = B0 + B1(X1) + E -​ Linear regression plots these values onto a graph, where the independent variable is the X-axis, and the dependent variable is in the Y-axis. -​ Residual error is the difference between where the data points fall and where the regression line predicts they will fall. The lower the amount of residual, the better the data fits the line -​ The slope of the line tells you how much the predicted value of the dependent variable changes when there is a one-unit change in the dependent variable (this is shown as B0 in our class slides) -​ You use multiple regression when you want to consider how more independent variables affect the dependent variable (such as the length of pregnancy in this example) -​ Y = B0 + B1(X1) + B2(X2) + E (for this example b/c 2 independent variables) -​ Y = dependent variable -​ -​ -​ -​ -​ B0 = the Y-intercept, or when X = 0 B1 = the value of the regression coefficient for the 1st independent variable X1 = the value of the 1st independent variable B2 = the value of the regression coefficient for the 2nd independent variable E = error Study Example 2 -​ Model 1 is when we only consider age, Model 2 is when we consider age and height -​ R is the multiple correlation coefficient that you can square to get the R-squared (R^2) value. It tells you the percentage of variance in the dependent variable that is explained by the model you have built -​ In this example, the R-squared value of 0.850 means that including both age and height explains 85% of the variance seen in weight -​ The adjusted R-squared (which is used to prevent overestimation of R-squared), is used when you have a large number of independent variables with a relatively small sample size -​ The standard error of the estimate tells you the average amount of error there will be in the predicted outcome. For your prediction to be as accurate as possible you would want this value as close to 0 as possible -​ So we now know the R-squared of 85%, but you still have to look at the p-value to see if it is significant, and if each of the independent variables are significant as well -​ -​ This shows that when adding the variable of age, the R-squared went from 0 to 0.430 with a significant p-value of 0.028 (assuming alpha is 0.05). When you add height the R-squared goes from 0.430 to 0.850 (from explaining 43% of the variance to explaining 85% of the variance) with a significant p-value of 0.001. Adding the second independent variable actually increased the accuracy of the predictions made by this model -​ -​ When we use regression to make predictions we should look at the column for unstandardized coefficients (B). You can see the constant values of B (93.552 and -584.801). Then you will see the beta coefficients for the independent variables that tells us what a one-unit change in the independent variable will do to the outcome or dependent variable when the other variables are held constant. -​ In this example we can see that as b1 = 1.712 and b2 = 10.372 which means that when we control for height, every additional year of age adds 1.71 pounds, and when we control for age, every additional inch adds 10.37 pounds. Multicollinearity → this is not mentioned in our notes Logistic Regression -​ Logistic regression is used when you have a categorical dependent variable with 2 categories (nominal or ordinal) -​ Logistic regression creates an odds ratio (OR), which is the odds or probability of the outcome occurring divided by the odds or probability of the outcome not occurring Summary -​ Regression analysis is a statistical procedure that allows us to develop a regression equation that we can use to infer or predict future events -​ Linear regression analyzes the relationship between a single independent variable and a single (interval/ratio) dependent variable -​ The slope (b) tells us how much the predicted dependent value changes when there is a change in the independent variable -​ The residual is the prediction error, or how far away the actual data falls from the prediction line -​ Multiple regression is used to predict how more independent variables affect a dependent variable. It uses regression coefficients (b) to show the change in the dependent variable per one-unit increase of the independent variable -​ Each regression model has a corresponding R-squared which tells you how much variance in the dependent variable is explained by the independent variable -​ When the sample size is small, researchers may report the adjusted R-squared to avoid overestimation on the amount of variance -​ Don’t forget to ensure your R-squared is significant by checking the p-value -​ Logistic regression is used when the outcome is binary (yes or no) and uses an odds ratio to tell them the odds/probability of the event occurring in one group vs another