Chapter 8: Multiple Linear Regression

MULTIPLE LINEAR REGRESSION CHAPTER8 1 Multiple Linear Regression Model As investment analysts, we often hypothesize that more than one variable explains the behavior of a variable in which we are interested. Multiple linear regression allows us to determine the effect of more than one independent variable on a particular dependent variable. 𝑌𝑖 = 𝑏0 + 𝑏1𝑋1𝑖 + 𝑏 2 𝑋 2𝑖 + ⋯ + 𝑏 𝑘 𝑋 𝑘𝑖 + 𝜀 𝑖 for i=1,2…,n • A slope coefficient, 𝑏𝑗 , measures how much the dependent variable, Y,changes when the independent variable, 𝑋𝑗, changes by one unit, holding all other independent variables constant. • In practice, software programs are used to estimate the multiple regression model. 2 Assumptions of the Multiple Linear Regression Model • The relationship between the dependent variable, Y, and the independent variables, X1, X2, . . . , Xk, is linear. • The independent variables (X1, X2, . . . , Xk) are not random. Also, no exact linear relation exists between two or more of the independent variables. • The expected value of the error term, conditioned on the independent variables, is 0: E(| X1, X2, . . . , Xk) = 0. • The variance of the error term is the same for all observations • The error term is uncorrelated across observations • The error term is normally distributed. 3 Testing Whether All Population Regression Coefficients Equal 0 • We illustrated how to conduct hypothesis tests on regression coefficients individually using a t-test. • But what about the significance of the regression as a whole? • –We test the null hypothesis that all the slope coefficients in a regression are simultaneously equal to 0 by using an F-test. 4 Testing Whether All Population Regression Coefficients Equal 0: ANOVA • Analysis of variance (ANOVA) is a statistical procedure for dividing the total variability of a variable into components that can be attributed to different sources. • In regression analysis, we use ANOVA to determine the usefulness of the independent variable or variables in explaining variation in the dependent variable. The F-statistic tests whether all the slope coefficients in a linear regression are equal to 0 𝑅𝑆𝑆/1 𝑀𝑒𝑎𝑛 𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒𝑠 𝐹= = 𝑆𝑆𝐸/(𝑛-(k+1) 𝑀𝑒𝑎𝑛 sum of 𝑠𝑞𝑢𝑎𝑟𝑒𝑑 𝑒𝑟𝑟𝑜𝑟s The F-statistic measures how well the regression equation explains the variation in the dependent variable. 5 F-Test example 6 Testing Whether All Population Regression 2 Coefficients Equal 0: 𝑅 In our coverage of simple regression, we presented the coefficient of determination, 𝑅 2 , as a measure of the goodness of fit of an estimated regression to the data. In a multiple linear regression, however, 𝑅 2 is less appropriate as a measure of whether a regression model fits the data well (goodness of fit). Adjusted 𝑅 2 is a measure of goodness of fit that accounts for additional explanatory variables. 𝑅2 = 1 − 𝑛−1 𝑛−𝑘−1 1 − 𝑅2 where n is the number of observations and k is the number of independent variables (the number of slope coefficients). The greater the 𝑅 2 , the more the variability in Yis explained by the regressors (independent variables). 7