Quantative Methods - Multiple Regression, CFA Level II

Level II - Quantitative Methods Multiple Regression 2022 Exam Contents and Introduction • Multiple Linear Regression • F-stat and Adjusted R-square • Using Dummy Variables in Regressions • Violations of Regression Assumptions • Model Specifications Errors • Models with Qualitative Dependent Variables 1.1 Multiple Linear Regression Multiple linear regression allows us to determine the effect of more than one independent variable on a particular dependent variable Yi = + blX1i + b2X2i+ Ei A regression equation has • 1 intercept coefficient • k slope coefficients A slope coefficient measures how much the dependent variable changes when the independent variable changes by one unit, holding all other independent variables constant. Evaluating Regression Output The multiple R2 (or simply R2 ) quantifies the degree of linear association between the dependent variable and all of the independent variables jointly. It is the square of the correlation between predicted and actual values of the dependent variable). A value of Ofor R2 indicates no linear association; a value of I indicates perfect linear association. Key Points from Example 1 (1/2) Regress In(Bid-Ask Spread/Price) on In(Number of Market Makers) and In(Market Capitalization) Assumptions of the Multiple Linear Regression Model In order to make a valid inference from a multiple linear regression model, we need to make the following six assumptions: 1. The relationship between the dependent variable, Y and the independent variables (X_1 X_2...X_k) is linear. 2. The independent variables (X_1 X_2...X_k) are not random. Also no exact linear relation exists between two or more of the independent variables. 3. The expected value of error term, conditioned on the independent variables, is 0. 4. The variance of the error term is the same for all observations. 5. The error term is uncorrelated across observations. 6. The error term is normally distributed. Do Examples 2 and 3 Predicting the Dependent Variable in a Multiple Regression Model Obtain estimates b_0, b_1, b_2,..., b_k of the regression parameters b_0, b_1, b_2,..., b_k Determine the assumed values of the independent variables X_1i, X_2i,..., X_ki Compute the predicted value of the dependent variable using the equation: Y_i=b_0+b_1X_11+b_2X_21+...+b_kX_ki Do Example 4 2. Testing whether All Population Regression Coefficients Equal Zero We run a regression with 124 observations and 3 independent variables. The regression output indicates that MSR = 3000 and MSE = 890. Test the null hypothesis that all of the slope coefficients in the multiple regression model are jointly equal to O (H_0 : b_1= b_2 = b_3 = 0) against the alternative hypothesis that at least one slope coefficient is not equal to 0. 2.1 Adjusted R^2 With one independent variable, the coefficient of determination measures the goodness of fit of an estimated regression to the data. As we add independent variables, R2 increases even if the amount they explain is not statistically significant. With multiple linear regression, it is better to use adjusted R2. If adjusted R2 is used to compare models, the dependent variable should be defined the same way. Adjusted R2 can be negative; a high adjusted R2 does not necessary mean that the model is well specified. 3. Using Dummy Variables In Regressions A dummy variable is a type of a qualitative variable Value = 1 if condition is true; value = 0 otherwise If n states of the world then use n - 1 dummy variables Example 5: Do stock returns differ based on month of year? Are stock returns in Jan different from other months? 4. Violations of Regression Assumptions • Heteroskedasticity • Serial Correlation • Multicollinearity • Summarizing the Issues • For each violation we should understand: • What is it • Impact on statistical inference • How to detect • How to correct 4.1 Heteroskedasticity (1/2) Linear regression assumptions: 1. The relationship between Y and the X_1 , X2,..., X_k is linear. 2. The independent variables are not random... 3. The expected value of error term is 0. 4. The variance of the error term is the same for all observations. 5. The error term is uncorrelated across observations. 6. The error term is normally distributed. Two types of heteroskedasticity • Unconditional -> not a problem • Conditional -> problem Consequences of heteroskedasticity • F-test for the overall significance of the regression is unreliable • Coefficient estimates fine but standard error understated • t-stat inflated • Might incorrectly reject the null hypothesis (Type I error) "Find significant relationships where none actually exist" 4.1 Heteroskedasticity (2/2) Testing for heteroskedasticity Breuch-Pagan test H_0 : No heteroskedasticity Regress residuals with X BP test stat = nR^2 Compare with critical value from chi-squared table Example 8 Regression:t-stat for b_1 = 2.29 BP test: BP test stat- 12 Critical value = 6.6 Conclusion: we have a conditional heteroskedasticity problem which needs to be corrected. Correcting for heteroskedasticity Robust standard errors: correct the standard errors of the linear regression model's estimated coefficients to account for conditional heteroskedasticity Generalized least squares: modify the original equation in an attempt to eliminate heteroskedasticity 5. Serial Correlation (1/2) Linear regression assumptions: 1. The relationship between Y and the Y and the X_1 , X2,..., X_k is linear. 2. The independent variables are not random... 3. The expected value of error term is O. 4. The variance of the error term is the same for all observations. 5. The error term is uncorrelated across observations. 6. The error term is normally distributed. Serial correlation (autocorrelation): errors correlated across observation Consequences of positive serial correlation F-statistic may be inflated Coefficient estimates fine but standard error understated t-stat inflated Might incorrectly reject the null hypothesis (Type I error) 6. Multicollinearity Linear regression assumptions 1. The relationship between the dependent variable, Y and the independent is linear. 2. The independent variables are not random. No exact linear relation exists between two or more of the independent variables. 3. ... Multicollinearity: two or more independent variables (or combinations of independentvariables) are highly (but not perfectly) correlated with each other Consequences of multicollinearity Estimates of regression coefficients are imprecise and unreliable • Inflated standard error t-stats understated Detecting multicollinearity • A matter of degree rather than absence or presence • Symptoms: high RZ and significant F-stat but low t-statfor coefficients • With exactly two independent variables, pairwise correlation can be considered Correcting for multicollinearity • Exclude one or more of the regression variables • Experiment with including or excluding different independent variables 7. Model Specification Errors 1. Principles of Model Specification 2. Misspecified Functional Form 3. Time-Series Misspecification (Independent Variables Correlated with Errors) 4. Other Types of Time-Series Misspecification 7.1 Principles of Model Specification • The model should be grounded in cogent economic reasoning • The functional form chosen for the variables in the regression should be appropriate given the nature of the variables • The model should be parsimonious • The model should be examined for violations of regression assumptions • The model should be tested on out of sample data 7.2 Misspecified Functional Form Whenever we estimate a regression, we must assume that the regression has the correct functional form. This assumption can fail in several ways: • One or more important variables could be omitted from regression (Example 10). One or more of the regression variables may need to be transformed (for example, by taking the natural logarithm of the variable) before estimating the regression (Examples 11 and 12). • The regression model pools data from different samples that should not be pooled. 7.3 Time Series Misspecification Regression assumption 3: error term has an expected value of O. With time series data, this assumption is frequently violated -5 estimated regression coefficients become biased and inconsistent. Three common problems that create this type of time-series misspecification are: 1. including lagged dependent variables as independent variables in regressions with serially correlated errors (Example 13) 2. including a function of a dependent variable as an independent variable, sometimes as a result of the incorrect dating of variables 3. independent variables that are measured with error (Example 14) 7.4 Other Types of Time-Series Misspecification Nonstationarity: mean and variance are not constant through time. Nonstationarity is the most frequent source of misspecification in linear regressions that use time series from two or more different variables. Situations where we need to use stationarity tests before we use regression statistical inference • Relations among time series with trends (for example, the relation between consumption and GDP) • Relations among time series that may be random walks 8. Models with Qualitative Dependent Variables Qualitative dependent variables are dummy variables used as dependent variables. • Example 15: Does at least one analyst cover a company? • Linear regression is not the best option. Probit and logit models estimate the probability of a discrete outcome given the values of the independent variables used to explain that outcome; models are estimated using maximum likelihood methods. • The probit model is based on the normal distribution; it estimates the probability that Y = 1 given the value of the independent variable X. • The logit model is based on the logistic distribution rather than the normal distribution. Discriminant analysis gives a linear function which can be used to create an overall score; categorize based on the score. Summary • ANOVA • F-stat, R2 and adjusted R2 • Dummy variables • Assumptions • Heteroskedasticity • Serial correlation • Multicollinearity • Model misspecifications • Qualitative dependent variables