Population Regression vs. Sample Regression, Ordinary Least (Solutions)

1 Population Regression vs. Sample Regression, Ordinary Least Squares (OLS) Estimation Exercise 1.1. (Stock & Watson, Review the Concepts 4.1) Explain the difference between βb1 and β1 ; between the residual u bi and the regression error ui ; and between the OLS predicted value Ybi and E(Yi |Xi )? • β1 is the slope of the population regression line. It is a constant and is unknown. βb1 is the slope of the sample regression line. It is our estimate of β1 using our sample. Note that βb1 is a random variable because the value of βb1 depends on the sample we get. • ui is the population regression error. Note that ui is unobserved and unknown, since ui = Yi −β0 −β1 Xi and β0 , β1 are unknown. u bi is the residual. In contrast to ui , u bi is observed. u bi is our estimator of ui . • E(Yi |Xi ) = β0 + β1 Xi is the conditional expectation function, it is unknown. In contrast, Ybi is the predicted value based on the sample regression line, and it is known. Exercise 1.2 (Final Exam, Fall 2014) Consider the population regression model Yi = β0 + β1 Xi + ui . (a) What minimization problem does OLS solve in order to estimate β0 and β1 ? Express the problem mathematically (there is no need to solve the minimization problem problem). PN minb0 ,b1 i=1 (Yi − b0 − b1 Xi )2 (b) The figure below shows data from a sample of 250 observations of X and Y . One of the lines is the sample regression line, βb0 + βb1 Xi ; the other is the population regression line β0 + β1 Xi . Is the sample regression line solid or dashed? Explain. The sample regression line is the solid line. This is because the sample regression line minimizes the sum of squared residuals and we can see that the solid line fits the scatter plot better than dashed line. 1 2 Interpreting OLS Regression Coefficients Exercise 2.1 A regression of wage (hourly wage, measured in 1979 dollars per hour) and educ (years of schooling) using data from a random sample of 526 American workers yields the following: wage [ = −0.90 + 0.54 · educ (a) Interpret the intercept of this regression. The intercept of -0.90 literally means that a person with 0 years of education has a predicted hourly wage of -$0.90 dollars per hour. This is non-sensical, so it would be difficult to attach any real-world meaning to it. (b) It turns out that all workers in the data have at least 8 years of education. Does this help reconcile your answer in part (a)? Yes. Since there are no workers in the sample who have very low levels of education, it is not surprising that the regression line does poorly at predicting wages when an individual has 0 years of schooling. In this case, the intercept lies far from the data (a case of extrapolation) so our estimate of wages for those with little schooling is not well determined. (c) Interpret the slope of this regression. The slope estimate of 0.54 implies that one more year of education is associated with an increase in the hourly wage of 0.54 cents per hour. (d) A worker has 16 years of education. What is the regression’s prediction for the worker’s hourly wage? wage [ = −0.90 + 0.54 · 16 = 7.74 (e) Suppose a worker obtains 4 more years of schooling. What is the regression’s prediction for the change in the worker’s hourly wage? The predicted wage would increase by 4 · 0.54 = 2.16 3 Measures of Fit Exercise 3.1 (Stock & Watson, Review the Concepts 4.3) SER and R2 are “measures of fit” for a regression. Explain how the SER measures the fit of a regression. What are the units of SER? Explain how R2 measures the fit of a regression. What are the units of R2 ? • The SER is the standard error of q the regression, which is an estimator of the SD of the population Pn 1 b2i . Since u bi is the difference between our prediction error ui . The formula is SER = i=1 u n−2 b Yi and the true data point Yi , looking at the spread of the u bi ’s (as measured by the SER) gives us a measure of the fit of the regression. • The units of u bi are the same as that of Yi , so the units of SER as the same as Yi . • R2 is the fraction of the variation in Y that is explained by X. The formula for R2 is R2 = ESS/T SS, Pn where ESS sum of squares, and TSS is the total sum of squares, ESS = i=1 (Ybi −Y )2 , Pn is the explained T SS = i=1 (Yi − Y )2 . To see how the R2 measures the fit of the regression, it is useful to look at two extreme cases. ◦ Case 1: Suppose that X explains all of the variation in Y , so Ybi = Yi for all i. Then, ESS = T SS, so R2 = 1. ◦ Case 2: Suppose that X explains none of the variation in Y , so that βb1 = 0. This means Yb = βb0 = Y , so ESS = 0 =⇒ R2 = 0. • R2 is scale-free (has no units). 2 Exercise 3.2 (Stock & Watson, Review the Concepts 4.4) Sketch a hypothetical scatterplot of data for an estimated regression with R2 = 0.9. Sketch a hypothetical scatterplot of data for a regression with R2 = 0.5. Answers will vary depending on how you sketched your scatterplot, but here are two plots with R2 = 0.9 (left graph) and R2 = 0.5 (right graph). As you can see, the plot with R2 = 0.9 has data points that are closer to the regression line than the plot with R2 = 0.5. Exercise 3.3 Suppose that the R2 from the regression in Exercise 2.1 is equal to 0.242. How would you interpret this R2 ? 24.2% of the variation in wages is explained by education. 4 Least Squares Assumptions Exercise 4.1. (Stock & Watson, Chapter 4, Review the Concepts, Exercise 2) For each least squares assumption, provide an example in which the assumption is valid and then provide an example in which the assumption fails. • Assumption 1: E(ui |Xi ) = 0. Valid when Xi is randomly assigned, because random assignment implies Xi and ui are independent, hence E(ui |Xi ) = 0. Fails when ui and Xi are correlated. • Assumption 2: (Xi , Yi ) are i.i.d. Valid when observations are drawn using random sampling. Fails when data is not from random sampling (e.g., we collect data from the first 10 people that choose to respond to our survey). Also fails when we have time series data: e.g., if X is US dollar exchange rate, and Y is US exports over time, we might think that the exchange last month will affect exports this month, in which case cov(X1 , Y2 ) 6= 0 violating independence between any pair of Xi and Yj . • Assumption 3: Outliers are unlikely, i.e., finite fourth moment 0 < E(X 4 ) < ∞, 0 < E(Y 4 ) < ∞. Valid when X and Y have a finite range since if the range is finite, the kurtosis (fatness of the tails) will also be finite. Fails when there are data entry errors that cause outliers. Exercise 4.2. (From Section Notes 4 in bCourses (All Students) > Files > Discussion Section) A professor decides to run an experiment to measure the effect of time pressure on final exam scores. He gives each of the 400 students in his course the same final exam, but some students have 90 minutes to complete the exam while others have 120 minutes. Each student is randomly assigned one of the exam times based on the flip of a coin. Let Yi denote the number of points scored on the exam by the ith student (0 ≤ Yi ≤ 100). Let Xi denote the amount of time the student has to complete the exam (Xi = {90, 120}), and consider the regression model Yi = β0 + β1 Xi + ui . 1. Explain what the term ui represents. Why will different students have different values of ui ? ui represents factors other than time that influence the student’s performance on the exam other than time allowed during the exam, including amount of time studying, aptitude for the material, and etc. Some students will have studied more than average, other less; some students will have higher than average aptitude for the subject, others lower, and so forth. 3 2. Explain why E[ui |Xi ] = 0 for this regression model. Because Xi is randomly assigned, ui is independent of Xi . There is no unobserved factor that is correlated with the amount of time a student receives to take the exam. Because u and X are independent E(ui |Xi ) = E(ui ) = 0. 3. Are the other least squares assumptions satisfied? Assumption #2 is satisfied if this year’s class is typical of other classes, that is, students in this year’s class can be viewed as random draws from the population of students that enroll in the class. Assumption #3 is satisfied because 0 ≤ Yi ≤ 100 and Xi can take on only two values (90 and 120). 5 Regressions in Stata Exercise 6.1 The table below shows regression output from Stata. Linear regression Number of obs = F( 1, 72) = Prob > F = R-squared = Root MSE = 74 17.28 0.0001 0.2196 2623.7 -----------------------------------------------------------------------------| Robust price | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------mpg | -238.8943 57.47701 -4.16 0.000 -353.4727 -124.316 _cons | 11253.06 1376.393 8.18 0.000 8509.272 13996.85 -----------------------------------------------------------------------------Identify the following from the above regression output. (a) Dependent and independent variables Dependent variable: price, independent variable: mpg (b) Sample size n = 74 (c) R2 R2 = 0.2196 (d) SER SER = 2623.7 (Root MSE in the above table) (e) βb0 and βb1 βb0 = 11253.06, βb1 = −238.8943, so the sample regression line is pbrice = 11253.06 − 238.8943 · mpg. 4