Linear Regression

Linear Regression 1 Correlation computations. Suppose we have the following 5 pairs of points: (10,15), (20,25), (32,28), (40,50), and (55,47) with descriptive statistics and scatterplot as follows: mean sd n x 30.4 15.83667 5 y 33.0 14.98332 5 To compute correlation we need to standardize each x and each y, then multiply the pairs, and then add the results up for all pairs, and finally, divide that sum by (n-1). The table is filled in accurate to 2 decimals (you can see actual values below). X Z_X Y Z_Y (Z_X)(Z_Y) 10 -1.29 15 -1.20 1.55 20 -.66 25 -.53 .35 32 .10 28 -.33 -.03 40 .61 50 1.13 .69 55 1.24 47 .93 1.16 Note the column sum of Z_X and Z_Y should each be 0 (ignoring rounding errors). Finishing the computation, we sum the last column and get 3.72. Then since n=5, we divide by 4. And the correlation is then r=3.72/4=.93 Rcmdr confirms and is a bit more accurate: cor(Dataset[,c("x","y")], use="complete.obs") x y x 1.0000000 0.9271516 y 0.9271516 1.0000000 Linear Regression 2 Example Scatterplots / Correlation Discussion For each plot, determine if linear regression is appropriate and estimate the correlation. Note: Correlation can be computed no matter what the shape in the scatterplot. It’s up to you to decide if the pattern is linear so that correlation is meaningful. Linear Regression 3 Illustrating Least Squares We return to our example with 5 pairs of points to examine least squares. The first scatterplot already has the least squares regression line on it, while the second is blank for illustration. Draw in the residuals (vertical distance from points to the line) for the least squares regression line. Which points have the largest residuals? What is the sum of the residuals? Draw in any other line (not the least squares line), and the corresponding residuals. Do these residuals appear larger or smaller in magnitude (on average) compared to the ones above? How would the sum of the residuals squared for this line compare to the line above? The least squares line is the “best” line in linear regression because it has the smallest sum of the residuals squared. Linear Regression 4 Example Scatterplot with Regression Line + Rcmdr Output for Regression (General Context) Does a linear regression model appear appropriate? RegModel.1 <- lm(y~x, data=Regexample) summary(RegModel.1) Call: lm(formula = y ~ x, data = Regexample) Residuals: Min 1Q -43.972 -19.455 Median -2.205 3Q 21.132 Max 59.001 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 27.1934 19.2592 1.412 0.169 x 5.4182 0.5331 10.163 4.57e-11 *** --Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 26.55 on 29 degrees of freedom Multiple R-squared: 0.7808, Adjusted R-squared: 0.7732 F-statistic: 103.3 on 1 and 29 DF, p-value: 4.567e-11 Provide the correlation and interpret this value. What is the estimated slope? Interpret this value. Give the equation of the least squares line. What is the value of R-squared? Interpret this value. Predict the value of the mean response when the explanatory variable takes on the value of 25. Should you use this linear regression model to predict the value of the mean response when the predictor takes on the value of 100? Explain. Linear Regression 5 Example in Context A chemical analysis of diesel engine exhaust from Environmental Science Technology in 1984 recorded the mass and elemental carbon (both in milligrams per cm squared) as part of a study about carbon aerosols. Assume we are interested in whether or not mass can be used to predict elemental carbon. Response: Explanatory: Does a linear regression model appear appropriate? Partial Output: Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 30.98933 5.04626 6.141 2.90e-06 Mass 0.73660 0.03441 21.404 < 2e-16 --Residual standard error: 11.83 on 23 degrees of freedom Multiple R-squared: 0.9522, Adjusted R-squared: 0.9501 F-statistic: 458.2 on 1 and 23 DF, p-value: < 2.2e-16 What is the estimated slope? Interpret this value. Give the equation of the least squares line. Predict the value of the mean elemental carbon in the exhaust when the mass is 200. Are you able to compute a residual for when mass is 200 using the predicted value you just computed? In the context of this regression, is it correct to say that 95.22% of the observed variation in mass is explained by the linear relationship between mass and elemental carbon? Explain. Linear Regression 6 Regression Example in Context An article in Environmental and Experimental Botany collected information on pea plants on the effects of UV-B radiation (among other things). Two variables collected were sunburn index and distance from an ultraviolet light source (cm). Response: Explanatory: Does a linear regression model appear appropriate? Partial Output: Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.500010 0.191313 23.52 4.86e-12 Distance -0.056486 0.004425 -12.77 9.92e-09 --Residual standard error: 0.2547 on 13 degrees of freedom Multiple R-squared: 0.9261, Adjusted R-squared: 0.9204 F-statistic: 162.9 on 1 and 13 DF, p-value: 9.925e-09 Provide the correlation and interpret this value. What is the estimated slope? Interpret this value. Give the equation of the least squares line. What is the value of R-squared? Interpret this value. Predict the value of the mean sunburn index when the distance is 40 cm. Estimate the residual from one of the observations when the distance is 40 cm. Note: For this particular example, the S shape in the scatterplot means a quadratic regression would be a better choice than lin. reg., especially when we consider the assumptions for linear regression. Linear Regression 7 Inference For inference in linear regression to be valid, four assumptions must be met. For each assumption, describe how it should be checked (assuming it can). Assumption How to Check Linear Relationship between x and y The residuals must be a random sample. The error terms must have constant variance (that does not depend on x). The error terms must have a normal distribution. Carbon Example continued. Recall that we are trying to predict elemental carbon using mass. Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 30.98933 5.04626 6.141 2.90e-06 Mass 0.73660 0.03441 21.404 < 2e-16 --Residual standard error: 11.83 on 23 degrees of freedom Multiple R-squared: 0.9522, Adjusted R-squared: 0.9501 F-statistic: 458.2 on 1 and 23 DF, p-value: < 2.2e-16 What is the value of the residual standard deviation? What does this value measure? Set up appropriate hypotheses and test whether or not mass is a significant predictor for elemental carbon. Null hypothesis: Alternative Hypothesis: (Assumptions checked below) Test Statistic: Dist. of Test Stat: Conclusion: p-value: Linear Regression 8 Remember that some of the regression assumptions can only be checked after the regression has been fit. Identify the plots below and discuss what they tell you about if the assumptions are met for the hypothesis test you just performed. Finally, create a confidence interval for the population slope for predicting elemental carbon using mass in diesel engine exhaust. Interpret your interval. Would you be able to conclude that it was possible the slope was exactly 1? Linear Regression 9 Pea Plants and Sunburn continued. Recall the two variables collected were sunburn index and distance from an ultraviolet light source (cm). Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.500010 0.191313 23.52 4.86e-12 Distance -0.056486 0.004425 -12.77 9.92e-09 --Residual standard error: 0.2547 on 13 degrees of freedom Multiple R-squared: 0.9261, Adjusted R-squared: 0.9204 F-statistic: 162.9 on 1 and 13 DF, p-value: 9.925e-09 Perform a hypothesis test to determine if distance has a significant negative linear relationship with sunburn index. Null hypothesis: Alternative Hypothesis: (Assumptions checked below) Test Statistic: p-value: Dist. of Test Stat.: Conclusion: Identify the plots below and discuss what they tell you about if the assumptions are met for the hypothesis test you just performed. Linear Regression 10 Time permitting: Using the carbon data (output below), compute a 95% confidence interval for the mean carbon when mass is 150. Then, compute a 95% prediction interval for an individual response (carbon value) when the mass is 150. Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 30.98933 5.04626 6.141 2.90e-06 Mass 0.73660 0.03441 21.404 < 2e-16 --Residual standard error: 11.83 on 23 degrees of freedom Multiple R-squared: 0.9522, Adjusted R-squared: 0.9501 F-statistic: 458.2 on 1 and 23 DF, p-value: < 2.2e-16 CI for mean response: Prediction interval for individual response: If you computed similar confidence intervals at a mass value of 300, would the prediction interval still be wider than the confidence interval for the mean response? Would the intervals associated with mass of 150 be narrower or wider than those associated with mass of 300? Why?