Chapter 14. Correction and Simple Linear Regression

• Scatter Diagram: A two-dimensional graph on which each observation in the sample of bivariate data is represented by a point (dot). • Negative (inverse) Relationship: A relationship between two variables, X and Y, in which large values of X are associated with small values of Y. • Positive (direct) Relationship: A relationship betwee two variables, X and Y, in which large values of X are associated with large values of Y. • Coefficient of Correlation (r): A statistical measure of the strength of the linear relationship between two variables, calculated from a sample of bivariate data. It is an estimate of the population coefficient of correlation. P (x−x)(y−y) r = pP P 2 (x−x) SCP XY √ = √ . (y−y) SS X SS Y −1 ≤ r ≤ 1 2 The larger |r| is, the stronger is the linear relationship. r ≈ 0 indicates that there is no linear relationship between X and Y. r = 1 or 1 implies that a perfect linear pattern exists between two variables. r > 0 tells the positive relationship and r < 0 means the negative one. P P • Sum of Squares: SS X = (x − x)2 , SS Y = (y − y)2 , P • Sum of Cross Product: SCP XY = (x−x)(y−y) q X • Standard Deviation of X: SX = SS n−1 q Y • Standard Deviation of Y: SY = SS n−1 • Covariance, cov(X,Y): A measure of the joint variation of the two variables. 1 P 1 cov(X,Y) = n−1 SCP XY. (x−x)(y−y) = n−1 1 • Note: cov(X’,Y’) = correlation between X and Y = r where X’ = variables). • Sample Correlation, r: r= X−X SX and Y’ = Y −Y SY (standardized Cov(X,Y ) SX SY . • Least Squares Line: The best line through a sample of bivariate data; it minimizes the sum of the squares of the vertical distances from each point to the line. • Sum of Squares of Error: The residual sum of squares is denoted by P 2 P (SCP )2 SSE = d = (y−by)2 = SS Y − SSXY X where XY by = bo +b1 X, where b1 = SCP SS X , b0 =-y− b1 -x. P P y x ( N ote : -y= n , -x= n ) • Dependent Variable: In simple linear regression, the variable (Y) that is predicted or explained using a single independent variable (X). • Independent Variable: In simple linear regression, the variable (X) used to predict or explain values of the dependent variable (Y). • Intercept (bo ) : The distance from the origin to the point where the least squares line crosses the vertical axis. It is calculated from the sample of bivariate data and estimates the population intercept, βo . • Slope (b1 ) : A value specifying the steepness (slant) of the least squares line. It is calculated from the sample of bivariate data and estimates the population slope, β1 . • Residual: For each observation in the sample, the actual value of the dependent variable minus the estimated value. • Standardized Residual: A value that can be used to identify sample observations that have an unusually large or small value of the dependent variable, Y. • Regression Analysis: A method of studying the relationship between two or more variables. In “simple linear regression”, we use only one predictor variable, X, to describe the behavior of the dependent variable, Y. • Statistical Model when we select to use a straight-line predictor: Y = βo + β1 X + e. • Deterministic Portion of the Model: In simple linear regression, the assumed line, βo + β1 x,about which all points will fall. 2 • Random Error Component (e): In simple linear regression, the error portion of the model, assumed to be independent, normally distributed random variables with zero mean and common variance. • Assumptions for the Simple Linear Regression Model: 1. The mean of each error component is zero. 2. Each error component (random variable) follows an approximate normal distribution. 3. The variance of the error component, σe2 , is the same for each value of X. (Fig. 14.10) The estimation of σe2 = s2 =bσ 2e = SSE n−2 . • Homoscedasticity: An assumption behind the sample linear regression model; it states that the variance of the error component remains constant for all values of the independent variable. • Heteroscedasticity: A violation of the assumptions behind the simple linear regression model; this violation occurs when the variance of the error component does not remain constant for all values of the independent variable. 4. The errors are independent of each other. • Inference on slope β1 : b1 −β t= √ 1 s/ SS X Test of Hypothesis on the Slope of the Regression Line: One Tailed Test: Ho : β1 ≤ 0, Reject Ho if t > tα,df Ha : β1 > 0 b1 where test statistic t = , Sb1 √ Sb1 = s/ SS X , df = n-2. Two Tailed Test: Ho : β1 = 0, Ha : β1 6= 0 Reject Ho if |t| > tα/2,df where test statistic t = b1 . Sb1 Confidence Interval for slope β1 : (1- α).100 % confidence interval for β1 = (b1 −tα/2,df Sb1 , b1 −tα/2,df Sb1 ) • Population Correlation Coefficient, ρ: Note that ρ = 0 iff β1 = 0. r Test statistic for ρ: t = q 1−r2 n−2 3 ∗∗ Danger of assuming Causality ** High statistical correlation does not imply causality. Even if the correlation between X and Y is extremely high, a unit increase in X doesn’t necessarily cause an increase in Y. • Coefficient of Determination (r2 ) : The percentage of total variation in the sample Y values (measured by SSY ) that has been explained using the simple linear regression model. r2 = 1 percentage of unexplained variation. SSE SS Y , where SSE SS Y is the • Total Variation: With the realtions SS(total) = SS(f actor) + SS(error) P P P and (y−y)2 = (by-y)2 + (y−by)2 , we can derive SS Y = SSR + SSE. Then, since SSE = SSY − (SCP XY )2 , SS X we get the sum of squares of regression, SSR = (SCP XY )2 . SS X ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗∗ • Autocorrelation: A term used to describe residuals in a regression analysis that are not independent, whereby neighboring residuals have roughly the same value (positive autocorrelation) or neighboring residuals are generally very unequal in size (negative autocorrelation). • Influential Observation: An observation that has a large impact on the calculation of the least squares line. If such an observation were removed from the sample, there would be a dramatic shift in the least squares regression line. These observations can be detected using Cook’s distance measure. • Cook’s istance Measure: A statistic that is calculated for each sample observation in order to detect influential observations. • Leverage: A statistic that is computed to identify observations in the sample that have unusual values of the independent variable, X. 4