Chapter 12 Slide Notes - Introduction to Regression Analysis

CH 12: SLIDES Introduction to regression analysis Correlation is there really a relationship between two variables? Regression How do we predict values for one variable, given particular values for another variable? Regression analysis ALWAYS one dependent variable and AT LEAST ONE independent variable eg. Years of education and income. Income is the dependent variable and years of education is the independent variable Income depends on years of education but NOT conversely. Regression analysis involves using years of education to predict income. Example Education Income (in 1000's) Person 1 12 33 2 15 41 3 8 18 5 17 10 28 Income is (at least to some extent) dependent on years of education Income is therefore the response (1) variable and is plotted an vertical στις 53 50 ५० Income Years of education Regression analysis is only effective if a relationship exists between the dependent and independent variables Scatterplots can be used as a preliminary method for identifying relationships between two variables. positive relationship negative relationship no relationship Building the regression model We attempt to fit a straight line (since we are dealing with linear regression) through our observed data. The regression equation is of the form = a + y= bx y is the dependent variable and X is the independent variable a and b are chosen so that the regression line "best fits the observed data Method of Least Squares the constants residuals. a and b are chosen to minimize the sum of squared A residual (e) is defined as the difference between the actual y and y (the predicted value of y). e=y-y If we have in pairs of observations then you'll have n residuals. Sum of squared residuals n we want to minimize this! (6) эзраилом Responsive vo 5 10 15 20 19 Explanatory variable (2) Correlation Analysis A measure of the linear relationship between x and y • It is a numerical measure of the strength and direction of a linear relationship between two quantative variables We can calculate the correlation for the population (true correlation value) as well as the sample. • The sample correlation coefficient (r) can be used to justify a particular regression analysis. • The correlation coefficient always lies between -1 and 1. r = Σ(xi - x̄)(yi - ȳ) / ((n-1)s_x * s_y) r = -1 r < 0 r ≈ 0 r > 0 r = +1 Coefficient of Determination The square of the correlation coefficient (r) is known as the coefficient of determination, R². R² is the proportion of the variation in Y that is explained by the variation in X. R² lies between 0 and 1. 1 - R² is the proportion of variation in Y that is explained by factors other than X. The closer R² is to 1, the better the regression model fits the data. The closer it is to 0, the poorer the fit. e.g. R² = 0.8697 is interpreted as: 86.97% of the variation in Y is explained by the variation in X. This is a large proportion so we conclude the regression model fits well. The intercept and slope terms as random variables We use sample data to compute a and b. These are therefore statistics as they are computed from a sample. Both a and b are random variables since they vary from sample to sample. They therefore both have sampling distributions which turn out to be normal. Why? The fixed, but unknown intercept and slope population parameters could theoretically be obtained by fitting a regression model to the entire population. Since this is often not practically possible, we use our sample statistics to draw inferences about the unknown population parameters of interest.