Correlation, Regression, Variation and Prediction Intervals

The relationship between TWO variables Chapter 10 So far we have dealt with data obtained from one variable (either categorical or quantitative).
In this chapter we will explore the relationship between two quantitative variables.
Section 10-2: Correlation Section 10-3: Regression Section 10-4: Variation and Prediction Intervals 1 Response and Explanatory Variables
In most studies involving two variables, each of the variables has a role. We distinguish between:  the response variable - the outcome of the study  the explanatory variable - the variable that claims to explain, predict or affect the response. 2 Scatterplots In a scatterplot one axis is used to represent each of the variables, and the data are plotted as points on the graph. Typically, the explanatory or independent variable is plotted on the x axis and the response or dependent variable is plotted on the y axis. 4 Example 1: Highway Signs

A Pennsylvania research firm conducted a study in which 30 drivers (of ages 18 to 82 years old) were sampled and for each one the maximum distance at which he/she could read a newly designed sign was determined. The goal of this study was to explore the relationship between driver's age and the maximum distance at which signs were legible, and then use the study's findings to improve safety for older drivers. Since the purpose of this study is to explore the effect of age on maximum legibility distance, the explanatory variable is Age, and the response variable is Distance. 32 1 Scatterplot Example 2
Here we have two quantitative variables for each of 16 students.  
How many beers they drank Their blood alcohol level We are interested in the relationship between the two variables: how is one affected by changes in the other one? Student Number of Beers Blood Alcohol Level 1 5 0.1 2 2 0.03 3 9 0.19 6 7 0.095 7 3 0.07 9 3 0.02 11 4 0.07 13 5 0.085 4 8 0.12 5 3 0.04 8 5 0.06 10 5 0.05 12 6 0.1 14 7 0.09 15 1 0.01 16 4 0.05 8 Scatterplot example BAC 1 5 0.1 2 2 0.03 3 9 0.19 6 7 0.095 7 3 0.07 9 3 0.02 11 4 0.07 13 5 0.085 4 8 0.12 5 3 0.04 8 5 0.06 10 5 0.05 12 6 0.1 14 7 0.09 15 1 0.01 16 4 0.05 Response(dependent) variable Beers Student Some plots don’t have clear explanatory and response variables. Do calories explain sodium amounts in hot dogs? Explanatory (independent) variable 9 Describing a Scatterplot Form: general shape linear clusters no relationship nonlinear 2 Direction Strength
The strength of the relationship is determined by how closely the data follow the form of the relationship. A positive (or increasing) relationship means that an increase in one of the variables is associated with an increase in the other. A negative (or decreasing) relationship means that an increase in one of the variables is associated with a decrease in the other Deviation from the pattern Back to Example 1 Form: linear Direction: negative Outliers Strength: moderately strong do not appear to be any outliers Back to Example 2 Form: linear Direction: positive Strength:strong do not appear to be any outliers This is a weak relationship. For a particular state median household income, you can’t predict the state per capita income very well. This is a very strong relationship. The daily amount of gas consumed can be predicted quite accurately for a given temperature value. 17 3 How to scale a scatterplot
How to scale a scatterplot Same data for all four plots:
Using an inappropriate scale for a scatterplot can give an incorrect impression. The straight-line pattern in the lower plot appears stronger because of the surrounding space. Both variables should be given a similar amount of space: • Plot roughly square • Points should occupy all the plot space (no blank space) 19 Example 3 20 Example 4 Form: linear Form: linear Direction: positive Strength: weak Outliers: 3 Direction: negative Strength: medium-strong Outliers: no 21 Adding categorical variables to scatterplots 22 The correlation coefficient, “r” The correlation coefficient is a measure of the direction and strength of a linear relationship. It is calculated using the mean and the standard deviation of both the x and y variables. “+” for northeastern states “·” for midwestern states The formal name for r is the Pearson product moment correlation coefficient. It is named after the English statistician Karl Pearson (1857-1936). 23 24 4 Correlation
Calculation: r is calculated using the following formula: r=
Back to Ex.1  x − x  y − y 1 1  =   ∑ ∑ zx ⋅ zy n − 1  s x   s y  n − 1 It looks scary, I know, but here’s the basic idea: convert x and y to standardized values (z-scores), and find their average product (well, almost, divide by (n-1)). “r” ranges from −1 to +1 Caution using correlation Use correlation only for linear relationships. “r” quantifies the strength and direction of a linear relationship between two quantitative variables. Strength: How closely the points follow a straight line. Direction is positive when individuals with higher x values tend to have higher values of y. Influential points Correlations are calculated using means and standard deviations and thus are NOT resistant to outliers. Properties of r





Just moving one point away from the general trend here decreases the correlation from −0.91 to −0.75.



Correlation requires that both variables be quantitative r has no units only measures the strength of a linear relationship ranges from -1 to 1 r is negative if the form of the relationship is negative r is positive if the form of the relationship is positive |r| is closer to 1 when the correlation is strong r is unchanged if you interchange x and y r is unchanged if you make a linear change of scale (ex. from feet to inches) The correlation is heavily influenced by outliers.
5 How to find “r” using the calculator
1st step: enter you two lists (explanatory and response variables) 
STAT  EDIT  1: Edit…  L1: enter your values of the explanatory variable, L2: enter your values of the response variable
Even if two variables have a high correlation coefficient, it does not mean that the explanatory variable CAUSED the changes in the response variable. 2nd step: find the correlation coefficient  
BUT association does not imply causation! STAT  CALC  8: LinReg(a+bx) LinReg(a+bx) L1,L2 r is the correlation coefficient 31 32 Association does not imply causation!


Example 1: During the months of March and April of a certain year, the weekly weight increases of a puppy in New York were collected. For the same time frame, the retail price increases of snowshoes in Alaska were collected. The data was examined and was found to have a very strong linear correlation. The weight of a growing puppy in New York (in pounds) The retail price of snowshoes in Alaska (in dollars) 8 32.45 8.5 32.95 9 33.45 9.6 34.00 10.1 34.50 10.7 35.10 11.5 35.63 So, this must mean that the weight increase of a puppy in New York is causing snowshoe prices in Alaska to increase, or the price increases of snowshoes are causing the puppy's weight to increase. Of course this is not true! 33 Association does not imply causation! The moral of this example is:


34
Be careful what you infer from your statistical analyses. Unfortunately, usually the situation is not as obvious as this one. Be sure your relationship makes sense. Also keep in mind that other factors may be involved in a potential cause and effect relationship.


35 Example 2: In the early 1930s the relationship between the human population (response variable) of Oldenburg, Germany, and number of storks nesting in the town (explanatory variable) was investigated. The correlation coefficient turned out to be 0.97. Does this mean that storks bring babies? Can you give a possible explanation for this strong association? 36 6
The thymus example (shocking) The thymus, a gland in your neck, unlike other organs of the body, doesn’t get larger as you grow—it actually gets smaller. Imagine the situation: many infants are dying of what seem to be respiratory obstructions, so doctors begin to do autopsies on infants who die with respiratory symptoms. They have done many autopsies in the past on adults who died of various causes, so they decide to rely on those autopsy results for comparison.

What stands out most when they did autopsies on the infants is that they all have thymus glands that look too big in comparison to their body size. So they concluded that the respiratory problems are caused by an enlarged thymus. It became quite common in the early 1900s for surgeons to treat respiratory problems in children by removing the thymus. In particular, in 1912, Dr. Charles Mayo published an article recommending removal of the thymus. He made this recommendation even though a third of the children who were operated on died. What’s the lurking variable in this shocking example? 38 37 What could be a lurking variable in these examples?


Important: Association does not imply causation! There is a strong positive correlation between the foot length of K-12 students and reading scores. Students who use tutors have lower test scores than students who don’t. A survey shows a strong positive correlation between the percentage of a country's inhabitants that use cell phones and the life expectancy in that country.


One of the most common mistakes people make is when they observe a high correlation between two variables and conclude that one must be causing the other. Scatterplots and correlation do NOT demonstrate causation. It’s hard to establish the nature and direction of causation, and there is always the risk of overlooking lurking variables. 40 39 Simpson’s paradox Simpson’s Paradox


A relationship between two variables that holds for each individual value of a third variable can be changed or even reversed when the data for all values of the third variable are combined. This is Simpson’s paradox. Simpson’s paradox is an example of the effect of lurking variables on an observed association. 41 Simpson’s paradox is a severe form of confounding in which there is a reversal in the direction of an association caused by a lurking variable. Overall direction of association: positive But when we color different habitats in different colors, the data is separated by a lurking variable (different habitats) into a series of negative linear associations. 42 7 Broken down according to the lurking variable "major…" Simpson’s Paradox Male Female Example: Is acceptance into a college (response variable) predicted by gender (explanatory variable)? Consider these data: Male Female Success 198 88 Failure 162 112 Total 360 200 Male Female Proportions accepted by gender: Male success rate = 198 / 360 = 0.55 Female success rate = 88 / 200 = 0.44 Conclude: males were accepted at a higher rate than females. Business Success Failure 18 102 24 96 Success 198 88 Failure 162 112 Total 120 120 Male Female Total 360 200 Art Success Failure 180 60 64 16 Total 240 80 Male proportion = 18 / 120 = 0.15 Female proportion = 24 / 120 = 0.20 Male proportion = 180 / 240 = 0.75 Female proportion = 64 / 80 = 0.80 Therefore: males were accepted at a lower rate than females. Therefore: males were accepted at a lower rate than females. 43 Summary of causation



44 Explanatory variables Association does not imply causation! Association does not imply causation! Association does not imply causation! The issue of lurking variables and Simpson's paradox occur equally in both quantitative and categorical situations. So, in either case, be careful with your conclusion, and remember: Association does not imply causation! A researcher wants to know if taking increasing amounts of ginkgo biloba will result in increased capacities of memory ability for different students. He administers it to the students in doses of 250 milligrams, 500 milligrams, and 1000 milligrams. What is the explanatory variable in this study? a) b) c) d) Amount of ginkgo biloba given to each student. Change in memory ability. Size of the student’s brain. Whether the student takes the ginkgo biloba. 45 Numeric bivariate data Scatterplots The first step in analyzing numeric bivariate data is to Look at the following scatterplot. Choose which description BEST fits the plot. a) b) c) Measure strength of linear relationship. Create a scatterplot. Model linear relationship with regression line. a) b) c) d) e) Direction: positive, form: Direction: negative, form: Direction: positive, form: Direction: negative, form: No relationship linear, strength: strong linear, strength: strong non-linear, strength: weak non-linear, strength: weak 8 Scatterplots Scatterplots Look at the following scatterplot. Choose which description BEST fits the plot. Look at the following scatterplot. Choose which description BEST fits the plot. positive, form: negative, form: positive, form: positive, form: d) Direction: Direction: Direction: Direction: e) No relationship a) b) c) non-linear, strength: strong linear, strength: strong linear, strength: weak non-linear, strength: weak a) b) c) d) e) Direction: positive, form: Direction: negative, form: Direction: positive, form: Direction: positive, form: No relationship non-linear, strength: strong linear, strength: strong linear, strength: weak non-linear, strength: weak Scatterplots Correlation Which of the following scatterplots displays the stronger linear relationship? For which of the following situations would it be appropriate to calculate r, the correlation coefficient? a) b) c) b) Plot A Plot B c) Same for both a) d) Time spent studying for statistics exam and score on the exam. Income for county employees and their respective counties. Eye color and hair color of selected participants. Party affiliation of senators and their vote on presidential impeachment. Correlation Correlation What is a FALSE statement about r, the correlation coefficient? Which scatterplot would give a larger value for r? a) b) c) d) It is a product of z-scores of X and Y. It can range in value from –1 to 1. It measures the strength and direction of the linear relationship between X and Y. It is measured in units of the X variable. a) b) c) Plot A Plot B It would be the same for both plots. 9 Correlation Correlation tells us about True or False? Computing r as a measure of the strength of the relationship between X and Y is appropriate for the data in the following scatterplot: strength (scatter) and direction of the linear relationship between two quantitative variables. In addition, we would like to have a numerical description of how both a) b) variables vary together. For instance, is one variable increasing faster True False than the other one? And we would like to make predictions based on that numerical description. A regression line Example 1 revisited A regression line is a straight line that describes how a response variable y changes as an explanatory variable x changes. We often use a regression line to predict the value of y for a given value of x.
Example 2 revisited Which line to use?


59 But which line best describes our data? Example 1. again In most cases, no line will pass exactly through all the points in a scatterplot. Different people will draw different lines by eye. We need a way to draw a regression line that doesn’t depend on our guess as to where the line should go. We will call this “best” line the Least-squares regression line. 60 10 Least-squares Regression Line
Equation of the Least-squares Regression Line For a set of data points (x,y) the least squares regression line is a line for which the sum of squared errors is as small as possible. y$ = b0 + b1 x Book’s notation yˆ = a + bx Calculator’s notation Predicted value All we need to do is calculate the intercept a, and the slope b. Other way to find a and b: How to find a and b using the calculator

b, from statistics we already know:  b=r sy s r is the correlation x sy is the standard deviation of the response variable y sx is the the standard deviation of the explanatory variable x STAT  EDIT  1: Edit…  L1: enter your values of the explanatory variable, L2: enter your values of the response variable 2nd step: find the correlation coefficient 
First we calculate the slope of the line, 1st step: enter you two lists (explanatory and response variables) Once we know b, the slope, we can calculate a, the y-intercept: STAT  CALC  8: LinReg(a+bx) LinReg(a+bx) L1,L2 where x and y are the sample means of the x and y variables a = y − bx a is the intercept, b is the slope
63 Facts about least-squares regression Ex.1 AGAIN
The yˆ = a + bx distinction between explanatory and response variables is essential in regression.
The least-squares regression line always passes through the point ( x , y ) y a = 576 b = -3 x y$ = 576 − 3x Distance = 576 feet − 3 ⋅ Age 65 11 Prediction: Interpolation Prediction: Interpolation
The equation of the least-squares regression allows you to predict y for any x within the range studied. This is called interpolating. Predict the maximum distance at which a sign is legible for a 60 year old. Distance

= 576 feet − 3 ⋅ Age Predicted distance = 576 feet - 3 · 60 = 396 396 feet is our best prediction for the maximum distance at which a sign is legible for a 60 year old. 67 Prediction Ex.1
Predict the maximum distance at which a sign is legible for a 90 year old. Distance = 576 feet − 3 ⋅ Age
Predicted distance = 576 feet - 3 · 90 = 306 306 feet is our best prediction for the maximum distance at which a sign is legible for a 90 year old. BUT…

68 But this prediction is NOT RELIABLE. It is called EXTRAPOLATION. 69 Example 2 AGAIN Extrapolation !!! Extrapolation is the use of a regression line for predictions outside the range of x values used to obtain the line. !!! y$ = 0.0008 + 0.0144 x Nobody in the study drank 6.5 beers, but by finding the value of ŷ from the regression line for x = 6.5, we would expect a blood alcohol content of 0.094 This can be a very silly thing to do, as seen here. mg/ml. 12 Residuals Ex.1 AGAIN The distances from each point to the least-squares regression are called “residuals.” The sum of these residuals is always 0. y$ = 576 − 3 ⋅ 32 = 480 y$ y Points above the line have a positive residual. Points below the line have a negative residual. ^ Predicted y Observed y residual dist. ( y − yˆ ) = residual y − y$ = 410 − 480 = −70 74 Sum of squared errors Slope Which least-squares regression line would have a smaller sum of squared errors? Look at the following scatterplot. What would be a correct interpretation of the slope? a) b) c) a) b) c) The line in Plot A. The line in Plot B. It would be the same for both plots. d) As we increase our CO content by 1 mg, we increase the tar content by 1.01 mg. As we increase our CO content by 0.66 mg, we increase the tar content by 1.01 mg. As we increase our CO content by 0.66 mg, we increase the tar content by 0.66 mg. As we increase our CO content by 1 mg, we increase the tar content by 0.66 mg. Residuals Residuals Look at the following least-squares regression line. Compare the residuals from the two Points A and B. Residual equals c) Point A’s would be greater than Point B’s. Point A’s would be less than Point B’s. Point A’s would be equal to Point B’s. d) There is not enough information. a) b) a) b) c) d) 13 Correlation or regression Correlation or regression Which of the following measures the direction and strength of the linear association between X and Y? Which of the following makes no distinction between explanatory and response variables? a) b) Correlation Regression Correlation b) Regression a) Correlation or regression Regression line Which of the following is used for prediction? A regression line always passes through the point a) a) b) Correlation Regression b) c) d) Linear regression The following graph shows the linear relationship between diamond size and price for diamonds size 0.35 carats or less. Using this relationship to predict the price of a diamond that is 1 carat is considered a) b) c) Extrapolation. An influential observation. Prediction. 14