Basic Statistics Part 1: Descriptive Statistics

Basic Statistics Descriptive Statistics Outline 1. Descriptive statistics I I I I Sample statistics (mean, variance, percentiles) Graphs (box plot, histogram) Data transformations (log transformation, unit of measure) Correlation vs. Causation 2. Probability theory I I Conditional probabilities and independence Bayes’ theorem 3. Probability distributions I I I I Discrete and continuous probability functions Probability density function & cumulative distribution function Binomial, Poisson and Normal distribution E[X] and V[X] 4. Statistical inference I I I I I Population vs. sample Law of large numbers Central limit theorem Confidence intervals Hypothesis testing and p-values 1 / 46 Descriptive statistics I In recent years, more and better-quality data have been recorded than any other time in history. I The increasing size of data sets that are readily available to us has enabled us to adopt new and more robust statistical tools. I Rising data availability has (unfortunately) led to empirical researchers to sometimes overlook some preliminary steps, such as summarizing and visually examining their data sets. I Ignoring these preliminary steps can lead to important issues and invalidate seemingly significant results. As we will see in this and following lectures, there are ways in which we can numerically summarize a data set. Before we discuss those approaches, let’s take a quick look at what’s available to us to visualize a data set graphically. 2 / 46 Descriptive statistics Histograms I Histograms are extremely useful in getting a good graphical representation of the distribution of data. These figures consist of adjacent rectangles over discrete intervals, whose areas are the frequency of observations in the interval. I Histograms are often normalized to show the proportion (or densities) of observations that fall into non-overlapping categories. In such cases, the total area under the bins equal 1. Remark The height of each bin in a normalized histogram represents density or proportion of observations that fall into that category. These can more easily be interpreted as percentages. 3 / 46 Descriptive statistics Histograms 0 .01 .02 .03 Density .04 .05 .06 .07 .08 1960 25 I I I 30 35 40 45 50 55 60 65 Life expectancy (in years) 70 75 80 85 Approximately, what is the average life expectancy in 1960? Roughly what percentage of countries had life expectancy above 65? What proportion of countries had a life expectancy less than 55 years? 4 / 46 Descriptive statistics Histograms 0 .01 .02 .03 Density .04 .05 .06 .07 .08 1960 25 30 35 40 45 50 55 60 65 Life expectancy (in years) 70 75 80 85 5 / 46 Descriptive statistics Histograms 0 .01 .02 .03 Density .04 .05 .06 .07 .08 1990 25 30 35 40 45 50 55 60 65 Life expectancy (in years) 70 75 80 85 5 / 46 Descriptive statistics Histograms 0 .01 .02 .03 Density .04 .05 .06 .07 .08 2011 25 30 35 40 45 50 55 60 65 Life expectancy (in years) 70 75 80 85 5 / 46 Descriptive statistics The Mean and Standard Deviation I A histogram can help summarize large amounts of data, but we often like to see an even shorter (and sometimes easier to interpret) summary. This is usually provided by the mean and the standard deviation. I The mean (and median) are frequently used to find the center, whereas standard deviation measures the spread. Definition The (arithmetic) mean of a list of numbers is their sum divided by how many there are. For example, the mean of 9, 1, 2, 2, 0 is More generally, mean = x̄ = 1 n n P xi ; 9+1+2+2+0 5 = 2.8. i = 1...n i=1 6 / 46 Descriptive statistics The Mean and Standard Deviation I I The standard deviation (SD) tells us how far numbers on a list deviate from their average. Usually, most numbers are within one SD around the mean. More specifically, for normally distributed variables, about 68% of entries are within one SD of the mean and about 95% of entries are within two SDs. 68% mean mean - one SD mean + one SD 95% mean mean - two SDs mean + two SDs 7 / 46 Descriptive statistics Computing the Standard Deviation Definition p Standard Deviation = mean of (deviations from the mean)2 where deviation from mean = entry − mean And in formal notation, σ = µ = N1 (x1 + ... + xN ). q 1 N PN i=1 (xi − µ)2 , where Example: Find the SD of 20, 10, 15, 15. Answer: mean = x̄ = 20+10+15+15 = 15 Then, the deviations are 4 5, −5, 0, 0,q respectively. q √ 2 2 2 2 So, SD = 5 +(−5)4 +0 +0 = 50 12.5 ≈ 3.5 4 = Remark The SD comes out in the same units as the data. For example, if the data are a set of individuals’ heights in inches, the SD is in inches too. 8 / 46 Descriptive statistics The Root-Mean-Square Consider the following list of numbers: 0, 5, −8, 7, −3. Question: How big are these numbers? What is their mean? The mean is 0.2, but this does not tell us much about the size of the numbers–it only implies that the positive numbers slightly outweigh the negative ones. To get a better sense of their size, we could use the mean of their absolute values. Statisticians tend to use another measure, though: The root-mean-square. Definition Root −mean − square (rms) = p average of (entries)2 9 / 46 Descriptive statistics The Root-Mean-Square and Standard Deviation There is an alternative way of calculating SD using root-mean-square: Remark SD = p mean of (entries)2 − (mean of entries)2 Recall the four numbers we used earlier to calculate SD: 20, 10, 15, 15. 2 2 2 2 +15 mean of (entries)2 = 20 +10 +15 = 950 4 4 = 237.5 60 2 20+10+15+15 2 2 ) = ( 4 ) = 225 Therefore, (mean of entries) = ( 4 √ √ SD = 237.5 − 225 = 12.5 ≈ 3.5, which agrees with what we found earlier. 10 / 46 Descriptive statistics Variance I In probability theory and statistics, variance gets mentioned nearly as often as the mean and standard deviation. It is very closely related to SD and is a measure of how far a set of numbers lie from their mean. I Variance is the second moment of a distribution (mean being the first moment), and therefore, tells us about the properties of the distribution (more on these later). Definition Variance = (Std. Dev.)2 = σ 2 11 / 46 Descriptive statistics Normal Approximation for Data and Percentiles S&P 500, January 2001 - December 2001 -1 s.d. mean +1 s.d. +2 s.d. +3 s.d. +4 s.d. 10,000 15,000 Volume (thousands) 0 10 20 Frequency 30 40 50 60 -2 s.d. 5,000 20,000 25,000 Source: Yahoo! Finance and Commodity Systems, Inc. Is the normal approximation satisfactory here? 12 / 46 Descriptive statistics Normal Approximation for Data and Percentiles 2011 with normal -1 s.d. mean +1 s.d. +2 s.d. 0 .01 .02 .03 Density .04 .05 .06 .07 .08 -2 s.d. 45 50 55 60 65 70 75 80 Life expectancy (in years) 85 90 95 How about here? 13 / 46 Descriptive statistics Normal Approximation for Data and Percentiles Remark The mean and SD can be used to effectively summarize data that follow the normal curve, but these summary statistics can be much less satisfactory for data that do not follow the normal curve. In such cases, statisticians often opt for using percentiles to summarize distributions. Table. Selected percentiles for life expectancy in 2011 Percentiles Value 1 48 10 52.6 25 63 50 73.4 75 76.9 95 81.8 99 82.7 14 / 46 Descriptive statistics Calculating percentiles 1. Order all the values in your data set in ascending order (i.e. smallest to largest). 2. Select a percentile, P, that you would like to calculate and multiply it by the total number of entries in your data set, n. The value you obtain here is called the index. 3. If the index is not a whole number, round it up to the next integer. 4. Count the entries in your list of numbers starting from the smallest one until you get to the number indicated by your index. 5. This entry is the kth percentile in your data set. 15 / 46 Descriptive statistics Calculating percentiles Example Consider the following list of 5 numbers: 10, 15, 20, 25, 30. What is the entry that corresponds to the 25th percentile? What is the median? To obtain the 25th percentile, all we need to do is 0.25 × 5 = 1.25. After rounding, this value becomes 1, so 25th percentile in this case is the first entry, 10. We were also asked to obtain the median. To do this, calculate 0.5 × 5 = 2.5. Rounding this to the nearest whole number gives 3. So, the median in this case is 20. 16 / 46 Descriptive statistics Percentiles The 1st percentile of the distribution is approximately 48, meaning that the life expectancy in 1% of countries in 2011 was 48 or less, and 99% of countries had life expectancy higher than that. Similarly, the fact that 25th percentile is 63 implies that 25% of countries had life expectancy of 63 or less, whereas 75% had a longer expected lifespan. Definition Interquartile range is defined as 75th percentile − 25th percentile and is sometimes used as a measure of spread, particularly when the SD would pay too much (or too little) attention to a small percentage of cases in the tails of the distribution. From the table above, the interquartile range equals 76.9 − 63 = 13.9 (and SD was 10.14). 17 / 46 Descriptive statistics Box plots The structure of a box plot: Adjacent line (Upper adjacent value) The largest value within 75th percentile + Whiskers 75th percentile/3rd quartile (upper hinge) Box 50th percentile (median) 25th percentile/1st quartile (lower hinge) The smallest value within 25th percentile - Whiskers Adjacent line (Lower adjacent value) Entries less than the lower adjacent value 18 / 46 Descriptive statistics Box plots Are there any clear patterns emerging from summarizing the data this way? Life expectancy (in years) 70 80 90 50 60 Life expectancy by region in 2011 EAS ECS LCN MEA NAC SAS SSF Legend EAS: East Asia & Pacific ECS: Europe & Central Asia LCN: Latin America & Caribbean MEA: Middle East & North Africa NAC: North America SAS: South Asia SSF: Sub-Saharan Africa 19 / 46 Descriptive statistics Box plots We might be able to spot some patterns that developed over time if we look at different years: Life expectancy by region 25 35 Life expectancy (in years) 65 55 75 45 85 1960 EAS ECS LCN MEA NAC SAS SSF 20 / 46 Descriptive statistics Box plots We might be able to spot some patterns that developed over time if we look at different years: Life expectancy by region 25 35 Life expectancy (in years) 65 55 75 45 85 1990 EAS ECS LCN MEA NAC SAS SSF 20 / 46 Descriptive statistics Box plots We might be able to spot some patterns that developed over time if we look at different years: Life expectancy by region 25 35 Life expectancy (in years) 65 55 75 45 85 2011 EAS ECS LCN MEA NAC SAS SSF 20 / 46 Data Transformations The effects of changing the unit of measure I I I I Now that we know how to summarize a dataset, let us turn to investigating the effects of changing the unit of measure for a variable on the mean and standard deviation. Such changes in the unit of measure could be for practical reasons or based on theory, but regardless of the reason, a statistician should know what to expect. To study this, let’s consider a dataset on 200 individuals’ weights and heights. Each entry is originally reported in kg and cm, respectively, and below are some summary statistics: Table. Variable Weight (kg) Height (cm) Summary statistics Mean Standard Deviation 65.8 15.1 170.02 12.01 21 / 46 Data Transformations The effects of changing the unit of measure And here are some diagrams that summarize the distribution of the two variables. Weight Height measured in kg measured in cm mean +1 s.d. +2 s.d. -2 s.d. -1 s.d. mean +1 s.d. +2 s.d. 0 0 .01 .01 Density .02 Density .02 .03 .03 .04 .04 -2 s.d. -1 s.d. 40 50 60 70 80 90 100 110 120 130 140 150 160 170 Measured weight in kg 140 150 160 170 180 Measured height in cm 190 200 Does the normal approximation look satisfactory? 22 / 46 Data Transformations The effects of changing the unit of measure And here are some diagrams that summarize the distribution of the two variables. 50 50 Measured weight in kg 150 100 Measured height in cm 100 150 200 Height (cm) by sex 200 Weight (kg) by sex F M F M 23 / 46 Data Transformations The effects of changing the unit of measure And here are some diagrams that summarize the distribution of the two variables. Weight Weight measured in kg measured in lb -2 s.d. -1 s.d. mean +1 s.d. +2 s.d. Density .01 0 0 .01 Density .02 .03 .02 mean +1 s.d. +2 s.d. .04 -2 s.d. -1 s.d. 40 50 60 70 80 90 100 110 120 130 140 150 160 170 Measured weight in kg 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 Measured weight in pounds Do you think the mean matches the original one (in correct units)? How about the standard deviation? 24 / 46 Data Transformations The effects of changing the unit of measure And here are some diagrams that summarize the distribution of the two variables. Height Height measured in cm -1 s.d. mean measured in in +1 s.d. +2 s.d. -2 s.d. -1 s.d. mean +1 s.d. +2 s.d. 65 70 Measured height in inches 75 0 0 .02 .01 .04 Density .06 .08 Density .02 .03 .1 .04 .12 -2 s.d. 140 150 160 170 180 Measured height in cm 190 200 55 60 80 Do you think the mean matches the original one (in correct units)? How about the standard deviation? 25 / 46 Data Transformations The effects of changing the unit of measure Here are the box plots with the transformed data: Height (in) by sex 20 100 Measured height in inches 40 60 Measured weight in pounds 200 250 300 150 350 80 Weight (lb) by sex F M F M 26 / 46 Data Transformations The effects of changing the unit of measure I I I Observations made using the figures are, of course, based on what statisticians and econometricians often call "eye-balling" the data. These observations are certainly not formal, but are a crucial part of effectively analyzing any dataset. In fact, you should make plotting, investigating and eye-balling your data a habit before you dive into complicated models and overlook important features of your dataset. Now that we have made our informal observations, let’s look at the actual numbers. Variable Weight (kg) Height (cm) Weight (lb) Height (in) Mean 65.8 170.02 145.06 66.94 Table. Summary statistics SD Mean (converted) 15.1 65.8 × 2.2 ≈ 145.06 12.01 66.94 33.28 145.06/2.2 ≈ 65.8 4.73 170.02 SD (converted) 15.1 × 2.2 ≈ 33.28 4.73 33.28/2.2 ≈ 15.1 12.01 27 / 46 Data Transformations The effects of changing the unit of measure I I I Observations made using the figures are, of course, based on what statisticians and econometricians often call "eye-balling" the data. These observations are certainly not formal, but are a crucial part of effectively analyzing any dataset. In fact, you should make plotting, investigating and eye-balling your data a habit before you dive into complicated models and overlook important features of your dataset. Now that we have made our informal observations, let’s look at the actual numbers. Variable Weight (kg) Height (cm) Weight (lb) Height (in) Mean 65.8 170.02 145.06 66.94 Table. Summary statistics SD Mean (converted) 15.1 145.06 12.01 170.02 × 0.4 ≈ 66.94 33.28 65.8 4.73 66.94 × 2.5 ≈ 170.02 SD (converted) 33.28 12.01 × 0.4 ≈ 4.73 15.1 4.73 × 2.5 ≈ 12.01 27 / 46 Data Transformations The effects of changing the unit of measure We have seen that the mean and the standard deviation remain the same when we change the unit of measure, but how does variance behave? Variable Weight (kg) Height (cm) Weight (lb) Height (in) Mean 65.8 170.02 145.06 66.94 Table. Summary statistics Variance Mean (converted) 228.01 65.8 × 2.2 ≈ 145.06 144.24 66.94 1107.56 145.06/2.2 ≈ 65.8 22.37 170.02 Variance (converted) 228.01 × 2.2 ≈ 502.68 56.79 1107.56/2.2 ≈ 502.38 56.81 28 / 46 Data Transformations The effects of changing the unit of measure We have seen that the mean and the standard deviation remain the same when we change the unit of measure, but how does variance behave? Variable Weight (kg) Height (cm) Weight (lb) Height (in) Mean 65.8 170.02 145.06 66.94 Table. Summary statistics Variance Mean (converted) 228.01 145.06 144.24 170.02 × 0.4 ≈ 66.94 1107.56 65.8 22.37 66.94 × 2.54 ≈ 170.02 Variance (converted) 502.68 144.24 × 0.4 ≈ 56.79 502.38 22.37 × 2.54 ≈ 56.81 1 Note that 1 inch = 2.54 cm and similarly, 1cm = 2.54 = 0.3937in. 2 Then, 22.37 ≈ (0.3937) × 144.24. The opposite is true as well: 144.24 ≈ (2.54)2 × 22.37. And we can apply the same to the weights in kg and lbs. And in general ... 28 / 46 Data Transformations Properties of Variance ...variance is scaled by the square of the constant by which all the values are scaled. While we are at it, here are some basic properties of variance: Basic properties of variance I Variance is non-negative: Var(X ) ≥ 0 I Variance of a constant random variable is zero: P(X = a) = 1 ↔ Var(X ) = 0 I Var(aX) = a2 Var(X) I However, Var(X + a) = Var(X ) I For two random variables X and Y , Var(aX + bY ) = a 2 Var(X ) + b2 Var(Y ) + 2abCov(X , Y ) I ...but Var(X − Y ) = Var(X ) + Var(Y ) − 2Cov(X , Y ) F 29 / 46 Data Transformations Log Transformation So far, we have only worked with transformations in which we multiply each value with a constant. However, more complicated transformations are quite common in statistics and econometrics. One of the most common and useful transformations uses the natural logarithm. Definition Data transformation refers to applying a specific operation to each point in a dataset, in which each data point is replaced with the transformed one. That is, xi are replaced by yi = f (xi ). In our previous example with heights, our function, f (x), was simply f (x) = 2.54x. Now, let us study a different function: the natural logarithm. 30 / 46 Output-side real GDP at current PPPs (in mil. 2005US$) 500000.000 1000000.000 1500000.000 2000000.000 UK Real GDP 1960 1970 1980 Year 1990 Natural log of output-side real GDP at current PPPs 13 13.5 14.5 14 Data Transformations Log Transformation Log transformation in action: 2000 2010 UK Real GDP 1960 1970 1980 Year 1990 2000 2010 31 / 46 Data Transformations Log Transformation UK Real GDP Natural log of real GDP at current PPPs (in mil. 2005US$) 14 13.5 14.5 UK Real GDP 13 0.000 Output-side real GDP at current PPPs (in mil. 2005US$) 1000000.000 2000000.000 Log transformation in action: 1960 1970 1980 1990 Year 2000 2010 1960 1970 1980 1990 2000 2010 Year 31 / 46 Data Transformations Log Transformation Log Life expectancy vs. Log Real GDP 3.8 50 Life expectancy (in years) 4.2 4 Life expectancy (in years) 60 70 80 4.4 90 Life Expectancy vs. Real GDP 0.000 5000000.000 10000000.000 Output-side real GDP at current PPPs (in mil. 2005US$) 15000000.000 6 8 10 12 14 16 Natural log of output-side real GDP at current PPPs Important note The log transformation can only be used for variables that have positive values (why?). If the variable has zeros, the transformation can be applied only after these figures are replaced (usually by one-half of the smallest positive value in the data set). 32 / 46 Data Transformations Log Transformation Year: 2011 Region JPN GBR 80 EAS USA Life expectancy (in years) [linear scale] ECS LCN CHN IDN MEA RUS NAC IND SAS SSF 60 Population (in million) ZAF 10 50 100 250 40 500 1000 0 10000 20000 30000 40000 50000 60000 Real GDP per capita (at constant 2005 national prices) [linear scale] 33 / 46 Data Transformations Log Transformation Year: 1960 Region 80 EAS Life expectancy (in years) [linear scale] ECS LCN GBR USA MEA JPN NAC SAS SSF 60 Population (in million) 10 ZAF CHN 50 IDN IND 100 250 40 500 1000 156.25 312.5 625 1250 2500 5000 10000 20000 40000 80000 Real GDP per capita (at constant 2005 national prices) [log scale] 33 / 46 Data Transformations Log Transformation Year: 1990 Region Life expectancy (in years) [linear scale] 80 EAS JPN GBRUSA CHN ECS LCN MEA RUS NAC SAS IDN 60 ZAF SSF IND Population (in million) 10 50 100 250 40 500 1000 156.25 312.5 625 1250 2500 5000 10000 20000 40000 80000 Real GDP per capita (at constant 2005 national prices) [log scale] 33 / 46 Data Transformations Log Transformation Year: 2011 Region JPN GBR USA 80 EAS Life expectancy (in years) [linear scale] ECS LCN CHN IDN MEA RUS NAC IND SAS SSF 60 Population (in million) ZAF 10 50 100 250 40 500 1000 156.25 312.5 625 1250 2500 5000 10000 20000 40000 80000 Real GDP per capita (at constant 2005 national prices) [log scale] 33 / 46 Data Transformations Log Transformation and growth A useful feature of the log transformation is the interpretation of its first difference as a percentage change (for small changes). This is because ln(1 + x) ≈ x for a small x: Wolfram Alpha Strictly speaking, a percentage change in Y from period t − 1 to t−1 , which is approximately equal to period t is defined as YtY−Y t−1 ln(Yt ) − ln(Yt−1 ). And the approximation is almost exact if the percentage change is small. To see this, consider the percentage change in US GDP from 2010 to 2011: Table. US Real GDP (in mil. 2005 US$) Year GDP Percentage change ln(Yt ) ln(Y2011 ) − ln(Y2010 ) 2010 12993576 1.803507 16.379966 1.787436 2011 13227916 . 16.39784 . And the difference in percentage change is 0.01803507 − 0.01787436 = 0.00016071—a discrepancy that we might be willing to live with. 34 / 46 Examining Relationships Covariance and Correlation Our daily lives (and not just within economics) are filled with statements about the relationship between two variables. For example, we might read about a study that found that men spend more money online than women. The relationship between gender and spending more online may not be this simple, of course–income might play a role in this observed pattern. Ideally, we would like to set up an experiment in which we control the behavior of one variable (keeping everything else the same) and observe its effect on another. This is often not feasible in economics (a lot more on this later!). For the time being, let’s focus on simple correlation. 35 / 46 Examining Relationships Covariance and Correlation Scatter plots are very useful in identifying the sign and strength of the relationship between two variables. Therefore, it’s always extremely useful to plot your data and investigate what the relationship between your two variables are: 65.00 Life expectancy 70.00 75.00 80.00 85.00 Life Expectancy (in years) vs. Internet usage 0.00 20.00 40.00 60.00 Internet users per 100 people 80.00 36 / 46 Examining Relationships Covariance and Correlation But these plots can also be misleading to the eye simply by changing the scale of the axes: 95.00 Life Expectancy (in years) vs. Internet usage 0.00 20.00 40.00 60.00 Internet users per 100 people 80.00 45.00 65.00 55.00 Life expectancy 75.00 65.00 Life expectancy 70.00 75.00 80.00 85.00 85.00 Life Expectancy (in years) vs. Internet usage 0.00 20.00 40.00 60.00 80.00 Internet users per 100 people 100.00 120.00 37 / 46 Examining Relationships Covariance and Correlation Therefore, it’s best to obtain a numerical measure of the relationship. And correlation is the measure statisticians and econometricians tend to use. Definition Correlation measures the strength and direction of a linear relationship between two variables and is usually denoted as r. rx,y = ry,x = sx,y sx sy where sx,y is the sample covariance, and sx and sy are sample standard deviations of x and y, respectively. The former (i.e. sample covariance) is calculated as: N 1 X sx,y = sy,x = (xi − x̄)(yi − ȳ). N − 1 i=1 38 / 46 Examining Relationships Understanding covariance To see how a scatter diagram can be read in terms of covariance between the two variables, consider the USA: Education and GDP per capita (2010) 12 Because xUSA > x̄ and yUSA > ȳ, the term (xUSA − x̄)(yUSA − ȳ) is positive. Also, (xCOD − x̄)(yCOD − ȳ) > 0, but (xKWT − x̄)(yKWT − ȳ) < 0. USA ) 7 8 9 10 yUSA −ȳ          11 KWT 6 Log of real GDP per capita (at constant 2005 national prices) xUSA −x̄ COD 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Thus, countries located in the top-right and bottom-left quadrants have a positive effect on sx,y , whereas countries in the top-left and bottom-right quadrants have a negative effect on sx,y . Average years of total schooling Question: Should we use covariance or correlation as a more "robust" measure of the relationship? Why? 39 / 46 Examining Relationships Understanding covariance To answer this question, let’s look more closely at how covariance behaves: A positive (negative) covariance indicates that x tends to be above its mean value whenever y is above (below) its mean value. A sample covariance of zero suggests that x and y are unrelated. In our example, sx,y = 2.69. This suggests that there is a positive relationship between x and y. But what does the value of 2.69 tell us about the strength of the relationship? — Nothing. Why not? — Suppose we wanted to measure schooling in decades instead of years. That is, we generate a new variable which equals school measured in years divided by 10. The new covariance is sx,y = 0.269 which is much closer to zero. Technically speaking, covariance is not invariant to linear transformations of the variables. 40 / 46 Examining Relationships Covariance versus Correlation The sample correlation coefficient addresses this problem. While sx,y may take any value between −∞ and +∞, the correlation coefficient is standardised such that r ∈ [−1, 1]. Recall that sx,y rx,y = ry,x = sx sy where sx,y is the covariance of x and y. sx and sy are the sample standard deviations of x and y, respectively. Note that because sx > 0 and sy > 0, the sign of the sample covariance is the same as the sign of the correlation coefficient. Correlation coefficient I rx,y > 0 indicates positive correlation. I rx,y < 0 indicates negative correlation. I rx,y = 0 indicates that x and y are unrelated. I rx,y = ±1 indicates perfect positive (negative) correlation. That is, there exists an exact linear relationship between x and y of the form y = a + bx. 41 / 46 Examining Relationships Correlation In our example, rx,y = 0.7763, which indicates positive correlation (because rx,y > 0) and that the relationship is reasonably strong (because rx,y is not too far away from 1). To get a better feeling for what is "strong" and "weak", we generate 100 observations of x and y with varying degrees of correlation and plot them on a scatter diagram. To get a better feeling for what is "strong" and "weak", we generate 100 observations of x and y with varying degrees of correlation and plot them on a scatter diagram. -4 -3 -2 -1 0 x 1 2 3 4 3 -4 -3 -2 -1 y 0 1 2 3 2 1 y 0 -1 -2 -3 -4 -4 -3 -2 -1 y 0 1 2 3 4 r(x,y)=.7 4 r(x,y)=-.9 4 r(x,y)=.9 -4 -3 -2 -1 0 x 1 2 3 4 -4 -3 -2 -1 0 x 1 2 3 4 42 / 46 Examining Relationships Correlation -4 -3 -2 -1 0 x 1 2 3 4 3 -4 -3 -2 -1 y 0 1 2 3 2 1 y 0 -1 -2 -3 -4 -4 -3 -2 -1 y 0 1 2 3 4 r(x,y)=0 4 r(x,y)=0 4 r(x,y)=.3 -4 -3 -2 -1 0 x 1 2 3 4 -4 -3 -2 -1 0 x 1 2 3 4 What’s unusual about the right-most diagram here? In the right-most diagram, the correlation coefficient indicates that x and y are unrelated, but the graph implies otherwise. In fact, there is a strong quadratic relationship between x and y in this case. 43 / 46 Examining Relationships Summary I Correlation, r, measures the strength and direction of a linear relationship between two variables. I The sign of r indicates the direction of the relationship: r > 0 for a positive association and r < 0 for a negative one. I r always lies within [−1, 1] and indicates the strength of a relationship by how close it is to 1 or −1. 44 / 46 Examining Relationships Correlation vs Causation You may have already encountered the statement that Correlation does not imply causation. This is an important concept to grasp, because even a strong correlation between two variables is not enough to draw conclusions about causation. For instance, consider the following examples: 1. Do televisions increase life expectancy? 2. Are big hospitals bad for you? 3. Do firefighters make fires worse? 45 / 46 Examining Relationships Correlation vs Causation You may have already encountered the statement that Correlation does not imply causation. This is an important concept to grasp, because even a strong correlation between two variables is not enough to draw conclusions about causation. For instance, consider the following examples: 1. Do televisions increase life expectancy? There is a high positive correlation between the number of television sets per person in a country and life expectancy in that country. That is, nations with more TV sets per person have higher life expectancies. Does this imply that we could extend people’s lives in a country just by shipping TVs to them? No, of course not. The correlation between these two variables stem from the nation’s income: Richer nations have more TVs per person than poorer ones. These nations also have access to better nutrition and health care. 2. Are big hospitals bad for you? 3. Do firefighters make fires worse? 45 / 46 Examining Relationships Correlation vs Causation You may have already encountered the statement that Correlation does not imply causation. This is an important concept to grasp, because even a strong correlation between two variables is not enough to draw conclusions about causation. For instance, consider the following examples: 1. Do televisions increase life expectancy? 2. Are big hospitals bad for you? A study has found positive correlation between the size of a hospital (measured by its number of beds) and the median number of days that patients remain in the hospital. Does this mean that you can shorten a hospital stay by choosing a small hospital? 3. Do firefighters make fires worse? 45 / 46 Examining Relationships Correlation vs Causation You may have already encountered the statement that Correlation does not imply causation. This is an important concept to grasp, because even a strong correlation between two variables is not enough to draw conclusions about causation. For instance, consider the following examples: 1. Do televisions increase life expectancy? 2. Are big hospitals bad for you? 3. Do firefighters make fires worse? A magazine has observed that "there’s a strong positive correlation between the number of firefighters at a fire and the damage the fire does. So sending lots of firefighters just causes more damage." Is this reasoning flawed? 45 / 46 Examining Relationships Reverse Causality In addition to correlation feeding through a third (sometimes unobserved) variable, in economics, we often run into reverse causality problems. Earlier, we showed that real GDP per capita and education (measured by average years of schooling) are positively correlated. This could be because: 1. Rich countries can afford more (and better) education. That is, an increase in GDP per capita causes an increase in schooling. 2. More (and better) education promotes innovation and productivity. That is, an increase in schooling causes an increase in GDP per capita. The relationship between GDP per capita and education suffers from reverse causality. To reiterate, although we can make the statement that x and y are correlated, we do not know whether y is caused by x or vice versa. This is one of the central problems in empirical research in economics. In the course of the MSc, you will learn methods that allow you to identify the causal mechanisms in the relationship between y and x. 46 / 46