18.05 S22 Class 6: Expectation, Variance and Standard Deviation for Continuous Random Variables

Expectation, Variance and Standard Deviation for Continuous Random Variables Class 6, 18.05 1 Learning Goals 1. Be able to compute and interpret expectation, variance, and standard deviation for continuous random variables. 2. Be able to compute and interpret quantiles for discrete and continuous random variables. 2 Introduction So far we have looked at expected value, standard deviation, and variance for discrete random variables. These summary statistics have the same meaning for continuous random variables: • The expected value 𝜇 = 𝐸[𝑋] is a measure of location or central tendency. • The standard deviation 𝜎 is a measure of the spread or scale. • The variance 𝜎2 = Var(𝑋) is the square of the standard deviation. To move from discrete to continuous, we will simply replace the sums in the formulas by integrals. We will do this carefully and go through many examples in the following sections. In the last section, we will introduce another type of summary statistic, quantiles. You may already be familiar with the 0.5 quantile of a distribution, otherwise known as the median or 50th percentile. 3 Expected value of a continuous random variable Definition: Let 𝑋 be a continuous random variable with range [𝑎, 𝑏] and probability density function 𝑓(𝑥). The expected value of 𝑋 is defined by 𝑏 𝐸[𝑋] = ∫ 𝑥𝑓(𝑥) 𝑑𝑥. 𝑎 Let’s see how this compares with the formula for a discrete random variable: 𝑛 𝐸[𝑋] = ∑ 𝑥𝑖 𝑝(𝑥𝑖 ). 𝑖=1 The discrete formula says to take a weighted sum of the values 𝑥𝑖 of 𝑋, where the weights are the probabilities 𝑝(𝑥𝑖 ). Recall that 𝑓(𝑥) is a probability density. Its units are 1 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 2 prob/(unit of 𝑋). So 𝑓(𝑥) 𝑑𝑥 represents the probability that 𝑋 is in an infinitesimal range of width 𝑑𝑥 around 𝑥. Thus we can interpret the formula for 𝐸[𝑋] as a weighted integral of the values 𝑥 of 𝑋, where the weights are the probabilities 𝑓(𝑥) 𝑑𝑥. As before, the expected value is also called the mean or average. 3.1 Examples Let’s go through several example computations. Where the solution requires an integration technique, we push the computation of the integral to the appendix. Example 1. Let 𝑋 ∼ uniform(0, 1). Find 𝐸[𝑋]. Solution: 𝑋 has range [0, 1] and density 𝑓(𝑥) = 1. Therefore, 1 1 𝑥2 1 𝐸[𝑋] = ∫ 𝑥 𝑑𝑥 = ∣ = . 2 0 2 0 Not surprisingly the mean is at the midpoint of the range. Example 2. Let 𝑋 have range [0, 2] and density 38 𝑥2 . Find 𝐸[𝑋]. Solution: 2 2 2 3 3𝑥4 3 𝐸[𝑋] = ∫ 𝑥𝑓(𝑥) 𝑑𝑥 = ∫ 𝑥3 𝑑𝑥 = ∣ = . 32 0 2 0 0 8 Does it make sense that this 𝑋 has mean is in the right half of its range? Solution: Yes. Since the probability density increases as 𝑥 increases over the range, the average value of 𝑥 should be in the right half of the range. 𝑓(𝑥) 𝑥 1 𝜇 = 1.5 𝜇 is “pulled” to the right of the midpoint 1 because there is more mass to the right. Example 3. Let 𝑋 ∼ exp(𝜆). Find 𝐸[𝑋]. Solution: The range of 𝑋 is [0, ∞) and its pdf is 𝑓(𝑥) = 𝜆e−𝜆𝑥 . So (details in appendix) ∞ ∞ 𝐸[𝑋] = ∫ −𝜆𝑥 𝑥𝜆e 𝑑𝑥 = −𝑥e 0 −𝜆𝑥 1 e−𝜆𝑥 − ∣ = . 𝜆 0 𝜆 𝑓(𝑥) = 𝜆e−𝜆𝑥 𝑥 𝜇 = 1/𝜆 Mean of an exponential random variable 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 3 Example 4. Let 𝑍 ∼ N(0, 1). Find 𝐸[𝑍]. 1 2 Solution: The range of 𝑍 is (−∞, ∞) and its pdf is 𝜙(𝑧) = √ e−𝑧 /2 . So (details in 2𝜋 appendix) ∞ ∞ 1 1 2 2 = 0. 𝐸[𝑍] = ∫ √ 𝑧e−𝑧 /2 𝑑𝑧 = − √ e−𝑧 /2 ∣ 2𝜋 2𝜋 −∞ −∞ 𝜙(𝑧) 𝑧 𝜇=0 The standard normal distribution is symmetric and has mean 0. 3.2 Properties of 𝐸[𝑋] The properties of 𝐸[𝑋] for continuous random variables are the same as for discrete ones: 1. If 𝑋 and 𝑌 are random variables on a sample space Ω then 𝐸[𝑋 + 𝑌 ] = 𝐸[𝑋] + 𝐸[𝑌 ]. (linearity I) 𝐸[𝑎𝑋 + 𝑏] = 𝑎𝐸[𝑋] + 𝑏. (linearity II) 2. If 𝑎 and 𝑏 are constants then Example 5. In this example we verify that for 𝑋 ∼ N(𝜇, 𝜎) we have 𝐸[𝑋] = 𝜇. Solution: Example (4) showed that for standard normal 𝑍, 𝐸[𝑍] = 0. We could mimic the calculation there to show that 𝐸[𝑋] = 𝜇. Instead we will use the linearity properties of 𝐸[𝑋]. In the class 5 notes on manipulating random variables we showed that if 𝑋 ∼ N(𝜇, 𝜎2 ) is a normal random variable we can standardize it: 𝑋−𝜇 𝑍= ∼ N(0, 1). 𝜎 Inverting this formula we have 𝑋 = 𝜎 𝑍 + 𝜇. The linearity of expected value now gives 𝐸[𝑋] = 𝐸[𝜎 𝑍 + 𝜇] = 𝜎 𝐸[𝑍] + 𝜇 = 𝜇 3.3 Expectation of Functions of 𝑋 This works exactly the same as the discrete case. if ℎ(𝑥) is a function then 𝑌 = ℎ(𝑋) is a random variable and ∞ 𝐸[𝑌 ] = 𝐸[ℎ(𝑋)] = ∫ ℎ(𝑥)𝑓𝑋 (𝑥) 𝑑𝑥. −∞ Example 6. Let 𝑋 ∼ exp(𝜆). Find 𝐸[𝑋 2 ]. Solution: Using integration by parts we have ∞ 2 𝐸[𝑋 ] = ∫ 0 2 −𝜆𝑥 𝑥 𝜆e 2 −𝜆𝑥 𝑑𝑥 = [−𝑥 e 2𝑥 −𝜆𝑥 2 −𝜆𝑥 ∞ 2 − e − 2e ] = 2 . 𝜆 𝜆 𝜆 0 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 4 4 Variance Now that we’ve defined expectation for continuous random variables, the definition of variance is identical to that of discrete random variables. Definition: Let 𝑋 be a continuous random variable with mean 𝜇. The variance of 𝑋 is Var(𝑋) = 𝐸[(𝑋 − 𝜇)2 ]. 4.1 Properties of Variance These are exactly the same as in the discrete case. 1. If 𝑋 and 𝑌 are independent then Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 ). 2. For constants 𝑎 and 𝑏, Var(𝑎𝑋 + 𝑏) = 𝑎2 Var(𝑋). 3. Theorem: Var(𝑋) = 𝐸[𝑋 2 ] − 𝐸[𝑋]2 = 𝐸[𝑋 2 ] − 𝜇2 . For Property 1, note carefully the requirement that 𝑋 and 𝑌 are independent. Property 3 gives a formula for Var(𝑋) that is often easier to use in hand calculations. The proofs of properties 2 and 3 are essentially identical to those in the discrete case. We will not give them here. Example 7. Let 𝑋 ∼ uniform(0, 1). Find Var(𝑋) and 𝜎𝑋 . Solution: In Example 1 we found 𝜇 = 1/2. Next we compute 1 Var(𝑋) = 𝐸[(𝑋 − 𝜇)2 ] = ∫ (𝑥 − 1/2)2 𝑑𝑥 = 0 1 . 12 Example 8. Let 𝑋 ∼ exp(𝜆). Find Var(𝑋) and 𝜎𝑋 . Solution: In Examples 3 and 6 we computed ∞ 𝐸[𝑋] = ∫ 𝑥𝜆e−𝜆𝑥 𝑑𝑥 = 0 1 𝜆 ∞ and 𝐸[𝑋 2 ] = ∫ 𝑥2 𝜆e−𝜆𝑥 𝑑𝑥 = 0 2 . 𝜆2 So by Property 3, Var(𝑋) = 𝐸[𝑋 2 ] − 𝐸[𝑋]2 = 2 1 1 − 2 = 2 2 𝜆 𝜆 𝜆 and 𝜎𝑋 = 1 . 𝜆 ∞ We could have skipped Property 3 and computed this directly from Var(𝑋) = ∫0 (𝑥 − 1/𝜆)2 𝜆e−𝜆𝑥 𝑑𝑥. Example 9. Let 𝑍 ∼ N(0, 1). Show Var(𝑍) = 1. Note: The notation for normal variables is 𝑋 ∼ N(𝜇, 𝜎2 ). This is certainly suggestive, but as mathematicians we need to prove that 𝐸[𝑋] = 𝜇 and Var(𝑋) = 𝜎2 . Above we showed 𝐸[𝑋] = 𝜇. This example shows that Var(𝑍) = 1, just as the notation suggests. In the next example we’ll show Var(𝑋) = 𝜎2 . Solution: Since 𝐸[𝑍] = 0, we have ∞ 1 2 Var(𝑍) = 𝐸[𝑍 2 ] = √ ∫ 𝑧2 e−𝑧 /2 𝑑𝑧. 2𝜋 −∞ 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 5 (using integration by parts with 𝑢 = 𝑧, 𝑣′ = 𝑧e−𝑧 2 /2 ⇒ 𝑢′ = 1, 𝑣 = −e−𝑧 2 /2 ) ∞ ∞ 1 1 2 2 = √ ( −𝑧e−𝑧 /2 ∣ ) + √ ∫ e−𝑧 /2 𝑑𝑧. −∞ 2𝜋 −∞ 2𝜋 The first term equals 0 because the exponential goes to zero much faster than 𝑧 grows at both ±∞. The second term equals 1 because it is exactly the total probability integral of the pdf 𝜑(𝑧) for N(0, 1). So Var(𝑋) = 1. Example 10. Let 𝑋 ∼ N(𝜇, 𝜎2 ). Show Var(𝑋) = 𝜎2 . Solution: This is an exercise in change of variables. Letting 𝑧 = (𝑥 − 𝜇)/𝜎, we have ∞ 1 2 2 ∫ (𝑥 − 𝜇)2 e−(𝑥−𝜇) /2𝜎 𝑑𝑥 2𝜋 𝜎 −∞ ∞ 𝜎2 2 = √ ∫ 𝑧 2 e−𝑧 /2 𝑑𝑧 = 𝜎2 . 2𝜋 −∞ Var(𝑋) = 𝐸[(𝑋 − 𝜇)2 ] = √ The integral in the last line is the same one we computed for Var(𝑍). 5 Quantiles Definition: The median of 𝑋 is the value 𝑥 for which 𝑃 (𝑋 ≤ 𝑥) = 0.5, i.e. the value of 𝑥 such that 𝑃 (𝑋 ≤ 𝑥) = 𝑃 (𝑋 ≥ 𝑥). In other words, 𝑋 has equal probability of being above or below the median, and each probability is therefore 1/2. In terms of the cdf 𝐹 (𝑥) = 𝑃 (𝑋 ≤ 𝑥), we can equivalently define the median as the value 𝑥 satisfying 𝐹 (𝑥) = 0.5. Think: What is the median of 𝑍? Solution: By symmetry, the median is 0. Example 11. Find the median of 𝑋 ∼ exp(𝜆). Solution: The cdf of 𝑋 is 𝐹 (𝑥) = 1 − e−𝜆𝑥 . So the median is the value of 𝑥 for which 𝐹 (𝑥) = 1 − e−𝜆𝑥 = 0.5.. Solving for 𝑥 we find: 𝑥 = (ln 2)/𝜆 . Think: In this case the median does not equal the mean of 𝜇 = 1/𝜆. Based on the graph of the pdf of 𝑋 can you argue why the median is to the left of the mean. Definition: The pth quantile of 𝑋 is the value 𝑞𝑝 such that 𝑃 (𝑋 ≤ 𝑞𝑝 ) = 𝑝. Notes. 1. In this notation the median is 𝑞0.5 . 2. We will usually write this in terms of the cdf: 𝐹 (𝑞𝑝 ) = 𝑝. With respect to the pdf 𝑓(𝑥), the quantile 𝑞𝑝 is the value such that there is an area of 𝑝 to the left of 𝑞𝑝 and an area of 1 − 𝑝 to the right of 𝑞𝑝 . In the examples below, note how we can represent the quantile graphically using either area of the pdf or height of the cdf. Example 12. Find the 0.6 quantile for 𝑋 ∼ 𝑈 (0, 1). 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 6 Solution: The cdf for 𝑋 is 𝐹 (𝑥) = 𝑥 on the range [0,1]. So 𝑞0.6 = 0.6. 𝑓(𝑥) 𝐹 (𝑥) left tail area = prob = 0.6 1 𝐹 (𝑞0.6 ) = 0.6 𝑥 𝑥 𝑞0.6 = 0.6 𝑞0.6 = 0.6 𝑞0.6 : left tail area = 0.6 ⇔ 𝐹 (𝑞0.6 ) = 0.6 Example 13. Find the 0.6 quantile of the standard normal distribution. Solution: We don’t have a formula for the cdf, so we use the R ‘quantile function’ qnorm. 𝑞0.6 = qnorm(0.6, 0, 1) = 0.25335 Φ(𝑧) 𝜙(𝑧) 1 𝐹 (𝑞0.6 ) = 0.6 left tail area = prob. = 0.6 𝑧 𝑧 𝑞0.6 = 0.253 𝑞0.6 = 0.253 𝑞0.6 : left tail area = 0.6 ⇔ 𝐹 (𝑞0.6 ) = 0.6 Quantiles give a useful measure of location for a random variable. We will use them more in coming lectures. 5.1 Percentiles, deciles, quartiles For convenience, quantiles are often described in terms of percentiles, deciles or quartiles. The 60th percentile is the same as the 0.6 quantile. For example you are in the 60th percentile for height if you are taller than 60 percent of the population, i.e. the probability that you are taller than a randomly chosen person is 60 percent. Likewise, deciles represent steps of 1/10. The third decile is the 0.3 quantile. Quartiles are in steps of 1/4. The third quartile is the 0.75 quantile and the 75th percentile. 6 Appendix: Integral Computation Details From Example 3 Let 𝑋 ∼ exp(𝜆). Find 𝐸[𝑋]. The range of 𝑋 is [0, ∞) and its pdf is 𝑓(𝑥) = 𝜆e−𝜆𝑥 . Therefore ∞ ∞ 𝐸[𝑋] = ∫ 𝑥𝑓(𝑥) 𝑑𝑥 = ∫ 0 0 𝜆𝑥e−𝜆𝑥 𝑑𝑥 18.05 Class 6, Expectation and Variance for Continuous Random Variables, Spring 2022 7 (using integration by parts with 𝑢 = 𝑥, 𝑣′ = 𝜆e−𝜆𝑥 ⇒ 𝑢′ = 1, 𝑣 = −e−𝜆𝑥 ) ∞ ∞ = −𝑥e−𝜆𝑥 ∣0 + ∫ ∞ e−𝜆𝑥 𝑑𝑥 0 1 e−𝜆𝑥 ∣ = . =0− 𝜆 0 𝜆 We used the fact that 𝑥e−𝜆𝑥 and e−𝜆𝑥 go to 0 as 𝑥 → ∞. From Example 4 Let 𝑍 ∼ N(0, 1). Find 𝐸[𝑍]. 1 2 The range of 𝑍 is (−∞, ∞) and its pdf is 𝜙(𝑧) = √ e−𝑧 /2 . By symmetry the mean must 2𝜋 be 0. The only mathematically tricky part is to show that the integral converges, i.e. that the mean exists at all (some random variable do not have means, but we will not encounter this very often.) For completeness we include the argument, though this is not something we will ask you to do. We first compute the integral from 0 to ∞: ∞ ∫ 0 1 𝑧𝜙(𝑧) 𝑑𝑧 = √ ∫ 2𝜋 0 ∞ 𝑧e−𝑧 2 /2 𝑑𝑧. The 𝑢-substitution 𝑢 = 𝑧 2 /2 gives 𝑑𝑢 = 𝑧 𝑑𝑧. So the integral becomes 1 √ ∫ 2𝜋 0 ∞ 𝑧e−𝑧 2 /2 1 𝑑𝑧. = √ ∫ 2𝜋 0 ∞ ∞ e−𝑢 𝑑𝑢 = −e−𝑢 |0 = 1 0 Similarly, ∫ 𝑧𝜙(𝑧) 𝑑𝑧 = −1. Adding the two pieces together gives 𝐸[𝑍] = 0. −∞ From Example 6 Let 𝑋 ∼ exp(𝜆). Find 𝐸[𝑋 2 ]. ∞ 2 ∞ 2 𝐸[𝑋 ] = ∫ 𝑥 𝑓(𝑥) 𝑑𝑥 = ∫ 0 0 𝜆𝑥2 e−𝜆𝑥 𝑑𝑥 (using integration by parts with 𝑢 = 𝑥2 , 𝑣′ = 𝜆e−𝜆𝑥 ⇒ 𝑢′ = 2𝑥, 𝑣 = −e−𝜆𝑥 ) ∞ ∞ = −𝑥2 e−𝜆𝑥 ∣0 + ∫ 2𝑥e−𝜆𝑥 𝑑𝑥 0 (the first term is 0, for the second term use integration by parts: 𝑢 = 2𝑥, 𝑣′ = e−𝜆𝑥 ⇒ −𝜆𝑥 𝑢′ = 2, 𝑣 = − e 𝜆 ) ∞ e−𝜆𝑥 ∣ +∫ = −2𝑥 𝜆 0 0 ∞ =0− 2 ∞ 2 e−𝜆𝑥 ∣ = 2. 𝜆2 0 𝜆 e−𝜆𝑥 𝑑𝑥 𝜆