18.05 S22 Class 5: Continuous Random Variables

Continuous Random Variables Class 5, 18.05 1 Learning Goals 1. Know the definition of a continuous random variable. 2. Know the definition of the probability density function (pdf) and cumulative distribution function (cdf). 3. Be able to explain why we use probability density for continuous random variables. 2 Introduction We now turn to continuous random variables. All random variables assign a number to each outcome in a sample space. Whereas discrete random variables take on a discrete set of possible values, continuous random variables have a continuous set of values. Computationally, to go from discrete to continuous we simply replace sums by integrals. It will help you to keep in mind that (informally) an integral is just a continuous sum. Example 1. Since time is continuous, the amount of time Jon is early (or late) for class is a continuous random variable. Let’s go over this example in some detail. Suppose you measure how early Jon arrives to class each day (in units of minutes). That is, the outcome of one trial in our experiment is a time in minutes. We’ll assume there are random fluctuations in the exact time he shows up. Since in principle Jon could arrive, say, 3.43 minutes early, or 2.7 minutes late (corresponding to the outcome -2.7), or at any other time, the sample space consists of all real numbers. So the random variable which gives the outcome itself has a continuous range of possible values. It is too cumbersome to keep writing ‘the random variable’, so in future examples we might write: Let 𝑇 = “time in minutes that Jon is early for class on any given day.” 3 Calculus Warmup While we will assume you can compute the most familiar forms of derivatives and integrals by hand, we do not expect you to be calculus whizzes. For tricky expressions, we’ll let the computer do most of the calculating. Conceptually, you should be comfortable with two views of a definite integral. 𝑏 1. ∫ 𝑓(𝑥) 𝑑𝑥 = area under the curve 𝑦 = 𝑓(𝑥). 𝑎 𝑏 2. ∫ 𝑓(𝑥) 𝑑𝑥 = ‘sum of 𝑓(𝑥) 𝑑𝑥’. 𝑎 1 18.05 Class 5, Continuous Random Variables, Spring 2022 2 The connection between the two is: 𝑛 area ≈ sum of rectangle areas = 𝑓(𝑥1 )Δ𝑥 + 𝑓(𝑥2 )Δ𝑥 + … + 𝑓(𝑥𝑛 )Δ𝑥 = ∑ 𝑓(𝑥𝑖 )Δ𝑥. 1 As the width Δ𝑥 of the intervals gets smaller the approximation becomes better. 𝑦 𝑦 𝑦 = 𝑓(𝑥) 𝑎 𝑏 𝑦 = 𝑓(𝑥) Area = 𝑓(𝑥𝑖 )Δ𝑥 𝑥0 𝑥1 𝑥2 𝑎 𝑥 Δ𝑥 ⋯ 𝑥𝑛 𝑏 𝑥 Area is approximately the sum of rectangles Note: In calculus you learned to compute integrals by finding antiderivatives. This is important for calculations, but don’t confuse this method for the reason we use integrals. Our interest in integrals comes primarily from its interpretation as a ‘sum’ and to a much lesser extent its interpretation as area. 4 Continuous Random Variables and Probability Density Functions A continuous random variable takes a range of values, which may be finite or infinite in extent. Here are a few examples of ranges: [0, 1], [0, ∞), (−∞, ∞), [𝑎, 𝑏]. Definition: A random variable 𝑋 is continuous if there is a function 𝑓(𝑥) such that for any 𝑐 ≤ 𝑑 we have 𝑑 𝑃 (𝑐 ≤ 𝑋 ≤ 𝑑) = ∫ 𝑓(𝑥) 𝑑𝑥. (1) 𝑐 The function 𝑓(𝑥) is called the probability density function (pdf). The pdf always satisfies the following properties: 1. 𝑓(𝑥) ≥ 0 (𝑓 is nonnegative). ∞ 2. ∫ 𝑓(𝑥) 𝑑𝑥 = 1 (This is equivalent to: 𝑃 (−∞ < 𝑋 < ∞) = 1). −∞ The probability density function 𝑓(𝑥) of a continuous random variable is the analogue of the probability mass function 𝑝(𝑥) of a discrete random variable. Here are two important differences: 1. Unlike 𝑝(𝑥), the pdf 𝑓(𝑥) is not a probability. You have to integrate it to get probability. (See section 4.2 below.) 2. Since 𝑓(𝑥) is not a probability, there is no restriction that 𝑓(𝑥) be less than or equal to 1. 18.05 Class 5, Continuous Random Variables, Spring 2022 3 Note: In Property 2, we integrated over (−∞, ∞) since we did not know the range of values taken by 𝑋. Formally, this makes sense because we just define 𝑓(𝑥) to be 0 outside of the range of 𝑋. In practice, we would integrate between bounds given by the range of 𝑋. 4.1 Graphical View of Probability If you graph the probability density function of a continuous random variable 𝑋 then 𝑃 (𝑐 ≤ 𝑋 ≤ 𝑑) = area under the graph between 𝑐 and 𝑑. 𝑓(𝑥) 𝑃 (𝑐 ≤ 𝑋 ≤ 𝑑) 𝑐 𝑥 𝑑 Think: What is the total area under the pdf 𝑓(𝑥)? 4.2 The terms ‘probability mass’ and ‘probability density’ Why do we use the terms mass and density to describe the pmf and pdf? What is the difference between the two? The simple answer is that these terms are completely analogous to the mass and density you saw in physics and calculus. We’ll review this first for the probability mass function and then discuss the probability density function. Mass as a sum: If masses 𝑚1 , 𝑚2 , 𝑚3 , and 𝑚4 are set in a row at positions 𝑥1 , 𝑥2 , 𝑥3 , and 𝑥4 , then the total mass is 𝑚1 + 𝑚2 + 𝑚3 + 𝑚4 . 𝑚1 𝑥1 𝑚2 𝑥2 𝑚3 𝑥3 𝑚4 𝑥4 𝑥 We can define a ‘mass function’ 𝑝(𝑥) with 𝑝(𝑥𝑗 ) = 𝑚𝑗 for 𝑗 = 1, 2, 3, 4, and 𝑝(𝑥) = 0 otherwise. In this notation the total mass is 𝑝(𝑥1 ) + 𝑝(𝑥2 ) + 𝑝(𝑥3 ) + 𝑝(𝑥4 ). The probability mass function behaves in exactly the same way, except it has the dimension of probability instead of mass. Mass as an integral of density: Suppose you have a rod of length 𝐿 meters with varying density 𝑓(𝑥) kg/m. (Note the units are mass/length.) Δ𝑥 0 𝑥1 𝑥2 𝑥3 𝑥𝑖 𝑥𝑛 = 𝐿 𝑥 mass of 𝑖th piece ≈ 𝑓(𝑥𝑖 )Δ𝑥 18.05 Class 5, Continuous Random Variables, Spring 2022 4 If the density varies continuously, we must find the total mass of the rod by integration: 𝐿 total mass = ∫ 𝑓(𝑥) 𝑑𝑥. 0 This formula comes from dividing the rod into small pieces and ’summing’ up the mass of each piece. That is: 𝑛 total mass ≈ ∑ 𝑓(𝑥𝑖 ) Δ𝑥 𝑖=1 In the limit as Δ𝑥 goes to zero the sum becomes the integral. The probability density function behaves exactly the same way, except it has units of probability/(unit 𝑥) instead of kg/m. Indeed, equation (1) is exactly analogous to the above integral for total mass. While we’re on a physics kick, note that for both discrete and continuous random variables, the expected value is simply the center of mass or balance point. Example 2. Suppose 𝑋 has pdf 𝑓(𝑥) = 3 on [0, 1/3] (this means 𝑓(𝑥) = 0 outside of [0, 1/3]). Graph the pdf and compute 𝑃 (0.1 ≤ 𝑋 ≤ 0.2) and 𝑃 (0.1 ≤ 𝑋 ≤ 1). Solution: 𝑃 (0.1 ≤ 𝑋 ≤ 0.2) is shown below at left. We can compute the integral: 0.2 𝑃 (0.1 ≤ 𝑋 ≤ 0.2) = ∫ 0.2 𝑓(𝑥) 𝑑𝑥 = ∫ 0.1 3 𝑑𝑥 = 0.3. 0.1 Or we can find the area geometrically: area of rectangle = 3 ⋅ 0.1 = 0.3. 𝑃 (0.1 ≤ 𝑋 ≤ 1) is shown below at right. Since there is only area under 𝑓(𝑥) up to 1/3, we have 𝑃 (0.1 ≤ 𝑋 ≤ 1) = 3 ⋅ (1/3 − 0.1) = 0.7. 𝑓(𝑥) 3 .1 .2 1/3 𝑓(𝑥) 3 𝑥 𝑃 (0.1 ≤ 𝑋 ≤ 0.2) .1 1/3 1 𝑃 (0.1 ≤ 𝑋 ≤ 1) Think: In the previous example 𝑓(𝑥) takes values greater than 1. Why does this not violate the rule that probabilities are always between 0 and 1? Note on notation. We can define a random variable by giving its range and probability density function. For example we might say, let 𝑋 be a random variable with range [0,1] 𝑥 18.05 Class 5, Continuous Random Variables, Spring 2022 5 and pdf 𝑓(𝑥) = 𝑥/2. Implicitly, this means that 𝑋 has no probability density outside of the given range. If we wanted to be absolutely rigorous, we would say explicitly that 𝑓(𝑥) = 0 outside of [0,1], but in practice this won’t be necessary. Example 3. Let 𝑋 be a random variable with range [0,1] and pdf 𝑓(𝑥) = 𝐶𝑥2 . What is the value of 𝐶? Solution: Since the total probability must be 1, we have 1 1 ∫ 𝑓(𝑥) 𝑑𝑥 = 1 ∫ 𝐶𝑥2 𝑑𝑥 = 1. ⇔ 0 0 By evaluating the integral, the equation at right becomes 𝐶/3 = 1 ⇒ 𝐶=3. Note: We say the constant 𝐶 above is needed to normalize the density so that the total probability is 1. Example 4. Let 𝑋 be the random variable in the Example 3. Find 𝑃 (𝑋 ≤ 1/2). 1/2 Solution: 𝑃 (𝑋 ≤ 1/2) = ∫ 0 1/2 3𝑥2 𝑑𝑥 = 𝑥3 ∣0 = 1 . 8 Think: For this 𝑋 (or any continuous random variable): • What is 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑎)? • What is 𝑃 (𝑋 = 0)? • Does 𝑃 (𝑋 = 𝑎) = 0 mean that 𝑋 can never equal 𝑎? In words the above questions get at the fact that the probability that a random person’s height is exactly 5’9” (to infinite precision, i.e. no rounding!) is 0. Yet it is still possible that someone’s height is exactly 5’9”. So the answers to the thinking questions are 0, 0, and No. 4.3 Cumulative Distribution Function The cumulative distribution function (cdf) of a continuous random variable 𝑋 is defined in exactly the same way as the cdf of a discrete random variable. 𝐹 (𝑏) = 𝑃 (𝑋 ≤ 𝑏). Note well that the definition is about probability. When using the cdf you should first think of it as a probability. Then when you go to calculate it you can use 𝑏 𝐹 (𝑏) = 𝑃 (𝑋 ≤ 𝑏) = ∫ 𝑓(𝑥) 𝑑𝑥, where 𝑓(𝑥) is the pdf of 𝑋. −∞ Notes: 1. For discrete random variables, we defined the cumulative distribution function but did 18.05 Class 5, Continuous Random Variables, Spring 2022 6 not have much occasion to use it. The cdf plays a far more prominent role for continuous random variables. 2. As before, we started the integral at −∞ because we did not know the precise range of 𝑋. Formally, this still makes sense since 𝑓(𝑥) = 0 outside the range of 𝑋. In practice, we’ll know the range and start the integral at the start of the range. 3. In practice we often say ‘𝑋 has distribution 𝐹 (𝑥)’ rather than ‘𝑋 has cumulative distribution function 𝐹 (𝑥).’ Example 5. Find the cumulative distribution function for the density in Example 2. 𝑎 𝑎 Solution: For 𝑎 in [0,1/3] we have 𝐹 (𝑎) = ∫ 𝑓(𝑥) 𝑑𝑥 = ∫ 3 𝑑𝑥 = 3𝑎. 0 0 Since 𝑓(𝑥) is 0 outside of [0,1/3] we know 𝐹 (𝑎) = 𝑃 (𝑋 ≤ 𝑎) = 0 for 𝑎 < 0 and 𝐹 (𝑎) = 1 for 𝑎 > 1/3. Putting this all together we have ⎧0 { 𝐹 (𝑎) = 3𝑎 ⎨ {1 ⎩ if 𝑎 < 0 if 0 ≤ 𝑎 ≤ 1/3 if 1/3 < 𝑎. Here are the graphs of 𝑓(𝑥) and 𝐹 (𝑥). 𝐹 (𝑥) 1 𝑓(𝑥) 3 1/3 𝑥 𝑥 1/3 Note the different scales on the vertical axes. Remember that the vertical axis for the pdf represents probability density and that of the cdf represents probability. Example 6. Find the cdf for the pdf in Example 3, 𝑓(𝑥) = 3𝑥2 on [0, 1]. Suppose 𝑋 is a random variable with this distribution. Find 𝑃 (𝑋 < 1/2). 𝑎 Solution: 𝑓(𝑥) = 3𝑥2 on [0,1] ⇒ 𝐹 (𝑎) = ∫ 3𝑥2 𝑑𝑥 = 𝑎3 on [0,1]. Therefore, 0 ⎧0 { 𝐹 (𝑎) = ⎨𝑎3 {1 ⎩ if 𝑎 < 0 if 0 ≤ 𝑎 ≤ 1 if 1 < 𝑎 Thus, 𝑃 (𝑋 < 1/2) = 𝐹 (1/2) = 1/8. Here are the graphs of 𝑓(𝑥) and 𝐹 (𝑥): 1 3 𝑓(𝑥) 𝐹 (𝑥) 𝑥 1 𝑥 1 18.05 Class 5, Continuous Random Variables, Spring 2022 4.4 7 Properties of cumulative distribution functions Here is a summarry of the most important properties of cumulative distribution functions (cdf) 1. (Definition) 𝐹 (𝑥) = 𝑃 (𝑋 ≤ 𝑥) 2. 0 ≤ 𝐹 (𝑥) ≤ 1 3. 𝐹 (𝑥) is non-decreasing, i.e. if 𝑎 ≤ 𝑏 then 𝐹 (𝑎) ≤ 𝐹 (𝑏). 4. lim 𝐹 (𝑥) = 1 𝑥→∞ and lim 𝐹 (𝑥) = 0 𝑥→−∞ 5. 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) = 𝐹 (𝑏) − 𝐹 (𝑎) 6. 𝐹 ′ (𝑥) = 𝑓(𝑥). Properties 2, 3, 4 are identical to those for discrete distributions. The graphs in the previous examples illustrate them. Property 5 can be seen algebraically: 𝑏 𝑎 𝑏 ∫ 𝑓(𝑥) 𝑑𝑥 = ∫ 𝑓(𝑥) 𝑑𝑥 + ∫ 𝑓(𝑥) 𝑑𝑥 −∞ 𝑏 −∞ 𝑏 𝑎 𝑎 ⇔ ∫ 𝑓(𝑥) 𝑑𝑥 = ∫ 𝑓(𝑥) 𝑑𝑥 − ∫ 𝑓(𝑥) 𝑑𝑥 𝑎 −∞ −∞ ⇔ 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) = 𝐹 (𝑏) − 𝐹 (𝑎). Property 5 can also be seen geometrically. The orange region below represents 𝐹 (𝑏) and the striped region represents 𝐹 (𝑎). Their difference is 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏). 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) 𝑎 𝑏 𝑥 Property 6 is the fundamental theorem of calculus. 4.5 Probability density as a dartboard We find it helpful to think of sampling values from a continuous random variable as throwing darts at a funny dartboard. Consider the region underneath the graph of a pdf as a dartboard. Divide the board into small equal size squares and suppose that when you throw a dart you are equally likely to land in any of the squares. The probability the dart lands in a given region is the fraction of the total area under the curve taken up by the region. Since the total area equals 1, this fraction is just the area of the region. If 𝑋 represents the 𝑥-coordinate of the dart, then the probability that the dart lands with 𝑥-coordinate between 𝑎 and 𝑏 is just 𝑏 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) = area under 𝑓(𝑥) between 𝑎 and 𝑏 = ∫ 𝑓(𝑥) 𝑑𝑥. 𝑎