18.05 S22 Class 2: Probability: Terminology and Examples

Probability: Terminology and Examples Class 2, 18.05 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to organize a scenario with randomness into an experiment and sample space. 3. Be able to make basic computations using a probability function. 2 2.1 Terminology Probability cast list • Experiment: a repeatable procedure with well-defined possible outcomes. • Sample space: the set of all possible outcomes. We usually denote the sample space by Ω, sometimes by 𝑆. • Event: a subset of the sample space. • Probability function: a function giving the probability for each outcome. Later in the course we will learn about • Probability density: a continuous distribution of probabilities. • Random variable: a random numerical outcome. 2.2 Simple examples Example 1. Toss a fair coin. Experiment: toss the coin, report if it lands heads or tails. Sample space: Ω = {𝐻, 𝑇 }. Probability function: 𝑃 (𝐻) = 0.5, 𝑃 (𝑇 ) = 0.5. Example 2. Toss a fair coin 3 times. Experiment: toss the coin 3 times, list the results. Sample space: Ω = {𝐻𝐻𝐻, 𝐻𝐻𝑇 , 𝐻𝑇 𝐻, 𝐻𝑇 𝑇 , 𝑇 𝐻𝐻, 𝑇 𝐻𝑇 , 𝑇 𝑇 𝐻, 𝑇 𝑇 𝑇 }. Probability function: Each outcome is equally likely with probability 1/8. For small sample spaces we can put the set of outcomes and probabilities into a probability table. Outcomes HHH HHT HTH HTT THH THT TTH TTT Probability 1/8 1/8 1/8 1/8 1/8 1/8 1/8 1/8 1 18.05 Class 2, Probability: Terminology and Examples, Spring 2022 2 Example 3. Measure the mass of a proton Experiment: follow some defined procedure to measure the mass and report the result. Sample space: Ω = [0, ∞), i.e. in principle we can get any positive value. Probability function: since there is a continuum of possible outcomes there is no probability function. Instead we need to use a probability density, which we will learn about later in the course. Example 4. Taxis (An infinite discrete sample space) Experiment: count the number of taxis that pass 77 Mass. Ave during an 18.05 class. Sample space: Ω = {0, 1, 2, 3, 4, …}. This is often modeled with the following probability function known as the Poisson distribution. (Do not worry about mastering the Poisson distribution just yet): 𝑃 (𝑘) = e−𝜆 𝜆𝑘 , 𝑘! where 𝜆 is the average number of taxis. We can put this in a table: Outcome Probability 0 1 2 3 … k … e−𝜆 e−𝜆 𝜆 e−𝜆 𝜆2 /2 e−𝜆 𝜆3 /3! … e−𝜆 𝜆𝑘 /𝑘! … ∞ Question: Accepting that this is a valid probability function, what is ∑ e−𝜆 𝑘=0 𝜆𝑘 ? 𝑘! Solution: This is the total probability of all possible outcomes, so the sum equals 1. ∞ 𝜆𝑛 (Note, this also follows from the Taylor series e𝜆 = ∑ .) 𝑛! 𝑛=0 In a given setup there can be more than one reasonable choice of sample space. Here is a simple example. Example 5. Two dice (Choice of sample space) Suppose you roll one die. Then the sample space and probability function are Outcome Probability: 1 1/6 2 1/6 3 1/6 4 1/6 5 1/6 6 1/6 Now suppose you roll two dice. What should be the sample space? Here are two options. 1. Record the pair of numbers showing on the dice (first die, second die). 2. Record the sum of the numbers on the dice. In this case there are 11 outcomes {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}. These outcomes are not all equally likely. As above, we can put this information in tables. For the first case, the sample space is the product of the sample spaces for each die {(1, 1), (2, 1), (3, 1), … (6, 6)} Each of the 36 outcomes is equally likely. (Why 36 outcomes?) For the probability function we will make a two dimensional table with the rows corresponding to the number on the first die, the columns the number on the second die and the entries the probability. 18.05 Class 2, Probability: Terminology and Examples, Spring 2022 Die 1 1 2 3 4 5 6 1 1/36 1/36 1/36 1/36 1/36 1/36 2 1/36 1/36 1/36 1/36 1/36 1/36 Die 2 3 4 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 5 1/36 1/36 1/36 1/36 1/36 1/36 3 6 1/36 1/36 1/36 1/36 1/36 1/36 Two dice in a two dimensional table In the second case we can present outcomes and probabilities in our usual table. outcome probability 2 1/36 3 2/36 4 3/36 5 4/36 6 5/36 7 6/36 8 5/36 9 4/36 10 3/36 11 2/36 12 1/36 The sum of two dice Think: What is the relationship between the two probability tables above? We will see that the best choice of sample space depends on the context. For now, simply note that given the outcome as a pair of numbers it is easy to find the sum. Note. Listing the experiment, sample space and probability function is a good way to start working systematically with probability. It can help you avoid some of the common pitfalls in the subject. Events. An event is a collection of outcomes, i.e. an event is a subset of the sample space Ω. This sounds odd, but it actually corresponds to the common meaning of the word. Example 6. Using the setup in Example ?? we would describe the event that you get exactly two heads in words by 𝐸 = ‘exactly 2 heads’. Written as a subset this becomes 𝐸 = {𝐻𝐻𝑇 , 𝐻𝑇 𝐻, 𝑇 𝐻𝐻}. You should get comfortable moving between describing events in words and as subsets of the sample space. The probability of an event 𝐸 is computed by adding up the probabilities of all of the outcomes in 𝐸. In this example each outcome has probability 1/8, so we have 𝑃 (𝐸) = 3/8. 2.3 Definition of a discrete sample space Definition. A discrete sample space is one that is listable, it can be either finite or infinite. Examples. {H, T}, {1, 2, 3}, {1, 2, 3, 4, …}, {2, 3, 5, 7, 11, 13, 17, …} are all discrete sets. The first two are finite and the last two are infinite. Example. The interval 0 ≤ 𝑥 ≤ 1 is not discrete, rather it is continuous. We will deal with continuous sample spaces in a few days. 18.05 Class 2, Probability: Terminology and Examples, Spring 2022 2.4 4 The probability function So far we’ve been using a casual definition of the probability function. Let’s give a more precise one. Careful definition of the probability function. For a discrete sample space 𝑆 a probability function 𝑃 assigns to each outcome 𝜔 a number 𝑃 (𝜔) called the probability of 𝜔. 𝑃 must satisfy two rules: • Rule 1. 0 ≤ 𝑃 (𝜔) ≤ 1 (probabilities are between 0 and 1). • Rule 2. The sum of the probabilities of all possible outcomes is 1 (something must occur) In symbols, Rule 2 says: if 𝑆 = {𝜔1 , 𝜔2 , … , 𝜔𝑛 } then 𝑃 (𝜔1 ) + 𝑃 (𝜔2 ) + … + 𝑃 (𝜔𝑛 ) = 1. Or, 𝑛 using summation notation: ∑ 𝑃 (𝜔𝑗 ) = 1. 𝑗=1 The probability of an event 𝐸 is the sum of the probabilities of all the outcomes in 𝐸. That is, 𝑃 (𝐸) = ∑ 𝑃 (𝜔). 𝜔∈𝐸 Think: Check Rules 1 and 2 on Examples 1 and 2 above. Example 7. Flip until heads (A classic example) Suppose we have a coin with probability 𝑝 of heads and we have the following scenario. Experiment: Toss the coin until the first heads. Report the number of tosses. Sample space: Ω = {1, 2, 3, …}. Probability function: 𝑃 (𝑛) = (1 − 𝑝)𝑛−1 𝑝. Challenge 1: show the sum of all the probabilities equals 1 (hint: geometric series). Challenge 2: justify the formula for 𝑃 (𝑛) (we will do this soon). Stopping problems. The previous toy example is an uncluttered version of a general class of problems called stopping rule problems. A stopping rule is a rule that tells you when to end a certain process. In the toy example above the process was flipping a coin and we stopped after the first heads. A more practical example is a rule for ending a series of medical treatments. Such a rule could depend on how well the treatments are working, how the patient is tolerating them and the probability that the treatments would continue to be effective. One could ask about the probability of stopping within a certain number of treatments or the average number of treatments you should expect before stopping. 3 Some rules of probability For events 𝐴, 𝐿 and 𝑅 contained in a sample space Ω. Rule 1. 𝑃 (𝐴𝑐 ) = 1 − 𝑃 (𝐴). Rule 2. If 𝐿 and 𝑅 are disjoint then 𝑃 (𝐿 ∪ 𝑅) = 𝑃 (𝐿) + 𝑃 (𝑅). 18.05 Class 2, Probability: Terminology and Examples, Spring 2022 5 Rule 3. If 𝐿 and 𝑅 are not disjoint, we have the inclusion-exclusion principle: 𝑃 (𝐿 ∪ 𝑅) = 𝑃 (𝐿) + 𝑃 (𝑅) − 𝑃 (𝐿 ∩ 𝑅) We visualize these rules using Venn diagrams. 𝐴𝑐 𝐴 Ω = 𝐴 ∪ 𝐴𝑐 , no overlap 𝐿 𝑅 𝐿 ∪ 𝑅, no overlap 𝐿 𝑅 𝐿 ∪ 𝑅, overlap = 𝐿 ∩ 𝑅 We can also justify them logically. Rule 1: 𝐴 and 𝐴𝑐 split Ω into two non-overlapping regions. Since the total probability 𝑃 (Ω) = 1 this rule says that the probabiity of 𝐴 and the probability of ’not 𝐴’ are complementary, i.e. sum to 1. Rule 2: 𝐿 and 𝑅 split 𝐿 ∪ 𝑅 into two non-overlapping regions. So the probability of 𝐿 ∪ 𝑅 is is split between 𝑃 (𝐿) and 𝑃 (𝑅) Rule 3: In the sum 𝑃 (𝐿) + 𝑃 (𝑅) the overlap 𝑃 (𝐿 ∩ 𝑅) gets counted twice. So 𝑃 (𝐿) + 𝑃 (𝑅) − 𝑃 (𝐿 ∩ 𝑅) counts everything in the union exactly once. Think: Rule 2 is a special case of Rule 3. For the following examples suppose we have an experiment that produces a random integer between 1 and 20. The probabilities are not necessarily uniform, i.e., not necessarily the same for each outcome. Example 8. If the probability of an even number is 0.6 what is the probability of an odd number? Solution: Since being odd is complementary to being even, the probability of being odd is 1-0.6 = 0.4. Let’s redo this example a bit more formally, so you see how it’s done. First, so we can refer to it, let’s name the random integer 𝑋. Let’s also name the event ‘𝑋 is even’ as 𝐴. Then the event ‘𝑋 is odd’ is 𝐴𝑐 . We are given that 𝑃 (𝐴) = 0.6. Therefore 𝑃 (𝐴𝑐 ) = 1 − 0.6 = 0.4 . Example 9. Consider the 2 events, 𝐴: ‘𝑋 is a multiple of 2’; 𝐵: ‘𝑋 is odd and less than 10’. Suppose 𝑃 (𝐴) = 0.6 and 𝑃 (𝐵) = 0.25. (i) What is 𝐴 ∩ 𝐵? (ii) What is the probability of 𝐴 ∪ 𝐵? Solution: (i) Since all numbers in 𝐴 are even and all numbers in 𝐵 are odd, these events are disjoint. That is, 𝐴 ∩ 𝐵 = ∅. (ii) Since 𝐴 and 𝐵 are disjoint 𝑃 (𝐴 ∪ 𝐵) = 𝑃 (𝐴) + 𝑃 (𝐵) = 0.85. Example 10. Let 𝐴, 𝐵 and 𝐶 be the events 𝑋 is a multiple of 2, 3 and 6 respectively. If 𝑃 (𝐴) = 0.6, 𝑃 (𝐵) = 0.3 and 𝑃 (𝐶) = 0.2 what is 𝑃 (𝐴 or 𝐵)? Solution: Note two things. First we used the word ‘or’ which means union: ‘𝐴 or 𝐵’ = 𝐴 ∪ 𝐵. Second, an integer is divisible by 6 if and only if it is divisible by both 2 and 3. 18.05 Class 2, Probability: Terminology and Examples, Spring 2022 This translates into 𝐶 = 𝐴 ∩ 𝐵. So the inclusion-exclusion principle says 𝑃 (𝐴 ∪ 𝐵) = 𝑃 (𝐴) + 𝑃 (𝐵) − 𝑃 (𝐴 ∩ 𝐵) = 0.6 + 0.3 − 0.2 = 0.7 . 6