Probability Review

13.42 Design Principles for Ocean Vehicles 1. Overview of basic probability Empirically, probability can be defined as the number of favorable outcomes divided by the total number of outcomes, in other words, the chance that an event will occur. Formally, the probability, p of an event can be described as the normalized “area” of some event within an event space, S , that contains several outcomes (events), Ai , which can include the null set, ∅ . The probability of the event space itself is equal to one, hence any other event has a probability ranging from zero (null space) to one (the whole space). Simple events are those which do not share any common area within an event space, i.e. they are non-overlapping, whereas composite events overlap (see figure 1). The probability that an event will be in the event space is one: p( S ) = 1. A2 A1 A3 A4 A1 S A3 Simple Events Ai 1. A2 A4 Composite Events Ai Simple and composite events within event space, S . 1 S DEFINE (see figure 2 for graphical representation): • • • UNION: The union of two regions defines an event that is either in A or in B or in both regions. INTERSECTION: The intersection of two regions defines an event must be in both A and B. COMPLEMENT: The complement A is everything in the event space that is not in A, i.e. A′ . A B intersection; A A B union; A B A' A not A; A' 2. Union, Intersection, and Complement. 2 B 1.1. Mutually Exclusive Events are said to be mutually exclusive if they have no outcomes in common. These are also called disjoint events. EXAMPLE: One store carries six kinds of cookies. Three kinds are made by Nabisco and three by Keebler. The cookies made by Nabisco are not made by Keebler. Observe the next person who comes into the store to buy cookies. They choose one bag. It can only be made by either Nabisco OR Keebler thus the probability that they choose one made by either company is zero. These events are mutually exclusive. p( A ∩ B) = 0 → Mutually Exclusive (1) AXIOMS: For any event A (1) p ( A) ≥ 0 (2) p( S ) = 1 (all events) (3) If A1 , A2 , A3 ,L, An are a collection of mutually exclusive events then: p ( A1 ∪ A2 ∪ A3 ∪ L ∪ An ) = ∑ i=1 p ( Ai ) n Probability can be seen as the normalized Area of the event, Ai . Since p( Ai ) = 1 − p( Ai ) ≤ 1 (2) then the probability of the null set is zero: p(∅) = 1 − p( S ) = 0. This holds since the probability of the event space, S , is exactly one. 3 (3) If A ∩ B ≠ 0 and A ∪ B = A ∪ (B ∩ A′) where A and (B ∩ A′) are mutually exclusive, then p( A ∪ B) = p( A) + p( B) − p( A ∩ B) (4) p( A ∪ B) = p ( A) + p( B ∩ A′) (5) becomes since B is simply the union of the part of B in A with the part of B not in A: B = (B ∩ A) ∪ (B ∩ A′). (6) These two parts are Mutually Exclusive thus we can sum their probabilities to get the probability of B. So p( B) = p ( B ∩ A) + p( B ∩ A′) (7) Looking back to equation 5 we can substitute in for p(B ∩ A′) with p(B) − p ( A ∩ B) . Therefore, the probability of the event A ∪ B is equal the probability of A plus the probability of event B minus the probability that A ∩ B , i.e. p( A ∪ B) = p( A) + p ( B) − p( A ∩ B). (8) Example 1: Toss a fair coin. Event A = heads and event B = tails . p( A) = 0.5; p( B) = 0.5 Example 2: If A, B, and C are the only three events in S and are mutually exclusive events, where 4 p ( A) = 49/100 and p ( B) = 48/100 then p (C ) = 3/100 . Example 3: Roll a six-sided die. Six possible outcomes, p( Ai ) = 1/ 6 . Probability of rolling an even number: p(even) = 1/ 2 = p(2) + p(4) + p(6) . 2. Conditional Probability Conditional probability is defined as the probability that a certain event will occur given that a composite event has also occurred. We write this conditional probability as p( A | B) and say "probability of A given B". Given that a composite event, M (see figure 3), has happened what is the probability that event Ai also happened? By stating that event M has happened we then have excluded all events that do not overlap with M as possible outcomes. The implication is that now the event space has “shrunk” from S to M . Therefore we must redefine the probabilities of the events such that p( M ) = 1 and all other events have p ( Ai ) = 0 if M ∩ Ai = 0 but if M ∩ Ai ≠ 0 (i.e. if Ai has some overlap with M ) then 0 ≤ p ( Ai ) ≤ 1 . The greater the overlap, the higher the probability of the event. 5 A2 A1 A3 A4 S M (composite event) 3. Composite Event M. Thus for any two events A and B with p ( B) > 0 the CONDITIONAL PROBABILITY of A given B has occurred is defined as: p( A | B) = p( A ∩ B) p( B) (9) which is conveniently rewritten as p( A ∩ B) = p( A | B) ⋅ p( B) (10) and is commonly referred to as the Multiplication Rule and is often an easier form of equation 9. Example 1: A gas station is trying to determine what the average customer needs from their station. The have determined the probability that a customer will check only his/her oil level or only his/her tire pressure and also the probability they will check both. p(check tires ) = p(T ) = 0.02 6 p(check oil ) = p( L) = 0.1 p (check both) = p ( B) = p (T ∩ L) = 0.01 Their next step is to determine the probability that a person checks their oil given they also checked their tire pressure. (1) Choose a random customer and find the probability that a customer has checked his tires given he/she checked the oil: p(T | L) = p(T ∩ L) 0.01 = = 0.1 p( L) 0.1 (11) (2) Choose a random customer and find the probability that a customer has checked his oil given he/she checked the tires: p( L | T ) = p ( L ∩ T ) 0.01 = = 0.05 p(T ) 0.2 (12) Example 2: What is the probability that the outcome of a roll of a dice is 2 ( A2 ) given that the outcome is even? Let the complex event M be the occurrence of all possible even numbers. Since a die is six sided with three possible even numbers, 2,4, and 6, the probability that an even number will be rolled is 0.5: M = A2 ∪ A4 ∪ A6 p ( M ) = 1/ 2 The probability that M intersects with event A2 , rolling a two, is p ( M ∩ A2 ) = p( A2 ) = 1/ 6 , 7 (13) since there are three possible even numbers and a 50% chance of rolling an even number. Thus the probability that a person will roll a two given that an even number is rolled is p( A2 | M ) = p( M ∩ A2 1/ 6 = = 1/ 3 = 33% 1/ 2 p( M ) (14) 3. Law of Total Probability & Bayes Theorem If all events, Ai ( i = 1 : n ), are mutually exclusive and exhaustive, then for any other event B, p( B) = p ( B | A1 ) p( A1 ) + L + p( B | An ) p ( An ) (15) or n p( B) = ∑ p( B | Ai ) p ( Ai ). (16) i=1 This is the Law of Total Probability. We can prove this simply by looking at the conditional probability of each of the events. Proof: Since the events, Ai , are mutually exclusive and exhaustive, in order for event B to occur it must exist in conjunction with exactly one of the events Ai . B = ( A1 and B ) or ( A2 and B) or L or ( An and B) = ( A1 ∩ B) ∪ L ∪ ( An ∩ B) p ( B ) = ∑ i=1 p ( Ai ∩ B ) = ∑ i=1 p ( B | Ai ) p ( Ai ) n n This formulation leads us to Bayes Theorem. Let A1 , A2 , A3 ,L, Ak be a collection of mutually exclusive events with p( Ak ) > 0 for all 8 k = 1 : n . The for any event B with p ( B) > 0 we get p( Ak | B) = p( Ak ∩ B) p( B) (17) using the multiplication rule and the Law to total Probability we get BAYES THEOREM p( Ak | B) = p ( B| Ak ) p ( Ak ) ∑ i=1 n p ( B| Ai ) p ( Ai ) ; for k = 1, 2, 3,..., n 4. Examples Let’s look at a few complex examples. Example 1: Sunken Treasure! You’ve been hired by a salvage company to determine in which of two regions the company should look to find a ship that sank in a hurricane with treasure worth billions. This is very exciting, so you pull out your probability book and get to work. You know that group X is using a side scan sonar that has a success rate of finding objects equal to 70% (probability that it finds an object is 0.7) and that group Y has equipment that has a success rate of 60%. You also know that meteorologists predict that there is an 80% probability that sunken ship and her treasure lie in Region I, at the edge of the continental shelf, and 20% probability that it is in Region II, beyond the shelf. 9 p ( X ) = PX = 0.7; p(Y ) = PY = 0.6; p( I ) = PI = 0.8; p( II ) = PI I = 0.2 The two groups have been searching the regions for some time now. What is the probability that the treasure is in region I, call this event AI , given that person Y did not find it in region I, call this event B ? p( AI | B) = p(treasure is in I | Person Y did not find it in I) = p (Y didn′t find it in I | it is in I )⋅ p ( it is in I ) p ( B| AI ) p ( AI )+ p ( B| AI ) p ( AI ) = = (0.6) PY (0.8) PI PII (0.2) (1−PY ) PI (1−PY ) PI + PII 0.4∗0.8 0.4∗0.8 + 0.2 = 0.62 = 62% Y finds in I =PYPI =0.48 is in I 1-PY (0.4) is in II Y does not find =(1-PY)PI in I given it is in I =0.32 Y did not find it in I =PII= 0.2 Probability that Y finds it in I if it is in I: PYPI Probability that Y doesn’t find it in I eventhough it � is in I: (1-PY)PI Probability that Y does not find it in I � given it is in II: PII 10 4. Probability tree for the sunken treasure problem. The next day you discover that in addition to Y, X did not find the treasure in area I. So what is the probability that X didn’t find it after Y didn’t find it and the probability that it is in area I even though both parties failed to locate it? See the probability tree in figure 5 (0.6) PY (0.8) PI (0.7) PX 1-PY (0.4) 1-P X (0.3) PII (0.2) Total probability that X does not find it in Area I given that Y did not find it there either: � PII + (1-PX)(1-PY)PI = 0.296 Probability that it is in I eventhough both X and Y didn’t find it: P(I | X,Y don’t find it) = (1-PX)(1-PY)PI PII+(1-PX)(1-PY)PI = 0.324 5. Probability tree for the sunken treasure problem–given that Y did not find it in area I. 11 5. Recap of Probability For events A & B in the p( A ∪ B) = p( A) + p( B) − p( A ∩ B) space S : p( A | B) = p ( A∩B ) p(B) p( A | B) = p( A) If A is wholly contained in B then p( A ∪ B) = p ( A) + p( B) Mutually exclusive events A,B: p ( A ∩ B) = 0 p( A | B) = 0 6. Probability given multiple trials Consider a group of n objects. We can determine the number of possible ways to pick k objects from this group at random, without keeping track of the order in which we pick the objects, as ⎛n⎞ ⎜ ⎟= ⎝k ⎠ n! k!( n − k )! Simply stated: n choose k . 12 If we perform an experiment with a success probability equal to p , then q = 1 − p is the probability of failure. If we repeat the experiment n times then the probability of k successes in n independent trials, again paying no attention to the order in which the successes are obtained, is ⎛n⎞ p(k ) = ⎜ ⎟ p k q n −k ⎝k ⎠ (18) Thus the single event containing (n − k) failures and k successes has the probability ⎛n⎞ p k q n − k and there are ⎜ ⎟ possible combinations. ⎝k ⎠ 7. Useful References There are many probability text books each slightly different. Additional textbooks are listed below. • Bertsakis and Tsikilis: Text book used for 6.431 • Devore, J "Probability and Statistics for Engineers and Scientists" • Triantafyllou and Chryssostomidis, (1980) "Environment Description, Force Prediction and Statistics for Design Applications in Ocean Engineering" Course Supplement. 13