18.05 S22 Class 13: Bayesian Updating with Continuous Priors

Bayesian Updating with Continuous Priors Class 13, 18.05 1 Learning Goals 1. Understand a parameterized family of distributions as representing a continuous range of hypotheses for the observed data. 2. Be able to state Bayes’ theorem and the law of total probability for continous densities. 3. Be able to apply Bayes’ theorem to update a prior probability density function to a posterior pdf given data and a likelihood function. 4. Be able to interpret and compute posterior predictive probabilities. 2 Introduction Up to now we have only done Bayesian updating when we had a finite number of hypothesis, e.g. our dice example had five hypotheses (4, 6, 8, 12 or 20 sides). Now we will study Bayesian updating when there is a continuous range of hypotheses. The Bayesian update process will be essentially the same as in the discrete case. As usual when moving from discrete to continuous we will need to replace the probability mass function by a probability density function, and sums by integrals. The first few sections of this note are devoted to working with pdfs. In particular we will cover the law of total probability and Bayes’ theorem. We encourage you to focus on how these are essentially identical to the discrete versions. After that, we will apply Bayes’ theorem and the law of total probability to Bayesian updating. 3 Examples with continuous ranges of hypotheses Here are three standard examples with continuous ranges of hypotheses. Example 1. Suppose you have a system that can succeed or fail with probability 𝑝. Then we can hypothesize that 𝑝 is anywhere in the range [0, 1]. That is, we have a continuous range of hypotheses. We will often model this example with a ‘bent’ coin with unknown probability 𝑝 of heads. Example 2. The lifetime of a certain isotope is modeled by an exponential distribution exp(𝜆). In principle, the mean lifetime 1/𝜆 can be any real number in (0, ∞). Example 3. We are not restricted to a single parameter. In principle, the parameters 𝜇 and 𝜎 of a normal distribution can be any real numbers in (−∞, ∞) and (0, ∞), respectively. If we model gestational length for single births by a normal distribution, then from millions of data points we know that 𝜇 is about 40 weeks and 𝜎 is about one week. 1 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 2 In all of these examples we modeled the random process giving rise to the data by a distribution with parameters –called a parametrized distribution. Every possible choice of the parameter(s) is a hypothesis, e.g. we can hypothesize that the probability of succcess in Example 1 is 𝑝 = 0.7313. We have a continuous set of hypotheses because we could take any value between 0 and 1. 4 4.1 Notational conventions Parametrized models As in the examples above our hypotheses often take the form a certain parameter has value 𝜃. We will often use the letter 𝜃 to stand for an arbitrary hypothesis. This will leave symbols like 𝑝, 𝑓, and 𝑥 to take there usual meanings as pmf, pdf, and data. Also, rather than saying ‘the hypothesis that the parameter of interest has value 𝜃’ we will simply say the hypothesis 𝜃. 4.2 Big and little letters We have two parallel notations for outcomes and probability: 1. (Big letters) Event 𝐴, probability function 𝑃 (𝐴). 2. (Little letters) Value 𝑥, pmf 𝑝(𝑥) or pdf 𝑓(𝑥). These notations are related by 𝑃 (𝑋 = 𝑥) = 𝑝(𝑥), where 𝑥 is a value the discrete random variable 𝑋 and ‘𝑋 = 𝑥’ is the corresponding event. We carry these notations over to the probabilities used in Bayesian updating. 1. (Big letters) From hypotheses ℋ and data 𝒟 we compute several associated probabilities 𝑃 (ℋ), 𝑃 (𝒟), 𝑃 (ℋ|𝒟), 𝑃 (𝒟|ℋ). In the coin example we might have ℋ0.6 = ‘the chosen coin has probability 0.6 of heads’, 𝒟 = ‘3 flips landed HHT’, so 𝑃 (𝒟|ℋ0.6 ) = (0.6)2 (0.4) 2. (Small letters) Hypothesis values 𝜃 and data values 𝑥 both have probabilities or probability densities: 𝑝(𝜃) 𝑓(𝜃) 𝑝(𝑥) 𝑓(𝑥) 𝑝(𝜃|𝑥) 𝑓(𝜃|𝑥) 𝑝(𝑥|𝜃) 𝑓(𝑥|𝜃) In the coin example we might have 𝜃 = 0.6 and 𝑥 is the sequence 1, 1, 0. So, 𝑝(𝑥|𝜃) = (0.6)2 (0.4). We might also write 𝑝(𝑥 = 1, 1, 0|𝜃 = 0.6) to emphasize the values of 𝑥 and 𝜃. Although we will still use both types of notation, from now on we will mostly use the small letter notation involving pmfs and pdfs. Hypotheses will usually be parameters represented by Greek letters (𝜃, 𝜆, 𝜇, 𝜎, … ) while data values will usually be represented by English letters (𝑥, 𝑥𝑖 , 𝑦, … ). 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 3 5 Quick review of pdf and probability Suppose 𝑋 is a random variable with pdf 𝑓(𝑥). Recall 𝑓(𝑥) is a density; its units are probability/(units of 𝑥). 𝑓(𝑥) 𝑓(𝑥) probability 𝑓(𝑥)𝑑𝑥 𝑃 (𝑐 ≤ 𝑋 ≤ 𝑑) 𝑐 𝑑𝑥 𝑥 𝑥 𝑑 𝑥 The probability that the value of 𝑋 is in [𝑐, 𝑑] is given by 𝑑 ∫ 𝑓(𝑥) 𝑑𝑥. 𝑐 The probability that 𝑋 is in an infinitesimal range 𝑑𝑥 around 𝑥 is 𝑓(𝑥) 𝑑𝑥. In fact, the integral formula is just the ‘sum’ of these infinitesimal probabilities. We can visualize these probabilities by viewing the integral as area under the graph of 𝑓(𝑥). In order to manipulate probabilities instead of densities in what follows, we will make frequent use of the notion that 𝑓(𝑥) 𝑑𝑥 is the probability that 𝑋 is in an infinitesimal range around 𝑥 of width 𝑑𝑥. Please make sure that you fully understand this notion. 6 Continuous priors, discrete likelihoods In the Bayesian framework we have probabilities of hypotheses –called prior and posterior probabilities– and probabilities of data given a hypothesis –called likelihoods. In earlier classes both the hypotheses and the data had discrete ranges of values. We saw in the introduction that we might have a continuous range of hypotheses. The same is true for the data, but for today we will assume that our data can only take a discrete set of values. In this case, the likelihood of data 𝑥 given hypothesis 𝜃 is written using a pmf: 𝑝(𝑥|𝜃). We will use the following coin example to explain these notions. We will carry this example through in each of the succeeding sections. Example 4. Suppose we have a bent coin with unknown probability 𝜃 of heads. In this case, we’ll say the coin is of ’type 𝜃’ and we’ll label the hypothesis that a random coin is of type 𝜃 by ℋ𝜃 . The value of 𝜃 is random and could be anywhere between 0 and 1. For this and the examples that follow we’ll suppose that the value of 𝜃 follows a distribution with continuous prior probability density 𝑓(𝜃) = 2𝜃. We have a discrete likelihood because tossing a coin has only two outcomes, 𝑥 = 1 for heads and 𝑥 = 0 for tails. 𝑝(𝑥 = 1|ℋ𝜃 ) = 𝜃, 𝑝(𝑥 = 0|ℋ𝜃 ) = 1 − 𝜃. As we stated earlier, we will often write 𝜃 for the hypothesis ℋ𝜃 . So the above probabilities become 𝑝(𝑥 = 1|𝜃) = 𝜃, 𝑝(𝑥 = 0|𝜃) = 1 − 𝜃. 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 4 Think: This can be tricky to wrap your mind around. We have a continuous range of types of coins –we identify the type by the value of the parameter 𝜃. We are able to choose a coin at random and the type chosen has a probability density 𝑓(𝜃). It may help to see that the discrete examples we did in previous classes are similar. In one example, we had three types of coin with probability of heads 0.5, 0.6, or 0.9. So, we called our hypotheses 𝐻0.5 , 𝐻0.6 , 𝐻0.9 and these had prior probabilities 𝑃 (𝐻0.5 ) etc. In other words, we had a type of coin with an unknown probability of heads, we had hypotheses about that probability and each of these hypotheses had a prior probability. 7 The law of total probability The law of total probability for continuous probability distributions is essentially the same as for discrete distributions. We replace the prior pmf by a prior pdf and the sum by an integral. We start by reviewing the law for the discrete case. Recall that for a discrete set of hypotheses ℋ1 , ℋ2 , … ℋ𝑛 the law of total probability says 𝑛 𝑃 (𝒟) = ∑ 𝑃 (𝒟|ℋ𝑖 )𝑃 (ℋ𝑖 ). (1) 𝑖=1 This is the total prior probability of 𝒟 because we used the prior probabilities 𝑃 (ℋ𝑖 ) In the little letter notation with 𝜃1 , 𝜃2 , … , 𝜃𝑛 for hypotheses and 𝑥 for data the law of total probability is written 𝑛 𝑝(𝑥) = ∑ 𝑝(𝑥|𝜃𝑖 )𝑝(𝜃𝑖 ). (2) 𝑖=1 We also called this the prior predictive probability of the outcome 𝑥 to distinguish it from the prior probability of the hypothesis 𝜃. Likewise, there is a law of total probability for continuous pdfs. We state it as a theorem using little letter notation. Theorem. Law of total probability. Suppose we have a continuous parameter 𝜃 in the range [𝑎, 𝑏], and discrete random data 𝑥. Assume 𝜃 is itself random with density 𝑓(𝜃) and that 𝑥 and 𝜃 have likelihood 𝑝(𝑥|𝜃). In this case, the total probability of 𝑥 is given by the formula. 𝑏 𝑝(𝑥) = ∫ 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 (3) 𝑎 Proof. Our proof will be by analogy to the discrete version: The probability term 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 is perfectly analogous to the term 𝑝(𝑥|𝜃𝑖 )𝑝(𝜃𝑖 ) in Equation 2 (or the term 𝑃 (𝒟|ℋ𝑖 )𝑃 (ℋ𝑖 ) in Equation 1). Continuing the analogy: the sum in Equation 2 becomes the integral in Equation 3 As in the discrete case, when we think of 𝜃 as a hypothesis explaining the probability of the data we call 𝑝(𝑥) the prior predictive probability for 𝑥. Example 5. (Law of total probability.) Continuing with Example 4. We have a bent coin with probability 𝜃 of heads. The value of 𝜃 is random with prior pdf 𝑓(𝜃) = 2𝜃 on [0, 1]. 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 5 Suppose I am about to flip the coin. What is the total probability of heads, i.e what is the prior predictive probability of heads? Solution: In Example 4 we noted that the likelihoods are 𝑝(𝑥 = 1|𝜃) = 𝜃 and 𝑝(𝑥 = 0|𝜃) = 1 − 𝜃. So the total probability of 𝑥 = 1 is 1 1 1 2 𝑝(𝑥 = 1) = ∫ 𝑝(𝑥 = 1|𝜃) 𝑓(𝜃) 𝑑𝜃 = ∫ 𝜃 ⋅ 2𝜃 𝑑𝜃 = ∫ 2𝜃2 𝑑𝜃 = . 3 0 0 0 Since the prior is weighted towards higher probabilities of heads, so is the total probability of heads. 8 Bayes’ theorem for continuous probability densities The statement of Bayes’ theorem for continuous pdfs is essentially identical to the statement for pmfs. We state it including 𝑑𝜃 so we have genuine probabilities: Theorem. Bayes’ Theorem. Use the same assumptions as in the law of total probability, i.e. 𝜃 is a continuous parameter with pdf 𝑓(𝜃) and range [𝑎, 𝑏]; 𝑥 is random discrete data; together they have likelihood 𝑝(𝑥|𝜃). With these assumptions: 𝑓(𝜃|𝑥) 𝑑𝜃 = 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 . = 𝑏 𝑝(𝑥) ∫𝑎 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 (4) Proof. Since this is a statement about probabilities it is just the usual statement of Bayes’ theorem. We hope this is clear. It is important enough to spell out somewhat formally: Let Θ be the random variable that produces the value 𝜃. Consider the events ℋ = ‘Θ is in an interval of width 𝑑𝜃 around the value 𝜃’ and 𝒟 = ‘the value of the data is 𝑥’. Then 𝑃 (ℋ) = 𝑓(𝜃) 𝑑𝜃, 𝑃 (𝒟) = 𝑝(𝑥), and 𝑃 (𝒟|ℋ) = 𝑝(𝑥|𝜃). Now our usual form of Bayes’ theorem becomes 𝑓(𝜃|𝑥) 𝑑𝜃 = 𝑃 (ℋ|𝒟) = 𝑃 (𝒟|ℋ)𝑃 (ℋ) 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 = 𝑃 (𝒟) 𝑝(𝑥) Looking at the first and last terms in this equation we see the new form of Bayes’ theorem. Finally, we firmly believe that it is more conducive to careful thinking about probability to keep the factor of 𝑑𝜃 in the statement of Bayes’ theorem. But because it appears in the numerator on both sides of Equation 4 many people drop the 𝑑𝜃 and write Bayes’ theorem in terms of densities as 𝑓(𝜃|𝑥) = 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑝(𝑥|𝜃)𝑓(𝜃) = 𝑏 . 𝑝(𝑥) ∫𝑎 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 6 9 Bayesian updating with continuous priors Now that we have Bayes’ theorem and the law of total probability we can finally get to Bayesian updating. Before continuing with Example 4, we point out two features of the Bayesian updating table that appears in the next example: 1. The table for continuous priors is very simple: since we cannot have a row for each of an infinite number of hypotheses we’ll have just one row which uses a variable to stand for all hypotheses ℋ𝜃 . 2. By including 𝑑𝜃, all the entries in the table are probabilities and all our usual probability rules apply. Example 6. (Bayesian updating.) Continuing Examples 4 and 5. We have a bent coin with unknown probability 𝜃 of heads. The value of 𝜃 is random with prior pdf 𝑓(𝜃) = 2𝜃. Suppose we flip the coin three times and get the sequence 𝐻𝑇 𝑇 . Compute the posterior pdf for 𝜃. Solution: We make the usual update table, with an added column giving the range of values that 𝜃 can take. We make the first row an abstract version of Bayesian updating and the second row is Bayesian updating for this particular example. In later examples we will skip that abstract version. hypothesis range prior likelihood Bayes numerator posterior ℋ𝜃 𝜃 range 𝑓(𝜃) 𝑑𝜃 𝑝(𝑥 = 1, 1, 0|𝜃) 𝑝(𝑥 = 1, 1, 0|𝜃)𝑓(𝜃) 𝑑𝜃 𝑓(𝜃|𝑥 = 1, 1, 0) 𝑑𝜃 ℋ𝜃 [0, 1] 2𝜃 𝑑𝜃 𝜃2 (1 − 𝜃) 2𝜃3 (1 − 𝜃) 𝑑𝜃 20𝜃3 (1 − 𝜃) 𝑑𝜃 total [0, 1] ∫𝑎 𝑓(𝜃) 𝑑𝜃 = 1 no sum 𝑝(𝑥 = 1, 1, 0) 1 = ∫0 2𝜃3 (1 − 𝜃) 𝑑𝜃 = 1/10 1 𝑏 Therefore the posterior pdf (after seeing HHT) is 𝑓(𝜃|𝑥) = 20𝜃3 (1 − 𝜃) . We have a number of comments: 1. Since we used the prior probability 𝑓(𝜃) 𝑑𝜃, the hypothesis should have been: ’the unknown paremeter is in an interval of width 𝑑𝜃 around 𝜃’. Even for us that is too much to write, so you will have to think it everytime we write that the hypothesis is 𝜃 or ℋ𝜃 . 2. The posterior pdf for 𝜃 is found by removing the 𝑑𝜃 from the posterior probability in the table. 𝑓(𝜃|𝑥) = 20𝜃3 (1 − 𝜃). 3. (i) As always 𝑝(𝑥) is the total probability. Since we have a continuous distribution instead of a sum we compute an integral. (ii) Notice that by including 𝑑𝜃 in the table, it is clear what integral we need to compute to find the total probability 𝑝(𝑥). 4. The table organizes the continuous version of Bayes’ theorem. Namely, the posterior pdf 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 7 is related to the prior pdf and likelihood function via: 𝑝(𝑥|𝜃) 𝑓(𝜃)𝑑𝜃 𝑓(𝜃|𝑥) 𝑑𝜃 = = 𝑏 ∫𝑎 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 𝑝(𝑥|𝜃) 𝑓(𝜃) 𝑑𝜃 𝑝(𝑥) Removing the 𝑑𝜃 in the numerator of both sides we have the statement in terms of densities. 5. Regarding both sides as functions of 𝜃, we can again express Bayes’ theorem in the form: 𝑓(𝜃|𝑥) ∝ 𝑝(𝑥|𝜃) ⋅ 𝑓(𝜃) posterior ∝ likelihood × prior. 9.1 Flat priors One important prior is called a flat or uniform prior. A flat prior assumes that every hypothesis is equally probable. For example, if 𝜃 has range [0, 1] then 𝑓(𝜃) = 1 is a flat prior. Example 7. (Flat priors.) We have a bent coin with unknown probability 𝜃 of heads. Suppose we toss it once and get heads. Assume a flat prior and find the posterior probability for 𝜃. Solution: This is similar Example 6 with a different prior and data. hypothesis 𝜃 range 𝜃 prior 𝑓(𝜃) 𝑑𝜃 likelihood 𝑝(𝑥 = 1|𝜃) Bayes numerator posterior 𝑓(𝜃|𝑥 = 1) 𝑑𝜃 𝜃 [0, 1] 1 ⋅ 𝑑𝜃 𝜃 𝜃 𝑑𝜃 2𝜃 𝑑𝜃 total [0, 1] ∫𝑎 𝑓(𝜃) 𝑑𝜃 = 1 no sum 𝑝(𝑥 = 1) = ∫ 𝜃 𝑑𝜃 = 1/2 9.2 1 𝑏 1 0 Using the posterior pdf Example 8. In the previous example the prior probability was flat. First show that this means that a priori the coin is equally like to be biased towards heads or tails. Then, after observing one heads, what is the (posterior) probability that the coin is biased towards heads? Solution: Since the parameter 𝜃 is the probability the coin lands heads, the first part of the problem asks us to show 𝑃 (𝜃 > 0.5) = 0.5 and the second part asks for 𝑃 (𝜃 > 0.5 | 𝑥 = 1). These are easily computed from the prior and posterior pdfs respectively. The prior probability that the coin is biased towards heads is 1 1 1 1 𝑃 (𝜃 > 0.5) = ∫ 𝑓(𝜃) 𝑑𝜃 = ∫ 1 ⋅ 𝑑𝜃 = 𝜃|0.5 = . 2 0.5 0.5 The probability of 1/2 means the coin is equally likely to be biased toward heads or tails. The posterior probabilitiy that it’s biased towards heads is 1 1 3 1 𝑃 (𝜃 > 0.5|𝑥 = 1) = ∫ 𝑓(𝜃|𝑥 = 1) 𝑑𝜃 = ∫ 2𝜃 𝑑𝜃 = 𝜃2 ∣0.5 = . 4 0.5 0.5 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 8 We see that observing one heads has increased the probability that the coin is biased towards heads from 1/2 to 3/4. 10 Predictive probabilities Just as in the discrete case we are also interested in using the posterior probabilities of the hypotheses to make predictions for what will happen next. Example 9. (Prior and posterior prediction.) Continuing Examples 4, 5, 6: we have a coin with unknown probability 𝜃 of heads and the value of 𝜃 has prior pdf 𝑓(𝜃) = 2𝜃. Find the prior predictive probability of heads. Then suppose the first flip was heads and find the posterior predictive probabilities of both heads and tails on the second flip. Solution: For notation let 𝑥1 be the result of the first flip and let 𝑥2 be the result of the second flip. The prior predictive probability is exactly the total probability computed in Examples 5 and 6. 1 1 2 𝑝(𝑥1 = 1) = ∫ 𝑝(𝑥1 = 1|𝜃)𝑓(𝜃) 𝑑𝜃 = ∫ 2𝜃2 𝑑𝜃 = . 3 0 0 The posterior predictive probabilities are the total probabilities computed using the posterior pdf. From Example 6 we know the posterior pdf is 𝑓(𝜃|𝑥1 = 1) = 3𝜃2 . So the posterior predictive probabilities are 1 1 𝑝(𝑥2 = 1|𝑥1 = 1) = ∫ 𝑝(𝑥2 = 1|𝜃, 𝑥1 = 1)𝑓(𝜃|𝑥1 = 1) 𝑑𝜃 = ∫ 𝜃 ⋅ 3𝜃2 𝑑𝜃 = 3/4 0 0 1 1 𝑝(𝑥2 = 0|𝑥1 = 1) = ∫ 𝑝(𝑥2 = 0|𝜃, 𝑥1 = 1)𝑓(𝜃|𝑥1 = 1) 𝑑𝜃 = ∫ (1 − 𝜃) ⋅ 3𝜃2 𝑑𝜃 = 1/4 0 0 (More simply, we could have computed 𝑝(𝑥2 = 0|𝑥1 = 1) = 1 − 𝑝(𝑥2 = 1|𝑥1 = 1) = 1/4.) 11 (Optional) From discrete to continuous Bayesian updating This section is optional. In it we will try to develop intuition for the transition from discrete to continuous Bayesian updating. We’ll walk a familiar road from calculus. Namely we will: (i) divide the continuous range of hypotheses into a finite number of short intervals. (ii) create the discrete updating table for the finite number of hypotheses. (iii) consider how the table changes as the number of hypotheses goes to infinity. In this way, we’ll see the prior and posterior pmfs converge to the prior and posterior pdfs. Example 10. To keep things concrete, we will work with the same prior and data as in Example 7. We have a ‘bent’ coin with a flat prior 𝑓(𝜃) = 1. Our data is we tossed the coin once and got heads. Our goal is to go from discrete to continuous by increasing the number of hypotheses. 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 9 4 hypotheses. Suppose we have four types of coins that have probability of heads 1/8, 3/8, 5/8 and 7/8 respectively. If one coin is chosen at random, our hypotheses for its type are ℋ1 ∶ 𝜃 = 1/8, ℋ2 ∶ 𝜃 = 3/8, ℋ3 ∶ 𝜃 = 5/8, ℋ4 ∶ 𝜃 = 7/8. To get this, we divided [0, 1] into 4 equal intervals: [0, 1/4], [1/4, 1/2], [1/2, 3/4], [3/4, 1]. Each interval has width Δ𝜃 = 1/4. We put our the value of 𝜃 for our coin types at the centers of the four intervals. (Just as with forming Riemann sums in calculus, it’s not important where in each interval we choose 𝜃. The center is one easy choice.) Let’s name each of these values 𝜃𝑗 = 𝑗/8, where 𝑗 = 1, 3, 5, 7. The flat prior gives each hypothesis a probability of 1/4 = 1 ⋅ Δ𝜃. We have the table: hypothesis prior likelihood Bayes num. posterior 𝜃 = 𝜃1 = 1 8 1 4 1 8 1 4 ⋅ 1 8 0.0625 𝜃 = 𝜃2 = 3 8 1 4 3 8 1 4 ⋅ 3 8 0.1875 𝜃 = 𝜃3 = 5 8 1 4 5 8 1 4 ⋅ 5 8 0.3125 𝜃 = 𝜃4 = 7 8 1 4 7 8 1 4 ⋅ 7 8 0.4375 𝑛 – 1 Total ∑ 𝜃𝑖 Δ𝜃 1 𝑖=1 Here are the density histograms of the prior and posterior pmf. The prior and posterior pdfs from Example 7 are superimposed on the histograms in red. 2 density 2 1.5 1.5 1 1 .5 .5 density 𝑥 1/8 3/8 5/8 7/8 𝑥 1/8 3/8 5/8 7/8 8 hypotheses. Next we slice [0,1] into 8 intervals each of width Δ𝜃 = 1/8 and use the center of each slice for our 8 hypotheses 𝜃𝑖 . 𝜃1 : 𝜃5 : ’𝜃 = 1/16’, ’𝜃 = 9/16’, 𝜃2 : 𝜃6 : ’𝜃 = 3/16’, ’𝜃 = 11/16’, 𝜃3 : 𝜃7 : ’𝜃 = 5/16’, ’𝜃 = 13/16’, 𝜃4 : 𝜃8 : ’𝜃 = 7/16’ ’𝜃 = 15/16’ The flat prior gives each hypothesis the probablility 1/8 = 1 ⋅ Δ𝜃. Here are the table and density histograms. 18.05 Class 13, Bayesian Updating with Continuous Priors, Spring 2022 hypothesis 𝜃 = 𝜃1 = 𝜃 = 𝜃2 = 𝜃 = 𝜃3 = 𝜃 = 𝜃4 = 𝜃 = 𝜃5 = 𝜃 = 𝜃6 = 𝜃 = 𝜃7 = 𝜃 = 𝜃8 = 1 16 3 16 5 16 7 16 9 16 11 16 13 16 15 16 Total prior likelihood 1 8 1 8 1 8 1 8 1 8 1 8 1 8 1 8 1 16 3 16 5 16 7 16 9 16 11 16 13 16 15 16 1 – Bayes num. 1 8 1 8 1 8 1 8 1 8 1 8 1 8 1 8 𝑛 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 10 posterior 1 16 3 16 5 16 7 16 9 16 11 16 13 16 15 16 0.0156 0.0469 0.0781 0.1094 0.1406 0.1719 0.2031 0.2344 ∑ 𝜃𝑖 Δ𝜃 1 𝑖=1 2 density 2 1.5 1.5 1 1 .5 .5 density 𝑥 1/16 3/16 5/16 7/16 9/16 11/16 13/16 𝑥 15/16 1/16 3/16 5/16 7/16 9/16 11/16 13/16 15/16 20 hypotheses. Finally we slice [0,1] into 20 pieces. This is essentially identical to the previous two cases. Let’s skip right to the density histograms. 2 density 2 1.5 1.5 1 1 .5 .5 density 𝑥 3/7 𝑥 3/7 Looking at the sequence of plots we see how the prior and posterior density histograms converge to the prior and posterior probability density functions.