18.05 S22 Class 15: Conjugate Priors: Beta and Normal

Conjugate priors: Beta and normal Class 15, 18.05 1 Learning Goals 1. Be familiar with the 2-parameter family of beta distributions and its normalization. 2. Understand the benefits of conjugate priors. 3. Be able to update a beta prior given a Bernoulli, binomial, or geometric likelihood. 4. Understand and be able to use the formula for updating a normal prior given a normal likelihood with known variance. 2 Introduction Our main goal here is to introduce the idea of conjugate priors and look at some specific conjugate pairs. These simplify the job of Bayesian updating to simple arithmetic. We’ll start by introducing the beta distribution and using it as a conjugate prior with a binomial likelihood. After that we’ll look at other conjugate pairs. 3 Beta distribution The beta distribution Beta(𝑎, 𝑏) is a two-parameter distribution with range [0, 1] and pdf 𝑓(𝜃) = (𝑎 + 𝑏 − 1)! 𝑎−1 𝜃 (1 − 𝜃)𝑏−1 (𝑎 − 1)!(𝑏 − 1)! We have made an applet so you can explore the shape of the beta distribution as you vary the parameters: https://mathlets.org/mathlets/beta-distribution/. As you can see in the applet, the beta distribution may be defined for any real numbers 𝑎 > 0 and 𝑏 > 0. In 18.05 we will stick to integers 𝑎 and 𝑏, but you can get the full story here: https://en.wikipedia.org/wiki/Beta_distribution In the context of Bayesian updating, 𝑎 and 𝑏 are often called hyperparameters to distinguish them from the unknown parameter 𝜃 representing our hypotheses. In a sense, 𝑎 and 𝑏 are ‘one level up’ from 𝜃 since they parameterize its pdf. 3.1 A simple but important observation! If a pdf 𝑓(𝜃) with range [0, 1] has the form 𝑐𝜃𝑎−1 (1 − 𝜃)𝑏−1 then 𝑓(𝜃) is a Beta(𝑎, 𝑏) distribution and the normalizing constant must be 𝑐= (𝑎 + 𝑏 − 1)! . (𝑎 − 1)! (𝑏 − 1)! 1 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 2 This follows because the constant 𝑐 must normalize the pdf to have total probability 1. There is only one such constant and it is given in the formula for the beta distribution. A similar observation holds for normal distributions, exponential distributions, and so on. 3.2 Beta priors and posteriors for binomial random variables Example 1. Suppose we have a bent coin with unknown probability 𝜃 of heads. We toss it 12 times and get 8 heads and 4 tails. Starting with a flat prior, show that the posterior pdf is a Beta(9, 5) distribution. Solution: This is nearly identical to examples from the previous class. We’ll call the data from all 12 tosses 𝑥1 . In the following table we call the leading constant factor in the posterior column 𝑐2 . Our simple observation will tell us that it has to be the constant factor from the beta pdf. The data is 8 heads and 4 tails. Since this comes from a binomial(12, 𝜃) distribution, the 12 likelihood 𝑝(𝑥1 |𝜃) = ( )𝜃8 (1 − 𝜃)4 . Thus the Bayesian update table is 8 hypothesis prior 𝜃 1 ⋅ 𝑑𝜃 total 1 Bayes numerator likelihood 8 (12 8 ) 𝜃 (1 − 𝜃)4 8 4 (12 8 ) 𝜃 (1 − 𝜃) 1 12 8 𝑇 =( 8 posterior 𝑑𝜃 𝑐2 𝜃8 (1 − 𝜃)4 𝑑𝜃 ) ∫ 𝜃 (1 − 𝜃)4 𝑑𝜃 1 0 For the posterior pdf, our simple observation holds with 𝑎 = 9 and 𝑏 = 5. Therefore the posterior pdf follows a Beta(9, 5) distribution and we have 𝑓(𝜃|𝑥1 ) = 𝑐2 𝜃8 (1 − 𝜃)4 , where 𝑐2 = 13! . 8! 4! Note: We explicitly included the binomial coefficient (12 8 ) in the likelihood. We could just as easily have given it a name, say 𝑐1 and not bothered making its value explicit. Example 2. Now suppose we toss the same coin again, getting 𝑛 heads and 𝑚 tails. Using the posterior pdf of the previous example as our new prior pdf, show that the new posterior pdf is that of a Beta(9 + 𝑛, 5 + 𝑚) distribution. Solution: It’s all in the table. We’ll call the data of these 𝑛 + 𝑚 additional tosses 𝑥2 . This time we won’t make the binomial coefficient explicit. Instead we’ll just call it 𝑐3 . Whenever we need a new label we will simply use 𝑐 with a new subscript. hyp. prior likelihood Bayes numerator posterior 𝜃 𝑐2 𝜃8 (1 − 𝜃)4 𝑑𝜃 𝑐3 𝜃𝑛 (1 − 𝜃)𝑚 𝑐2 𝑐3 𝜃𝑛+8 (1 − 𝜃)𝑚+4 𝑑𝜃 𝑐4 𝜃𝑛+8 (1 − 𝜃)𝑚+4 𝑑𝜃 1 total 1 𝑇 = ∫ 𝑐2 𝑐3 𝜃𝑛+8 (1 − 𝜃)𝑚+4 𝑑𝜃 0 Again our simple observation holds and therefore the posterior pdf 𝑓(𝜃|𝑥1 , 𝑥2 ) = 𝑐4 𝜃𝑛+8 (1 − 𝜃)𝑚+4 1 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 3 follows a Beta(𝑛 + 9, 𝑚 + 5) distribution. Note: Flat beta. The Beta(1, 1) distribution is the same as the uniform distribution on [0, 1], which we have also called the flat prior on 𝜃. This follows by plugging 𝑎 = 1 and 𝑏 = 1 into the definition of the beta distribution, giving 𝑓(𝜃) = 1. Summary: If the probability of heads is 𝜃, the number of heads in 𝑛 + 𝑚 tosses follows a binomial(𝑛 + 𝑚, 𝜃) distribution. We have seen that if the prior on 𝜃 is a beta distribution then so is the posterior; only the parameters 𝑎, 𝑏 of the beta distribution change! We summarize precisely how they change in a table. We assume the data is 𝑛 heads and 𝑚 tails in 𝑛 + 𝑚 tosses. hypothesis data prior likelihood posterior 𝜃 𝑥 = 𝑛, 𝑚 Beta(𝑎, 𝑏) binomial(𝑛 + 𝑚, 𝜃) Beta(𝑎 + 𝑛, 𝑏 + 𝑚) 𝜃 𝑥 = 𝑛, 𝑚 𝑐1 𝜃𝑎−1 (1 − 𝜃)𝑏−1 𝑑𝜃 𝑐2 𝜃𝑛 (1 − 𝜃)𝑚 𝑐3 𝜃𝑎+𝑛−1 (1 − 𝜃)𝑏+𝑚−1 𝑑𝜃 4 Conjugate priors The beta distribution is called a conjugate prior for the binomial distribution. This means that if the likelihood function is binomial, then a beta prior gives a beta posterior –this is what we saw in the previous examples. In fact, the beta distribution is a conjugate prior for the Bernoulli and geometric distributions as well. We will soon see another important example: the normal distribution is its own conjugate prior. In particular, if the likelihood function is normal with known variance, then a normal prior gives a normal posterior. Conjugate priors are useful because they reduce Bayesian updating to modifying the parameters of the prior distribution (so-called hyperparameters) rather than computing integrals. We saw this for the beta distribution in the last table. For many more examples see: https://en.wikipedia.org/wiki/Conjugate_prior_distribution We now give a definition of conjugate prior. It is best understood through the examples in the subsequent sections. Definition. Suppose we have data with likelihood function 𝜙(𝑥|𝜃) depending on a hypothesized parameter 𝜃. Also suppose the prior distribution for 𝜃 is one of a family of parametrized distributions. If the posterior distribution for 𝜃 is in this family then we say the the family of priors are conjugate priors for the likelihood. This definition will be illustrated with specific examples in the sections below. 5 Beta priors In this section, we will show that the beta distribution is a conjugate prior for binomial, Bernoulli, and geometric likelihoods. 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 5.1 4 Binomial likelihood We saw above that the beta distribution is a conjugate prior for the binomial distribution. This means that if the likelihood function is binomial and the prior distribution is beta then the posterior is also beta. More specifically, suppose that the likelihood follows a binomial(𝑁 , 𝜃) distribution where 𝑁 is known and 𝜃 is the (unknown) parameter of interest. We also have that the data 𝑥 from one trial is an integer between 0 and 𝑁 . Then for a beta prior we have the following table: hypoth. data 𝜃 𝑥 prior Beta(𝑎, 𝑏) 𝑓(𝜃) = 𝑐1 𝜃𝑎−1 (1 − 𝜃)𝑏−1 likelihood binomial(𝑁 , 𝜃) 𝑝(𝑥|𝜃) = 𝑐2 𝜃𝑥 (1 − 𝜃)𝑁−𝑥 posterior Beta(𝑎 + 𝑥, 𝑏 + 𝑁 − 𝑥) 𝑓(𝜃|𝑥) = 𝑐3 𝜃𝑎+𝑥−1 (1 − 𝜃)𝑏+𝑁−𝑥−1 The table is simplified by writing the normalizing coefficients as 𝑐1 , 𝑐2 and 𝑐3 respectively. If needed, we can recover the values of the 𝑐1 and 𝑐2 by recalling (or looking up) the normalizations of the beta and binomial distributions. 𝑐1 = 5.2 (𝑎 + 𝑏 − 1)! (𝑎 − 1)! (𝑏 − 1)! 𝑁! 𝑁 𝑐2 = ( ) = 𝑥 𝑥! (𝑁 − 𝑥)! 𝑐3 = (𝑎 + 𝑏 + 𝑁 − 1)! (𝑎 + 𝑥 − 1)! (𝑏 + 𝑁 − 𝑥 − 1)! Bernoulli likelihood The beta distribution is a conjugate prior for the Bernoulli distribution. This is actually a special case of the binomial distribution, since Bernoulli(𝜃) is the same as binomial(1, 𝜃). We do it separately because it is slightly simpler and of special importance. In the table below, we show the updates corresponding to success (𝑥 = 1) and failure (𝑥 = 0) on separate rows. hypothesis 𝜃 𝜃 𝜃 data 𝑥 𝑥=1 𝑥=0 prior Beta(𝑎, 𝑏) 𝑓(𝜃) = 𝑐1 𝜃𝑎−1 (1 − 𝜃)𝑏−1 𝑓(𝜃) = 𝑐1 𝜃𝑎−1 (1 − 𝜃)𝑏−1 likelihood Bernoulli(𝜃) 𝑝(𝑥|𝜃) = 𝜃 𝑝(𝑥|𝜃) = 1 − 𝜃 posterior Beta(𝑎 + 1, 𝑏) or Beta(𝑎, 𝑏 + 1) Beta(𝑎 + 1, 𝑏): 𝑓(𝜃|𝑥) = 𝑐3 𝜃𝑎 (1 − 𝜃)𝑏−1 Beta(𝑎, 𝑏 + 1): 𝑓(𝜃|𝑥) = 𝑐4 𝜃𝑎−1 (1 − 𝜃)𝑏 The constants 𝑐1 , 𝑐3 and 𝑐4 have the same formulas as in the previous (binomial likelihood case) with 𝑁 = 1. 5.3 Geometric likelihood Recall that the geometric(𝜃) distribution describes the probability of 𝑥 successes before the first failure, where the probability of success on any single independent trial is 𝜃. The corresponding pmf is given by 𝑝(𝑥) = 𝜃𝑥 (1 − 𝜃). Now suppose that we have a data point 𝑥, and our hypothesis 𝜃 is that 𝑥 is drawn from a geometric(𝜃) distribution. From the table we see that the beta distribution is a conjugate prior for a geometric likelihood as well: hypothesis data 𝜃 𝑥 prior Beta(𝑎, 𝑏) = 𝑓(𝜃) = 𝑐1 𝜃𝑎−1 (1 − 𝜃)𝑏−1 likelihood geometric(𝜃) = 𝑝(𝑥|𝜃) = 𝜃𝑥 (1 − 𝜃) posterior Beta(𝑎 + 𝑥, 𝑏 + 1) 𝑓(𝜃|𝑥) = 𝑐3 𝜃𝑎+𝑥−1 (1 − 𝜃)𝑏 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 5 At first it may seem strange that the beta distribution is a conjugate prior for both the binomial and geometric distributions. The key reason is that the geometric likelihood is proportional to a binomial likelihood as a function of 𝜃. Let’s illustrate this in a concrete example. Example 3. While traveling through the Mushroom Kingdom, Mario and Luigi find some rather unusual coins. They agree on a prior of 𝑓(𝜃) ∼ Beta(5,5) for the probability of heads, though they disagree on what experiment to run to investigate 𝜃 further. (a) Mario decides to flip a coin 5 times. He gets four heads in five flips. (b) Luigi decides to flip a coin until the first tails. He gets four heads before the first tail. Show that Mario and Luigi will arrive at the same posterior on 𝜃, and calculate this posterior. Solution: We will show that both Mario and Luigi find the posterior pdf for 𝜃 is a Beta(9, 6) distribution. Mario’s table hypothesis 𝜃 data 𝑥=4 prior Beta(5, 5) = 𝑐1 𝜃4 (1 − 𝜃)4 likelihood binomial(5, 𝜃) = (54)𝜃4 (1 − 𝜃) posterior ??? = 𝑐3 𝜃8 (1 − 𝜃)5 Luigi’s table hypothesis data 𝜃 𝑥=4 prior Beta(5, 5) = 𝑐1 𝜃4 (1 − 𝜃)4 likelihood geometric(𝜃) = 𝜃4 (1 − 𝜃) posterior ??? 8 = 𝑐3 𝜃 (1 − 𝜃)5 Since both Mario and Luigi’s posteriors have the form of a Beta(9, 6) distribution that’s what they both must be. The normalizing factor must be the same in both cases because it’s determined by requiring the total probability to be 1. 6 Bayesian updating with continuous hypotheses and continuous data The idea here is essentially identical to the Bayesian updating we’ve already done. The only change is, with a continuous likelihood, we have to compute the total probability of the data (i.e. sum of the Bayes numerator column, i.e. normalizing factor) as an integral instead of a sum. We will cover this briefly. For those who are interested, a bit more detail is given in an optional note. Notation • Hypotheses 𝜃. For continuous hypotheses, this really means that we hypothesize that the parameter is in a small interval of size 𝑑𝜃 around 𝜃. • Data 𝑥. For continuous data, this really means that the data is in a small interval of size 𝑑𝑥 around 𝑥. • Prior 𝑓(𝜃)𝑑𝜃. This is our initial belief about the probability that the parameter is in a small interval of size 𝑑𝜃 around 𝜃. 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 6 • Likelihood 𝜙(𝑥 | 𝜃). So the probability that the data is in a small interval of size 𝑑𝑥 around 𝑥, ASSUMING the hypothesis 𝜃 is 𝜙(𝑥 | 𝜃) 𝑑𝑥 • Posterior 𝑓(𝜃 | 𝑥) 𝑑𝜃. This is the (calculated) probability that the parameter is in a small interval of size 𝑑𝜃 around 𝜃, GIVEN the data 𝑥. Bayes hypoth. prior likelihood numerator 𝜃 𝑓(𝜃) 𝑑𝜃 𝜙(𝑥 | 𝜃) 𝜙(𝑥 | 𝜃)𝑓(𝜃) 𝑑𝜃 total 1 no sum 𝜙(𝑥) = ∫ 𝜙(𝑥 | 𝜃)𝑓(𝜃) 𝑑𝜃 (integrate over 𝜃) posterior 𝑓(𝜃 | 𝑥) 𝑑𝜃 = 𝜙(𝑥 | 𝜃)𝑓(𝜃) 𝑑𝜃 𝜙(𝑥) 1 = prior prob. density for data 𝑥 Continuous-continuous Bayesian update table To summarize: the prior probabilities of hypotheses and the likelihoods of data given hypothesis were given; the Bayes numerator is the product of the prior and likelihood; the total likelihood 𝜙(𝑥) is the integral of the probabilities in the Bayes numerator column; we divide by 𝜙(𝑥) to normalize the Bayes numerator. 7 Normal begets normal We now turn to an important example of coninuous-continuous updating: the normal distribution is its own conjugate prior. In particular, if the likelihood function is normal with known variance, then a normal prior gives a normal posterior. Now both the hypotheses and the data are continuous. Suppose we have a measurement 𝑥 ∼ 𝑁 (𝜃, 𝜎2 ) where the variance 𝜎2 is known. That is, the mean 𝜃 is our unknown parameter of interest and we are given that the likelihood comes from a normal distribution with variance 𝜎2 . If we choose a normal prior pdf 2 ) 𝑓(𝜃) ∼ N(𝜇prior , 𝜎prior 2 then the posterior pdf is also normal: 𝑓(𝜃|𝑥) ∼ N(𝜇post , 𝜎post ) where 𝜇post 𝜇prior 𝑥 = 2 + 2, 2 𝜎 𝜎post 𝜎prior 1 2 𝜎post = 1 2 𝜎prior + 1 𝜎2 (1) The following form of these formulas is easier to read and shows that 𝜇post is a weighted average between 𝜇prior and the data 𝑥. 𝑎= 1 2 𝜎prior 𝑏= 1 , 𝜎2 𝜇post = 𝑎𝜇prior + 𝑏𝑥 , 𝑎+𝑏 2 𝜎post = With these formulas in mind, we can express the update via the table: 1 . 𝑎+𝑏 (2) 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 hypothesis data 𝜃 𝑥 prior 2 𝑓(𝜃) ∼ N(𝜇prior , 𝜎prior ) = 𝑐1 exp ( 7 likelihood 𝜙(𝑥|𝜃) ∼ N(𝜃, 𝜎2 ) −(𝜃−𝜇prior )2 ) 2 2𝜎prior posterior 2 ) 𝑓(𝜃|𝑥) ∼ N(𝜇post , 𝜎post 2 = 𝑐2 exp ( −(𝑥−𝜃) 2𝜎2 ) = 𝑐3 exp ( −(𝜃−𝜇post )2 ) 2 2𝜎post We leave the proof of the general formulas to the problem set. It is an involved algebraic manipulation which is essentially the same as the following numerical example. Example 4. Suppose we have prior 𝜃 ∼ N(4, 8), and likelihood function likelihood 𝑥 ∼ N(𝜃, 5). Suppose also that we have one measurement 𝑥1 = 3. Show the posterior distribution is normal. Solution: We will show this by grinding through the algebra which involves completing the square. prior: 𝑓(𝜃) = 𝑐1 e−(𝜃−4) 2 /16 likelihood: 𝜙(𝑥1 |𝜃) = 𝑐2 e−(𝑥1 −𝜃) ; 2 /10 = 𝑐2 e−(3−𝜃) We multiply the prior and likelihood to get the posterior: 𝑓(𝜃|𝑥1 ) = 𝑐3 e−(𝜃−4) 2 = 𝑐3 exp (− /16 −(3−𝜃)2 /10 e (𝜃 − 4)2 (3 − 𝜃)2 ) − 16 10 We complete the square in the exponent − (𝜃 − 4)2 (3 − 𝜃)2 5(𝜃 − 4)2 + 8(3 − 𝜃)2 − =− 16 10 80 13𝜃2 − 88𝜃 + 152 =− 80 152 𝜃2 − 88 𝜃 13 + 13 =− 80/13 (𝜃 − 44/13)2 + 152/13 − (44/13)2 =− . 80/13 Therefore the posterior is 𝑓(𝜃|𝑥1 ) = 𝑐3 e− (𝜃−44/13)2 +152/13−(44/13)2 80/13 This has the form of the pdf for N(44/13, 40/13). = 𝑐4 e− (𝜃−44/13)2 80/13 . QED For practice we check this against the formulas (2). 𝜇prior = 4, 2 𝜎prior = 8, 1 𝜎2 = 5 ⇒ 𝑎 = , 8 Therefore 𝑎𝜇prior + 𝑏𝑥 44 = = 3.38 𝑎+𝑏 13 1 40 = = = 3.08. 𝑎+𝑏 13 𝜇post = 2 𝜎post 1 𝑏= . 5 2 /10 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 7.1 8 A word on weighted averages The updating formula 2 gives 𝜇post as a weighted average of the 𝜇prior and the data. The weight on 𝜇prior is 𝑎/(𝑎 + 𝑏), and the weight on the data is 𝑏/(𝑎 + 𝑏). These weights are always positive numbers summing to 1. If 𝑏 is very large (that is, if the data has a tiny 2 variance) then most of the weight is on the data. If 𝑎 is very large (that is, 𝜎prior is small, i.e. if you are very confident in your prior) then most of the weight is on the prior. In the above example the variance on the prior was bigger than the variance on the data, so 𝑎 was smaller than 𝑏; so the weight was mostly on the data. The posterior 3.38 for the mean was closer to the data 3 than to the prior 4 for the mean. 7.2 Examples of normal-normal updating Example 5. Suppose that we know the data 𝑥 ∼ N(𝜃, 4/9) and we have prior N(0, 1). We get one data value 𝑥 = 6.5. Describe the changes to the pdf for 𝜃 in updating from the prior to the posterior. 2 = 1, 𝜎2 = 4/9. So, using the updating formulas 2 we have Solution: 𝜇prior = 0, 𝜎prior 𝑎 = 1, 𝑏= 1 9 = , 4/9 4 𝜇post = 𝑎𝜇prior + 𝑏𝑥 = 4.5, 𝑎+𝑏 2 𝜎post = 1 4 = . 𝑎+𝑏 13 0.0 0.2 0.4 0.6 Here is a graph of the prior and posterior pdfs with the data point marked by a red line. −2 0 2 4 6 8 10 Prior in blue, posterior in orange, data = red line We see that the posterior mean is closer to the data point than the prior mean We also see that the posterior distribution is taller and narrower than the prior, i.e. it has a smaller variance. The smaller variance says that we are now more certain about where the value of 𝜃 lies. Example 6. Use the formulas 2 to show that for normal-normal Bayesian updating we have: 1. The posterior mean is always between the data point and the prior mean. 2. The posterior variance is smaller than both the prior variance and 𝜎. That is, our 18.05 Class 15, Conjugate priors: Beta and normal, Spring 2022 9 posterior uncetainty is smaller than both our prior uncertainty and the uncertainty in the data. Solution: Using the update formulas 2, we have The posterior mean is the weighted average of the prior mean and the data, so it must lie between the prior mean and the data. Also, the posterior variance is 1 1 2 < = 𝜎prior 𝑎+𝑏 𝑎 That is the posterior has smaller variance than the prior, i.e. data makes us more certain about where in its range 𝜃 lies. 1 1 2 < = 𝜎2 . So, the posterior variance is smaller than 𝜎2 . Likewise 𝜎post = 𝑎+𝑏 𝑏 2 = 𝜎post 7.3 More than one data point Example 7. Suppose we have data 𝑥1 , 𝑥2 , 𝑥3 . Use the formulas (1) to update sequentially. Solution: Let’s label the prior mean and variance as 𝜇0 and 𝜎02 . The updated means and variances will be 𝜇𝑖 and 𝜎𝑖2 . In sequence we have 1 1 1 = 2 + 2; 2 𝜎 𝜎1 𝜎0 1 1 1 1 2 = 2 + 2 = 2 + 2; 𝜎 𝜎 𝜎22 𝜎1 𝜎0 1 1 1 1 3 = 2 + 2 = 2 + 2; 2 𝜎 𝜎 𝜎3 𝜎2 𝜎0 𝜇1 𝜇 𝑥 = 20 + 12 2 𝜎 𝜎1 𝜎0 𝜇2 𝜇 𝑥 𝜇 𝑥 +𝑥 = 21 + 22 = 20 + 1 2 2 𝜎 𝜎 𝜎22 𝜎1 𝜎0 𝜇3 𝜇 𝜇 𝑥 𝑥 + 𝑥2 + 𝑥3 = 22 + 32 = 20 + 1 2 𝜎 𝜎2 𝜎3 𝜎2 𝜎0 The example generalizes to 𝑛 data values 𝑥1 , … , 𝑥𝑛 : Normal-normal update formulas for 𝑛 data points 𝜇post 𝜇prior 𝑛𝑥̄ = 2 + 2, 2 𝜎 𝜎post 𝜎prior 1 2 𝜎post = 1 2 𝜎prior + 𝑛 , 𝜎2 𝑥̄ = 𝑥1 + … + 𝑥 𝑛 . 𝑛 (3) Again we give the easier to read form, showing 𝜇post is a weighted average of 𝜇prior and the sample average 𝑥:̄ 𝑎= 1 2 𝜎prior 𝑏= 𝑛 , 𝜎2 𝜇post = 𝑎𝜇prior + 𝑏𝑥̄ , 𝑎+𝑏 2 𝜎post = 1 . 𝑎+𝑏 (4) Interpretation: 𝜇post is a weighted average of 𝜇prior and 𝑥.̄ If the number of data points is 2 large then the weight 𝑏 is large and 𝑥̄ will have a strong influence on the posterior. If 𝜎prior is small then the weight 𝑎 is large and 𝜇prior will have a strong influence on the posterior. To summarize: 1. Lots of data has a big influence on the posterior. 2. High certainty (low variance) in the prior has a big influence on the posterior. The actual posterior is a balance of these two influences.