University:
Massachusetts Institute of Technology
Course:
18.05 | Introduction to Probability and Statistics
Academic year:

2022
Views:

222

Pages:

2
Author:

Mark Stone

Notational conventions Class 13, 18.05 1 Learning Goals 1. Be able to work with the various notations and terms we use to describe probabilities and likelihood. 2 Introduction We’ve introduced a number of different notations for probability, hypotheses and data. We collect them here, to have them in one place. 3 Notation and terminology for data and hypotheses The problem of labeling data and hypotheses is a tricky one. When we started the course we talked about outcomes, e.g. heads or tails. Then when we introduced random variables we gave outcomes numerical values, e.g. 1 for heads and 0 for tails. This allowed us to do things like compute means and variances. We need to do something similar now. Recall our notational conventions: • Events are labeled with capital letters, e.g. 𝐴, 𝐵, 𝐶. • A random variable is capital 𝑋 and takes values small 𝑥. • The connection between values and events: ‘𝑋 = 𝑥’ is the event that 𝑋 takes the value 𝑥. • The probability of an event is capital 𝑃 (𝐴). • A discrete random variable has a probability mass function small 𝑝(𝑥) The connection between 𝑃 and 𝑝 is that 𝑃 (𝑋 = 𝑥) = 𝑝(𝑥). • A continuous random variable has a probability density function 𝑓(𝑥) The connection 𝑏 between 𝑃 and 𝑓 is that 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) = ∫𝑎 𝑓(𝑥) 𝑑𝑥. • For a continuous random variable 𝑋 the probability that 𝑋 is in an infinitesimal interval of width 𝑑𝑥 around 𝑥 is 𝑓(𝑥) 𝑑𝑥. In the context of Bayesian updating we have similar conventions. • We use capital letters, especially ℋ, to indicate a hypothesis, e.g. ℋ = ’the coin is fair’. 1 18.05 Class 13, Notational conventions, Spring 2022 2 • We use lower case letters, especially 𝜃, to indicate the hypothesized value of a model parameter, e.g. the probability the coin lands heads is 𝜃 = 0.5. • We use upper case letters, especially 𝒟, when talking about data as events. For example, 𝒟 = ‘the sequence of tosses was HTH. • We use lower case letters, especially 𝑥, when talking about data as values. For example, the sequence of data was 𝑥1 , 𝑥2 , 𝑥3 = 1, 0, 1. • When the set of hypotheses is discrete we can use the probability of individual hypotheses, e.g. 𝑝(𝜃). When the set is continuous we need to use the probability for an infinitesimal range of hypotheses, e.g. 𝑓(𝜃) 𝑑𝜃. The following table summarizes this for discrete 𝜃 and continuous 𝜃. In both cases we are assuming a discrete set of possible outcomes (data) 𝑥. Tomorrow we will deal with a continuous set of outcomes. Bayes likelihood numerator posterior hypothesis prior ℋ 𝑃 (ℋ) 𝑃 (𝒟|ℋ) 𝑃 (𝒟|ℋ)𝑃 (ℋ) 𝑃 (ℋ|𝒟) 𝑝(𝑥|𝜃) 𝑝(𝑥|𝜃)𝑝(𝜃) 𝑝(𝜃|𝑥) Discrete 𝜃: 𝜃 𝑝(𝜃) Continuous 𝜃: 𝜃 𝑓(𝜃) 𝑑𝜃 𝑝(𝑥|𝜃) 𝑝(𝑥|𝜃)𝑓(𝜃) 𝑑𝜃 𝑓(𝜃|𝑥) 𝑑𝜃 Remember the continuous hypothesis 𝜃 is really a shorthand for ‘the parameter 𝜃 is in an interval of width 𝑑𝜃 around 𝜃’.