The Normal Distribution Notes

1 The Normal Distribution The normal distribution is the most widely used distribution in statistics. It is really important for this course because we will rely heavily on it for many topics. • A normal random variable is a continuous r.v. that can take on any value. We say that X has a normal distribution with expected value µ and variance σ 2 , written as X ∼ N (µ, σ 2 ). The normal pdf is symmetric about it mean, µ. • A special case of the normal distribution occurs when the mean is 0 and the variance is 1. This is called the standard normal distribution. ◦ If the r.v. Z has N (0, 1) distribution, then we say Z is a standard normal random variable. ◦ Notation: The letter Z is usually used to denote a standard normal r.v. ◦ The pdf of the standard normal r.v. is denoted φ(z). ◦ The cdf of the normal distribution is denoted Φ(z). It is obtained as the area under φ, to the left of z. Further, Φ(z) = P (Z ≤ z). ◦ The values of the probabilities Φ(z) of a standard normal r.v. are tabulated in the back of your textbook (see Appendix Table 1 ). It is very important to understand how this table works, since we will be using it over the course of the semester. • Important property: Any normal r.v. X can be transformed into a standard normal r.v. ◦ Mathematically, if X ∼ N (µ, σ 2 ), then (X − µ)/σ ∼ N (0, 1). ◦ The term (X − µ)/σ is also sometimes called the Z-statistic. ◦ The above property means that to turn X into a normal r.v., we need to subtract its mean µ and divide by the sd σ. ◦ This property is important because it allows us to find the probabilities of any normal r.v., by first transforming it to a standard normal and then using the Appendix Table 1 at the back of the textbook. • Important formulas to remember when working with a normal random variable Z: 1. P (Z > z) = 1 − P (Z ≤ z) = 1 − Φ(z) 1 2. Φ(−z) = P (Z < −z) = P (Z > z). This holds because of symmetry of the normal normal distribution. 3. P (|Z| > z) = P (Z > z) + P (Z < −z) = 2P (Z > z) = 2P (Z < −z) = 2Φ(−z) Again, this holds because of symmetry of the normal distribution. 4. For any constants a and b with a < b, P (a ≤ Z ≤ b) = Φ(b) − Φ(a) 2 Sample Average (a.k.a. Sample Mean) • We denote the sample mean as X. It is the average of n individual measurements from a population, Pn where n is our sample size. Mathematically, we state this formally as X = X1 +X2n+...+Xn = n1 i=1 Xi . • If the Xi ’s (that is, X1 , . . . , Xn ) are independent and identically distributed (i.i.d.), this means the following. ◦ Independent: Each of the Xi ’s are independent from each other. For example, X1 and X5 are independent, X2 and X3 are independent, etc. As a consequence of independence, the covariance of any pair of Xi ’s is zero. For example, cov(X1 , X5 ) = 0, cov(X2 , X3 ) = 0. ◦ Identically Distributed : All individual measurement Xi have the same distribution. In particular, this means that all Xi ’s have the same mean µX and the same standard deviation σX . • Simple random sampling produces n random observations X1 , . . . , Xn that are i.i.d. • The sample average X is a random variable and has a sampling distribution, because the value of the sample mean is different every time we take a different sample. In other words, there is only population but many possible samples, so the value of the sample statistic we obtain would differ from sample to sample. This variability is called sampling variation. • Important formulas to know about X (make sure you know how to derive these, see equation 2.44 and 2.45 of the textbook): ◦ E(X) = µX ◦ var(X) = ◦ sd(X) = 3 2 σX n σX √ n Law of Large Numbers (LLN) • The Law of Large Numbers (LLN) states that if X1 , . . . , Xn are i.i.d. with E(Xi ) = µX and if p large outliers are unlikely, then X −→ µX . p • What does X −→ µX mean? This is “convergence in probability.” We say that X converges in probability to µX if for any constant c, the probability that X is in the range (µX − c) to (µX + c) becomes close to 1 as n increases. • Intuitively, the LLN says that as the sample size n increases, with very high probability (approaching 1), the sample average X will be arbitrarily close to the expected value µX 4 Central Limit Theorem (CLT) • In general, we do not know what the distribution of the sample mean X is. • However, the Central Limit Theorem (CLT) tells us that if (1) X1 , . . . , Xn are i.i.d. with E(Xi ) = 2 2 2 µX and var(Xi ) = σX , (2) var(Xi ) = σX is finite, i.e. var(Xi ) = σX < ∞, and (3) the sample size n 2 is sufficiently large, then X is approximately normally distributed with mean µX and variance σX /n. That is,   σ2 X ∼ N µX , X n 2