Lecture 20: Weak Convergence

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Lecture 20 Fall 2013 11/25/2013 Introduction to the theory of weak convergence Content. 1. σ-fields on metric spaces. 2. Kolmogorov σ-field on C[0, T ]. 3. Weak convergence. In the first two sections we review some concepts from measure theory on metric spaces. Then in the last section we begin the discussion of the theory of weak convergence, by stating and proving important Portmentau Theorem, which gives four equivalent definitions of weak convergence. 1 Borel σ-fields on metric space We consider a metric space (S, ρ). To discuss probability measures on metric spaces we first need to introduce σ-fields. Definition 1. Borel σ-field B on S is the field generated by the set of all open sets U ⊂ S. Lemma 1. Suppose S is Polish. Then every open set U ⊂ S can be represented as a countable union of balls B(x, r). Proof. Since S is Polish, we can identify a countable set x1 , . . . , xn , . . . which is dense. For each xi ∈ U , since U is open we can find a radius ri such that B(xi , ri ) ⊂ U . Fix a constant M > 0. Consider ri∗ = min(arg sup{r : B(x, r) ⊂ U }, M ) > 0 and set ri = ri∗ /2. Then ∪i:xi ∈U B(xi , ri ) ⊂ U . In order to finish the proof we need to show that equality ∪i:xi ∈U B(xi , ri ) = U holds. Consider any x ∈ U . There exists 0 < r ≤ M such that B(x, r) ⊂ U . Since set (xi ) is dense there exists xk such that ρ(xk , x) < r/4. Then, by triangle inequality, B(xk , 3r/4) ⊂ B(x, r) ⊂ U . This means, by defini­ tion of rk∗ and rk , that rk ≥ 3r/8. But x ∈ B(xk , 3r/8) since the distance 1 ρ(xk , x) ≤ r/4. This shows x ∈ B(xk , rk ). Since x was arbitrary, then U ⊂ ∪i:xi ∈U B(xi , ri ). From this Lemma we obtain immediately that Corollary 1. Suppose S is Polish. Then B is generated by a countable collection of balls B(x, r), x ∈ S, r ≥ 0. 2 Kolmogorov σ-field on C[0, T ] Now let us focus on C[0, T ] and the Borel σ field B on it. For each t ≥ 0 define a projection πt : C[0, T ] → R as πt (x) = x(t). Observe that πt is a uniformly continuous mapping. Indeed |πt (x) − πt (y)| = |x(t) − y(t)| ≤ Ix − yI. This immediately implies uniform continuity. The family of projection mappings πt give rise to an alternative field. Definition 2. The Kolmogorov σ-field K on C[0, T ] is the σ-field generated by πt−1 (B), t ∈ [0, T ], B ∈ B, where B is the Borel field of R. It turns out (and this will be useful) that the two fields are identical: Theorem 1. The Kolmogorov σ field K is identical to the Borel field B of C[0, T ]. Proof. First we show that K ⊂ B. Since πt is continuous then, for every open set U ⊂ R πt−1 (U ) is open in C[0, T ]. This applies to all open intervals U . Thus each πt−1 (U ) ∈ B. This shows K ⊂ B. Now we show the other direction. Since C[0, T ] is Polish, then by Corol­ lary 1, it suffices to check that every ball B(x, r) ∈ K. Fix x ∈ C[0, T ], r ≥ 0. For each rational q ∈ [0, T ], consider Bq £ πq−1 ([x(q) − r, x(q) + r]). This is the set of all functions y such that |y(q) − x(q)| ≤ r. Consider ∩q Bq . As a countable intersection, this is a set in K. We claim that ∩q Bq = B. This implies the result. / B. To establish the claim, note that B ⊂ Bq for each q. Now suppose y ∈ Namely, for some t ∈ [0, T ] we have |y(t) − x(t)| ≥ r + δ > r. Find a sequence of rational values qn converging to t. By continuity of x, y we have x(qn ) → x(t), y(qn ) → y(t). Therefore for all sufficiently large n we have / ∩Bq . |y(qn ) − x(qn )| > r. This means y ∈ 2 3 Weak convergence We now turn to a very important concept of weak convergence or convergence of probability measures. Recall the convergence in distribution of r.v. Xn ⇒ X. Now consider random variables X : Ω → S which take values in some metric space (S, ρ). Again we define X to be a random variable if X is a measurable transformation. We would like to give a meaning to Xn ⇒ X. In order to do this we first define convergence of probability measures. Given a metric space (S, ρ) and the corresponding Borel σ-field B, suppose we have a sequence P, Pn , n = 1, 2, . . . of probability measures on (S, B). Let Cb (S) denote the set of all continuous bounded real valued functions on S. In particular every function X ∈ Cb (S) is measurable. Theorem 2 (Portmentau Theorem). The following conditions are equivalent. 1. limn EPn [X] = EP [X], ∀X ∈ Cb (S). 2. For every closed set F ⊂ S, lim supn Pn (F ) ≤ P(F ). 3. For every open set U ⊂ S, lim inf n Pn (U ) ≥ P(U ). 4. For every set A ∈ B such that P(∂A) = 0, the convergence limn Pn (A) = P(A) holds. What is the meaning of this theorem? Ideally we would like to say that a sequence of measures Pn converges to measure P if for every set A, Pn (A) → P(A). However, this turns out to be too restrictive. In some sense the fourth part of the theorem is the meaningful part. Second and third are technical. The first is a very useful implication. Definition 3. A sequence of measures Pn is said to converge weakly to P if one of the four equivalent conditions in Theorem 2 holds. Proof. (a)⇒(b) Consider a closed set F . For any E > 0, define F E = {x ∈ S : ρ(x, S) ≤ E}, where the distance between the sets is defined as the smallest distance between any two points in them. Let us first show that ∩E F E = F . The / F. inclusion ∩E F E ⊃ F is immediate. For the other side, consider any x ∈ Since S \ F is open, then B(x, r) ⊂ S \ F for some r. This means x ∈ / F r and the assertion is established. Invoking the continuity theorem, the equality ∩E F E = F implies lim P(F E ) = P(F ). E→0 3 (1) Define XE : S → [0, 1] by XE (x) = (1 − ρ(x, F )E−1 )+ . Then XE = 1 on F and XE = 0 outside of F E . Specifically, 1{F } ≤ XE ≤ 1{F E }, which implies Pn (F ) ≤ EPn [XE ] ≤ Pn (F E ), P(F ) ≤ EP [XE ] ≤ P(F E ), Note that XE is a continuous bounded function. Therefore, by assumption limn EPn [XE ] = E[XE ]. Combining, we obtain that lim sup Pn (F ) ≤ lim sup EPn [XE ] = lim EPn [XE ] = EP [XE ] ≤ P(F E ) n n n Finally, using (1) we conclude that lim supn Pn (F ) ≤ P(F ). (b)⇒ (c) This part follows immediately by observing that F = U c is a closed set and P(F ) = 1 − P(U ), Pn (F ) = 1 − Pn (U ). Moreover (c)⇒(b). (c)⇒ (d) Given any set A, consider its closure A¯ ⊃ A and interior Ao ⊂ A. Applying (b) and (c), we have P(Ā) ≥ lim sup Pn (Ā) ≥ lim sup Pn (A) ≥ lim inf Pn (A) ≥ lim inf Pn (Ao ) ≥ P(Ao ). n n n n Now if P(∂A) = P(Ā \ Ao ) = 0, then we obtain an equality across and, in particular, Pn (A) → P(A). w.l.g that 0 ≤ X ≤ 1. Then (d)⇒(a) Let t 1X ∈ Cb (S). We may assume t1 EPn [X] = 0 Pn (X > t)dt, EP [X] = 0 P(X > t)dt. Since X is continuous, for any set A(t) = {x ∈ S : X(x) > t}, its boundary satisfies ∂A ⊂ 1{X = t} (exercise). Observe that probability associated with set 1{X = t} is zero, except for countably many points t. This can be obtained by looking at sets of the form 1{X = t} with probability weights at least 1/m and taking a union over m. Thus for almost all t, the set A(t) is a continuity set, i.e., P(∂A) = 0 and hence Pn (A(t)) → P(A(t)). Since also 0 ≤ P, Pn ≤ 1, then using the Bounded Convergence Theorem applied to ”random” t, we obtain that 1 1 Pn (A(t))dt → 0 P(A(t))dt. 0 This implies the result. 4 Using the definition of weak convergence of measures we can define weak convergence of metric space valued random variables. Definition 4. Given a probability space (Ω, F, P) and a metric space (S, ρ), a sequence of measurable transformations Xn : Ω → S is said to converge weakly or in distribution to a transformation X : Ω → S, if the probability measures on (S, ρ) generated by Xn converge weakly to the probability measure generated by X on (S, ρ). 4 Additional reading materials • Notes distributed in the class, Chapter 5. • Billigsley [1] Chapter 1, Section 2. References [1] P. Billingsley, Convergence of probability measures, Wiley-Interscience publication, 1999. 5