Abstract Integration (II)

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Lecture 8 Fall 2018 ABSTRACT INTEGRATION — II Contents 1. Borel-Cantelli revisited 2. Connections between abstract integration and elementary definitions of integrals and expectations 3. Fatou’s lemma 4. Dominated convergence theorem In the previous lecture: (a) We defined the Rnotion of an integral of a measurable function with respect to a measure R ( g dµ), which subsumes the special case of expectations (E[X] = X dP), where X is a random variable, and P is a probability measure. (b) We saw that integrals are always well-defined, though possibly infinite, if the function being integrated is nonnegative. (c) For a general function g, we decompose it as the sum g = g+ − g− of a positive and a negative function, and R integrate each R piece separately. The integral is well defined unless both g+ dµ and g− dµ happen to be infinite. (d) We obeyR a long list natural properties, including linearity: R saw that integrals R (g + h) dµ = g dµ + h dµ. (e) We stated the Monotone Convergence Theorem (MCT), according to which, if {gn } is a nondecreasing sequence of nonnegative measurable R Rfunctions that converge pointwise to a function g, then limn→∞ gn dµ = g dµ. (f) Finally, we saw that for every nonnegative measurable function g, we can find an nondecreasing sequence of nonnegative simple functions that converges (pointwise) to g. 1 1 BOREL-CANTELLI REVISITED P Recall that one of the Borel-Cantelli lemmas states that if ∞ i=1 P(Ai ) < ∞, then P(Ai i.o.) = 0. In this section, we rederive this result using the new machinery that we have available. function of the event Ai , so that E[XP Let Xi be the indicator i ] = P(Ai ). P n Thus, by assumption ∞ E[X ] < ∞. The random variables i i=1 i=1 Xi are nonnegative and form an increasing sequence, as n increases. Furthermore, lim n→∞ n X Xi = i=1 ∞ X Xi , i=1 Pn P pointwise; that is, for every !, we have limn→∞ i=1 Xi (!) = ∞ i=1 Xi (!). We can now apply the MCT, and then the linearity property of expectations (for finite sums), to obtain ∞ n hX i hX i Xi E Xi = lim E n→∞ i=1 = lim n→∞ = lim n→∞ = ∞ X i=1 n X i=1 n X E[Xi ] P(Ai ) i=1 P(Ai ) i=1 < ∞. P∞ This implies that i=1 Xi < ∞, a.s. (This is intuitively obvious, but a short formal proof is actually needed.) It follows that, with probability 1, only finitely many of the events Ai can occur. Equivalently, the probability that infinitely many of the events Ai occur is zero, i.e., P(Ai i.o.) = 0. 2 CONNECTIONS BETWEEN ABSTRACT INTEGRATION AND ELEMENTARY DEFINITIONS OF INTEGRALS AND EXPECTATIONS Abstract integration would not be useful theory if it were inconsistent with the more elementary notions of integration. For discrete random variables taking values in a finite range, this consistency is automatic because of the definition of an integral of a simple function. We will now verify some additional aspects of this consistency. 2 2.1 Connection with Riemann integration. We state here the following reassuring result. Let Ω = R and  be the Lebesgue measure (considered on either Borel ˙-algebra, or its completion the Lebesgue ˙-algebra). Suppose that f is a Riemann integrable function on some interval [a, b]. Then, f is Lebesgue-measurable and its Lebesgue integral equals the Riemann integral: Z b Z Z f (x) dx = f d = 1[a,b] f d [a,b] a R In particular, every Borel function’s Lebesgue integral coincides with its Riemann integral whenever the latter exists. Proof (optional). Consider an arbitrary finite partition ˙ of [a, b]. Corresponding to each ˙ there is a piece-wise constant (hence simple) function f˙ (x) ≤ f (x) and f˙′ (x) ≥ f (x) such that Z f˙ d = L(˙) (1) [a,b] Z f˙′ d = U (˙) (2) [a,b] where L(˙) and U (˙) are lower and upper Darboux sums (defined in Lecture 7). There exists a sequence of partitions ˙n , each refining the previous one, such that L(˙n ) ր sup L(˙) (3) U (˙n ) ց inf U (˙) . (4) ˙ ˙ On the other hand the corresponding sequences of functions f˙n and f˙′ n are monotone, hence converging: f˙n (x) ր f (x) ≤ f (x) ∀x ∈ [a, b] (5) f˙′ n (x) ց f (x) ≥ f (x) ∀x ∈ [a, b] (6) From (1)-(2) and Riemann integrability we conclude that Z Z Z b f d = f d = g(x)dx . [a,b] Consequently, R [a,b] a |f − f |d = 0 and thus f (x) = f x = f (x) 3 for a.e. x (7) implying f is Lebesgue measurable (and coincides with some Borel measurable function except on a set of measure zero). The equality of Lebesgue and Riemann integrals of f in turn follows from (7). 2.2 Evaluating expectations by integrating on different spaces Consider a probability space (Ω, F, P). Let X : Ω → R, be a random variable. We then obtain a second probability space (R, B, PX ), where B is the Borel ˙-field, and PX is the probability law of X, defined by PX (A) = P({! ∈ | X(!) ∈ A}), A ∈ B. Consider now a measurable function g : R → R, and use it to define a new random variable Y = g(X), and a corresponding probability space (R, B, PY ). The expectation of Y can be evaluated in three different ways, that is, by integrating over either of the three spaces we have introduced. Theorem 1. We have Z Y dP = Z g dPX = Z y dPY , R and all three ’s exist or do not exist simultaneously. Proof: We follow the “standard program”: first establish the result for simple functions, then take the limit to deal with nonnegative functions, and finally generalize. Let g be a simple function, which takes values in a finite set y1 , . . . , yk . Using the definition of the integral of a simple function we have Z X Y dP = yi PY (Y = yi ) yi = X yi P({! | Y (!) = yi }) X yi P({! | g(X(!)) = yi }). yi = yi Similarly, Z g dPX = X yi PX ({x | g(x) = yi }). yi 4 However, from the definition of PX , we obtain PX ({x | g(x) = yi }) = PX (g−1 (yi )) = P({! | X(!) ∈ g−1 (yi )}) = P({! | g(X(!)) = yi }), and the equalities in the theorem follow, for simple functions. Let now g be nonnegative function, and let {gn } be an increasing sequence of nonnegative simple functions that converges to g. Note that gn (X) converges monotonically to g(X). We then have Z Z Z Z Z gn dPX = g dPX . Y dP = g(X) dP = lim gn (X) dP = lim n→∞ n→∞ (The second equality is the MCT; the third is the result that we already proved for simple functions; the last equality is once more the MCT.) The case of general (not just nonnegative) functions follows easily from the above – the details are omitted. This proves the theorem. 2.3 The case of continuous random variables, described by PDFs We can now revisit the development of continuous random variables (Lecture 4), in a more rigorous manner. We say that a random variable X : Ω → R is continuous if its CDF can be written in the form Z FX (x) = P(X ≤ x) = 1(−∞,x] f d, ∀x ∈ R, where  is Lebesgue measure, and f is a nonnegative measurable function wint R R f d = 1. Recall that by Theorem 4 of Lecture 4 to each CDF there corresponds a unique probability measure PX on (R, B). In this case PX has a particularly simple expression: Z f d (8) PX (A) = A for any Borel set A. (Obviously, the CDF of PX is FX . The fact that (8) defines a valid measure is property 10 from Lecture 7.) When f is Riemann integrable and the set A = [a, b] is an interval, we Rb can also write PX (A) = a f (x) dx, where the latter integral is an ordinary Riemann integral. 5 Theorem 2. For any measurable function g we have Z Z E[g(X)] = g dPX = (gf ) d R where all E and ’s exist or do not exist simultaneously. Note: Since integrals of non-negative functions always exist, this also gives R a convenient criterion: E[g(X)] is finite iff |g|f d < ∞. Proof: The first equality was shown in Theorem 1. So, let us concentrate on the second. Following the usual program, P let us first consider the case where g is a simple function, of the form g = ki=1 ai 1Ai , for some measurable disjoint subsets Ai of the real line. We have Z g dPX = = = k X i=1 k X ai PX (Ai ) ai i=1 k Z X Z f d Ai ai 1Ai f d i=1 = Z X k = Z ai 1Ai f d i=1 (gf ) d. The first equality is the definition of the integral for simple functions. The second uses Eq. (8). The fourth uses linearity of integrals. The fifth uses the definition of g. Suppose now that g is a nonnegative function, and let {gn } be an increasing sequence of nonnegative functions that converges to g, pointwise. Since f is nonnegative, note that gn f also increases monotonically and converges to gf . Then, Z Z Z Z gn dPX = lim (gn f ) d = (gf ) d. g dPX = lim n→∞ n→∞ The first and the third equality above is the MCT. The middle equality is the result we already proved, for the case of a simple function gn . 6 Finally, if g is not nonnegative, the result is proved by considering separately the positive and negative parts of g. When g and f are “nice” functions, e.g., piecewise continuous, Theorem 2 yields the familiar formula Z ∞ E[g(X)] = g(x)f (x) dx, −∞ where the integral is now an ordinary (improper) Riemann integral. 3 FATOU’S LEMMA Note that for any two random variables, we have min{X, Y } ≤ X and min{X, Y } ≤ Y . Taking expectations, we obtain E[min{X, Y }] ≤ min{E[X], E[Y ]}. Fatou’s lemma is in the same spirit, except that infinitely many random variables are involved, as well as a limiting operation, so some additional technical conditions are needed. Theorem 3. Let fn ≥ 0 be measurable, then Z Z lim inf fn dµ ≤ lim inf fn dµ n→∞ n→∞ Proof: Fix some n. We have inf fk ≤ fm , ∀ m ≥ n. k≥n Integrating both sides, we obtain Z Z inf fk dµ ≤ fm dµ, k≥n ∀ m ≥ n. Taking the infimum of both sides with respect to m, we obtain Z Z inf fk dµ ≤ inf fm dµ m≥n k≥n (9) The statement of the Theorem follows from (9) after taking the limit limn→∞ . Indeed, the sequence inf k≥n fk is nonnegative and nondecreasing with n, and converges to lim inf n→∞ fn . Therefore, from MCT we obtain Z Z Z lim inf fk dµ = lim inf fk , lim inf fn dµ n→∞ k≥n n→∞ k≥n 7 n→∞ Similarly, the limit as n → ∞ of the right-hand side of (9) converges to lim inf R fn dµ. Corollary 1. Let Y be a random variable that satisfies E[|Y |] < ∞. (a) If Y ≤ Xn , for all n, then E[lim inf n→∞ Xn ] ≤ lim inf n→∞ E[Xn ]. (b) If Xn ≤ Y , for all n, then E[lim supn→∞ Xn ] ≥ lim supn→∞ E[Xn ]. Proof: Apply Theorem 3 to Xn − Y or Y − Xn . 4 DOMINATED CONVERGENCE THEOREM The dominated convergence theorem complements the MCT by providing an alternative set of conditions under which a limit and an expectation can be interchanged. Theorem 4. (DCT) Consider a sequence of random variables {Xn } that converges to X a.e. Suppose that |Xn | ≤ Y , for all n, where Y is a nonnegative random variable that satisfies E[Y ] < ∞. Then, limn→∞ E[Xn ] = E[X]. Proof: Let A ⊂ Ω be the set of outcomes ! along which Xn (!) → X(!) as n → ∞. Then P(Ac ) = 0. Let X̃n (!) = Xn (!) for ! ∈ A and = 0 ˜ otherwise. Similarly, let X(!) = X(!), ! ∈ A and = 0 otherwise. Then E[X̃n ] = E[Xn ], E[X̃] = E[X] and X̃n → X̃ for all !. Thus we may assume, without the loss of generality that Xn (!) → X(!) for all !. Since −Y ≤ Xn ≤ Y , we can apply both parts of Fatou’s lemma , to obtain E[X] = E[lim inf Xn ] ≤ lim inf E[Xn ] ≤ lim sup E[Xn ] ≤ E[lim sup Xn ] = E[X]. n→∞ n→∞ n→∞ n→∞ This proves that E[X] = lim inf E[Xn ] = lim sup E[Xn ]. n→∞ n→∞ In particular, the limit limn→∞ E[Xn ] exists and equals E[X]. Remark: We note that the DCT remains valid for general measures, not just for probability measures (the proof is the same). However, the following statement (Bounded Convergence Theorem), is specific to probability measures: If there exists a constant c ∈ R such that |Xn | ≤ c, a.s., for all n, then limn E[Xn ] = E[limn Xn ]. 8 Corollary 2. Suppose that P∞ ∞ X n=1 E[|Zn |] E[Zn ] = E n=1 < ∞. Then, ∞ hX n=1 i Zn . Proof: By the monotone convergence theorem, applied to Yn = have ∞ ∞ hX i X E[|Zn |] < ∞. E |Zn | = n=1 Pn k=1 |Zk |, we n=1 Pn P∞ Let Xn = P∞ i=1 Zi and note that limn→∞ Xn = i=1 Zn . We observe that |Xn | ≤ i=1 |Zi |, which has finite expectation, as shown earlier. The result follows from the dominated convergence theorem. Exercise: Can you prove Corollary 1 directly from the monotonone convergence theorem, without appealing to the DCT or Fatou’s lemma? Theorems such as MCT and DCT impose assumptions additional to the assumption that Xn → X a.e. that insure that limn E[Xn ] = E[X]. It should not be surprising that, in general just having Xn → X a.e. is not enough. Here is a counter-example. Let Ω = [0, 1], let F be the Borel sigma-field B on [0, 1], and let P be the uniform (Lebesgue) probability measure. Let X(!) = 0 for all ! ∈ [0, 1]. Let ˆ n, when ! ∈ (0, n1 ); Xn (!) = 0, when ! = 0 or ! ∈ ( n1 , 1]. Verify that Xn (!) → 0 for all !, but E[Xn ] = n(1/n) = 1 and thus E[Xn ] → 0 does not hold. The example above shows that DCT does not hold unless we make an additional assumption, such as |Xn | ≤ Y for some random variable Y with E[Y ] < ∞. However, the sequence Xn is not increasing. Exercise: (a) Establish the following generalization of the MCT. Suppose Xn is a.e. increasing sequence of random variables, but suppose Xn are not necessarily non-negative. Let limn Xn = X a.e. Suppose Xn ≥ Y a.e. for some random variable Y . Finally, suppose the expectations of Xn , X and Y are all finite. Establish that limn E[Xn ] = E[X]. (b) Construct a sequence of random variables Xn which is increasing a.e., but E[Xn ] does not converge to E[X], where X = limn Xn a.e. 9