1.010 - Brief Notes #6

1.010 - Brief Notes # 6 Second-Moment Characterization of Random Variables and Vectors. Second-Moment(SM) and First-Order Second-Moment(FOSM) Propagation of Uncertainty (a) Random Variables . • Second-Moment Characterization • Mean (expected value) of a random variable � E[X] = mX = xi PX (xi ) (discrete case) all xi � ∞ = xfX (x)dx (continuous case) −∞ • Variance (second central moment) of a random variable � 2 σX = V ar[X] = E[(X − mX )2 ] = (xi − mX )2 PX (xi ) all xi 2 σX � ∞ (x − mX )2 fX (x)dx = (continuous case) −∞ • Examples • Poisson distribution PY (y) = (λt)y e−λt , y! y = 0, 1, 2, . . . mY = λt σY2 = ∞ � (y − λt)2 PY (y) = λt = mY y=0 1 (discrete case) 2 • Exponential distribution fX (x) = λe−λx , mX = 2 σX = x≥0 1 λ �∞ � 0 1 x− λ �2 fX (x)dx = 12 = m2X λ • Notation X ∼ (m, σ 2 ) indicates that X is a random variable with mean value m and variance σ 2 . • Other measures of location • Mode x̃ = value that maximizes PX or fX • Median x50 = value such that FX (x50 ) = 0.5 • Other measures of dispersion • Standard deviation σX = � 2 σX (same dimension as X) • Coefficient of variation VX = σX mX (dimensionless quantity) • Expectation of a Function of a Random Variable. Initial and Central Moments. • Expected value of a function of a random variable Let Y = g(X) be a function of a random variable X. Then the mean value of Y is: 3 E[Y ] = E[g(X)] = �∞ −∞ yfY (y)dy Importantly, it can be shown that E[Y ] can also be found directly from fX , as: E[Y ] = �∞ −∞ g(x)fX (x)dx • Linearity of expectation It follows directly from the above and from linearity of integration that, for any constants a1 and a2 and for any functions g1 (X) and g2 (X): E[a1 g1 (X) + a2 g2 (X)] = a1 E[g1 (X)] + a2 E[g2 (X)] • Expectation of some important functions 1. E[X n ] = �∞ −∞ xn fX (x)dx (called initial moments; the mean mX is also the first initial moment) 2. E[(X − mX )n ] = �∞ −∞ (x − mX )n fX (x)dx 2 (called central moments; the variance σX is also called the second central moment) • Consequences of Linearity of Expectation. Uncertainty for Linear Functions. Second-Moment(SM) Propagation of 2 1. σX = V ar[X] = E[(X − mX )2 ] = E[X 2 ] − 2mX E[X] + m2X = E[X 2 ] − m2X 2 2 ⇒ E[X 2 ] = σX + mX 2. Let Y = a + bX, where a and b are constants. Using linearity of expectation, one obtains the following expressions for the mean value and variance of Y : mY = a + bE[X] = a + bmX 2 σY2 = E[(Y − mY )2 ] = b2 σX 4 • First-Order Second-Moment(FOSM) Propagation of Uncertainty for Nonlinear Functions Usually, with knowledge of only the mean value and variance of X, it is impossible to calculate mY and σY2 . However, a so-called first-order second-moment(FOSM) approximation can be obtained as follows. 2 Given X ∼ (mX , σX ) and Y = g(X), a generic nonlinear function of X, find the mean value and variance of Y . → Replace g(X) by a linear function of X, usually by linear Taylor expansion around mX . This gives the following approximation to g(X): � dg(X) �� Y = g(X) ≈ g(mX ) + (X − mX ) dX �mX Then approximate values for mY and σY2 are: � mY = g(mX ), σY2 = � �2 dg(X) �� 2 σX dX �mX (b) Random Vectors . • Second-Moment Characterization. Initial and Central Moments. Consider a random vector X with components X1 , X2 , . . . , Xn . • Expected value ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ X1 E[X1 ] m1 ⎢ . ⎥ ⎢ . ⎥ ⎢ . ⎥ . ⎥ ⎢ . ⎥ ⎢ . ⎥ E[X] = E ⎢ ⎣ . ⎦ = ⎣ . ⎦ = ⎣ . ⎦ = m (mean value vector) Xn E[Xn ] mn • Expected value of a scalar function of X Let Y = g(X) be a function of X. Then, extending a result given previously for functions of single variables, one finds that E[Y ] may be calculated as: 5 E[Y ] = � g(x)fX (x)dx Rn Again, it is clear that linearity applies, in the sense that, for any given constants a1 and a2 and any given functions g1 (X) and g2 (X): E[a1 g1 (X) + a2 g2 (X)] = a1 E[g1 (X)] + a2 E[g2 (X)] • Expectation of some special functions • Initial moments 1. Order 1: E[Xi ] = mi ⇔ E[X] = m, �∞ �∞ 2. Order 2: E[Xi Xj ] = 3. Order 3: E[Xi Xj Xk ] = . . . , −∞ −∞ i = 1, 2, . . . , n xi xj fXi ,Xj (xi , xj )dxi dxj , i, j = 1, 2, . . . , n i, j, k = 1, 2, . . . , n • Central moments 1. Order 1: E[Xi − mi ] = 0, i = 1, 2, . . . , n 2. Order 2 (covariance between two variables): Cov[Xi , Xj ] = E[(Xi − mi )(Xj − mj )], i, j = 1, 2, . . . , n � ∞� ∞ = (xi − mi )(xj − mj )fXi ,Xj (xi , xj )dxi dxj −∞ −∞ • Covariance in terms of first and second initial moments Using linearity of expectation, Cov[Xi , Xj ] = E[(Xi − mi )(Xj − mj )] = E[Xi Xj − Xi mj − mi Xj + mi mj ] = E[Xi Xj ] − mi mj ⇒ E[Xi Xj ] = Cov[Xi , Xj ] + mi mj 6 • Covariance Matrix and Correlation Coefficients • Covariance matrix ΣX ⎡ Cov[Xi , Xj ] ⎢ ⎢ =⎣ ⎤ .. ⎥ ⎥ ⎦ . (i, j = 1, 2, . . . , n) = E[(X − mx )(X − mx )T ] - For n = 2: � ΣX = σ12 Cov[X1 , X2 ] Cov[X2 , X1 ] σ22 � 2 - ΣX is the matrix equivalent of σX - ΣX is symmetrical: ΣX = ΣTX • Correlation coefficient between two variables ρij = Cov[Xi , Xj ] , σi σj i, j = 1, 2, . . . , n, −1 ≤ ρij ≤ 1 - ρij is a measure of linear dependence between two random variables; - ρij has values between -1 and 1, and is dimensionless. 7 • SM Propagation of Uncertainty for Linear Functions of Several Variables Let Y = a0 + n � ai Xi = a0 + a1 X1 + a2 X2 + · · · + an Xn be a linear function of the vector X. Using i=1 linearity of expectation, one finds the following important results: � � n n � � E[Y ] = E a0 + ai Xi = a0 + a i mi i=1 V ar[Y ] = n � i=1 a2i V ar[Xi ] + 2 i=1 n n � � i=1 j=i+1 ai aj Cov[Xi , Xj ] 8 • For n = 2: Y = a0 + a1 X1 + a2 X2 E[Y ] = a0 + a1 E[X1 ] + a2 E[X2 ] V ar[Y ] = a21 V ar[X1 ] + a22 V ar[X2 ] + 2a1 a2 Cov[X1 , X2 ] • For uncorrelated random variables: V ar[Y ] = n � i=1 a2i V ar[Xi ] • Extension to several linear functions of several variables Let Y be a vector whose components Yi are linear functions of a random vector X. Then, one can write Y = a + B X, where a is a given vector and B is a given matrix. One can show that: mY = a + B m X ΣY = B Σ X B T • FOSM Propagation of Uncertainty for Nonlinear Functions of Several Variables Let X ∼ (mX , ΣX ) be a random vector with mean value vector mX and covariance matrix ΣX . Consider a nonlinear function of X, say Y = g(X). In general, mY and σY2 depend on the entire joint distribution of the vector X. However, simple approximations to mY and σY2 are obtained by linearizing g(X) and then using the exact SM results for linear functions. If linearization is obtained through linear Taylor expansion about mX , then the function that replaces g(X) is: � n ∂g(X) � � � g(X) ≈ g(mX ) + ∂Xi � i=1 (Xi − mi ) X=mX where mi is the mean value of Xi . The approximate mean and variance of Y are then: mY = g(mX ), 9 σY2 = n � n � bi bj Cov[Xi , Xj ] i=1 j=1 where bi = � ∂g(X) �� ∂Xi �X=m X This way of propagating uncertainty is called FOSM analysis.