Distributions

Lecture 6 Gamma distribution, �2-distribution, Student t-distribution, Fisher F -distribution. Gamma distribution. Let us take two parameters � > 0 and � > 0. Gamma function �(�) is defined by � � �(�) = x�−1 e−x dx. 0 If we divide both sides by �(�) we get � � � � � 1 �−1 −x � 1= x e dx = y �−1 e−�y dy �(�) �(�) 0 0 where we made a change of variables x = �y. Therefore, if we define � � � �−1 −�x x e , x∼0 �(�) f (x|�, �) = 0, x<0 then f (x|�, �) will be a probability density function since it is nonnegative and it integrates to one. Definition. The distribution with p.d.f. f (x|�, �) is called Gamma distribution with parameters � and � and it is denoted as �(�, �). Next, let us recall some properties of gamma function �(�). If we take � > 1 then using integration by parts we can write: � � � � �−1 −x �(�) = x e dx = x�−1 d(−e−x ) 0 0 �� � � � = x�−1 (−e−x )� − (−e−x )(� − 1)x�−2 dx 0 0 � � = (� − 1) x(�−1)−1 e−x dx = (� − 1)�(� − 1). 0 Since for � = 1 we have �(1) = we can write � � e−x dx = 1 0 �(2) = 1 · 1, �(3) = 2 · 1, �(4) = 3 · 2 · 1, �(5) = 4 · 3 · 2 · 1 and proceeding by induction we get that �(n) = (n − 1)! Let us compute the kth moment of gamma distribution. We have, � � � � � �� k k � �−1 −�x x x e dx = x(�+k)−1 e−�x dx EX = �(�) �(�) 0 0 � � � �+k � � �(� + k) x�+k−1 e−�x dx = �(�) � �+k �(� + k) �0 �� � p.d.f. of �(� + k, �) integrates to 1 � � �(� + k) �(� + k) (� + k − 1)�(� + k − 1) = = = �+k k �(�) � �(�)� �(�)� k (� + k − 1)(� + k − 2) . . . ��(�) (� + k − 1) · · · � = = . k �k �(�)� Therefore, the mean is EX = the second moment is EX 2 = � � (� + 1)� �2 and the variance (� + 1)� � � �2 � Var(X) = EX − (EX) = − = 2. 2 � � � 2 2 Below we will need the following property of Gamma distribution. Lemma. If we have a sequence of independent random variables X1 � �(�1 , �), . . . , Xn � �(�n , �) then X1 + . . . + Xn has distribution �(�1 + . . . + �n , �) Proof. If X � �(�, �) then a moment generating function (m.g.f.) of X is � � � � � � � tX tx � �−1 −�x Ee = e x e dx = x�−1 e−(�−t)x dx �(�) �(�) 0 0 � � (� − t)� � � x�−1 e−(�−t)x dx . = (� − t)� �(�) 0 � �� � 36 The function in the last (underbraced) integral is a p.d.f. of gamma distribution �(�, � − t) and, therefore, it integrates to 1. We get, � � �� tX Ee = . �−t � Moment generating function of the sum ni=1 Xi is Ee t Pn i=1 Xi =E n � e tXi = i=1 n � Ee tXi i=1 P n � � � �� i � � � � i = = �−t �−t i=1 and this is again a m.g.f. of Gamma distibution, which means that n � i=1 Xi � � n �� i=1 � �i , � . ∂2n -distribution. In the previous lecture we defined a ∂2n -distribution with n degrees of freedom as a distribution of the sum X12 + . . . + Xn2 , where Xi s are i.i.d. standard normal. We will now show that which ∂2n -distribution coincides with a gamma distribution �( n2 , 12 ), i.e. �n 1� ∂2n = � , . 2 2 Consider a standard normal random variable X � N(0, 1). Let us compute the distribution of X 2 . The c.d.f. of X 2 is given by � �x ≥ ≥ t2 1 2 P(X � x) = P(− x � X � x) = � ≥ e− 2 dt. 2α − x d The p.d.f. can be computed by taking a derivative dx P(X � x) and as a result the p.d.f. of 2 X is � �x d 1 − t2 1 − (�x)2 ≥ � 1 − (−�x)2 ≥ � 2 2 2 dt = ≥ ≥ fX 2 (x) = ( x) − ≥ (− x) e e e dx −�x 2α 2α 2α x x 1 1 1 1 = ≥ ≥ e− 2 = ≥ x 2 −1 e− 2 . 2α x 2α We see that this is p.d.f. of Gamma Distribution �( 12 , 12 ), i.e. we proved that X 2 � �( 12 , 12 ). Using Lemma above proves that X12 + . . . + Xn2 � �( n2 , 12 ). Fisher F -distribution. Let us consider two independent random variables, �k 1� �m 1� 2 2 X � ∂k = � , and Y � ∂m = � , . 2 2 2 2 Definition: Distribution of the random variable Z= X/k Y /m 37 is called a Fisher distribution with degrees of freedom k and m, is denoted by F k,m . First of all, let us notice that since X � ∂2k can be represented as X12 + . . . + Xk2 for i.i.d. standard normal X1 , . . . , Xk , by law of large numbers, 1 2 (X + . . . + Xk2 ) � EX12 = 1 k 1 when k � →. This means that when k is large, the numerator X/k will ’concentrate’ near 1. Similarly, when m gets large, the denominator Y /m will concentrate near 1. This means that when both k and m get large, the distribution Fk,m will concentrate near 1. Another property that is sometimes useful when using the tables of F -distribution is that � 1� Fk,m (c, →) = Fm,k 0, . c This is because � X/k � � Y /m � 1� 1� Fk,m (c, →) = P ∼c =P � = Fm,k 0, . Y /m X/k c c Next we will compute the p.d.f. of Z � Fk,m . Let us first compute the p.d.f. of k X Z= . m Y The p.d.f. of X and Y are k m ( 12 ) 2 k −1 − 1 x ( 12 ) 2 m −1 − 1 y 2 2 f (x) = k x e and g(y) = m y 2 e 2 �( 2 ) �( 2 ) correspondingly, where x ∼ 0 and y ∼ 0. To find the p.d.f of the ratio X/Y, let us first write its c.d.f. Since X and Y are always positive, their ratio is also positive and, therefore, for t ∼ 0 we can write: � � �� ty �X � � P � t = P(X � tY ) = f (x)g(y)dx dy Y 0 0 since f (x)g(y) is the joint density of X, Y. Since we integrate over the set {x � ty} the limits of integration for x vary from 0 to ty. Since p.d.f. is the derivative of c.d.f., the p.d.f. of the ratio X/Y can be computed as follows: � � � � � d �X d � ty P �t = f (x)g(y)dxdy = f (ty)g(y)ydy dt Y dt 0 0 0 m � � 1 k ( 12 ) 2 m −1 − 1 y (2)2 1 k −1 − ty = (ty) 2 e 2 y 2 e 2 ydy �( m2 ) �( k2 ) 0 k+m � � ( 12 ) 2 k k+m 1 −1 = t2 y ( 2 )−1 e− 2 (t+1)y dy k m �( 2 )�( 2 ) �0 �� � The function in the underbraced integral almost looks like a p.d.f. of gamma distribution �(�, �) with parameters � = (k + m)/2 and � = 1/2, only the constant in front is missing. If we miltiply and divide by this constant, we will get that, k+m k+m � � 1 � ( 12 ) 2 ( 2 (t + 1)) 2 ( k+m )−1 − 1 (t+1)y �( k+2m ) k d �X −1 �t = y 2 t2 e 2 dy P k+m dt Y ) �( k+m �( k2 )�( m2 ) ( 12 (t + 1)) 2 0 2 = �( k+2m ) k −1 k+m t 2 (1 + t) 2 , k m �( 2 )�( 2 ) since the p.d.f. integrates to 1. To summarize, we proved that the p.d.f. of (k/m)Z = X/Y is given by �( k+m ) k k+m fX/Y (t) = k 2 m t 2 −1 (1 + t)− 2 . �( 2 )�( 2 ) Since � kt � k kt � � =≤ fZ (t) = P(Z � t) = fX/Y , Y m �t m m this proves that the p.d.f. of Fk,m -distribution is P(Z � t) = P �X � fk,m (t) = = �( k+m ) k � kt � k2 −1 � kt �− k+2m 2 1 + . �( k2 )�( m2 ) m m m �( k+2m ) k/2 m/2 k −1 k+m k m t 2 (m + kt)− 2 . k m �( 2 )�( 2 ) Student tn -distribution. Let us recall that we defined tn -distibution as the distribution of a random variable X1 T =� 1 (Y12 + · · · + Yn2 ) n if X1 , Y1 , . . . , Yn are i.i.d. standard normal. Let us compute the p.d.f. of T. First, we can write, � � X12 2 � t . P(−t � T � t) = P(T 2 � t2 ) = P (Y12 + · · · + Yn2 )/n If fT (x) denotes the p.d.f. of T then the left hand side can be written as � t P(−t � T � t) = fT (x)dx. −t On the other hand, by definition, X12 (Y12 + . . . + Yn2 )/n has Fisher F1,n -distribution and, therefore, the right hand side can be written as � t2 f1,n (x)dx. 0 We get that, � t fT (x)dx = −t � t2 f1,n (x)dx. 0 Taking derivative of both side with respect to t gives fT (t) + fT (−t) = f1,n (t2 )2t. But fT (t) = fT (−t) since the distribution of T is obviously symmetric, because the numerator X has symmetric distribution N (0, 1). This, finally, proves that fT (t) = f1,n (t2 )t = �( n+1 ) 1 t2 − n+1 2 ≥ (1 + ) 2 . n �( 12 )�( n2 ) n