University:
Massachusetts Institute of Technology
Course:
9.07 | Statistics for Brain and Cognitive Science
Academic year:

2024
Views:

450

Pages:

2
Author:

Leyla Diaz

Lecture 9 Addendum 1: The Observed Fisher Information and Fisher Information for the Binomial Model Recall that k = n ∑ xi is the number of correct responses in the binomial model. The log i =1 likelihood function for this model is log L( p) ∝ k log p (n − k ) log(1 − p ) . (A.1) ∂ log L( p) k n − k = − ∂p p (1 − p ) (A.2) The score function is and the second derivative ∂ 2 log L( p ) ∂p 2 = −[ k p 2 We defined the observed Fisher information as + n−k (1 − p ) 2 ]. (A.3) −∂ 2 log L( p) ∂p 2 If we compute the ML estimate pˆ ML k n as pˆ ML = , then we get I ( pˆ ML ) = −[ ∂ 2 log L( p ) ∂p 2 ]=[ k n−k ] (1 − p )2 p = pˆ ML n k n−k ] = [ + n pˆ ML 2 (1 − pˆ ML ) 2 (1 − pˆ ML ) pˆ ] = n[ ML + 2 (1 − pˆ ML )2 pˆ ML p = n[ 2 + 1 pˆ ML + (A.4) 1 ] (1 − pˆ ML ) = n[ pˆ ML (1 − pˆ ML )]−1 Recall that E (k ) = np. Therefore from Eqs. 9.38 and 9.39, the Fisher information for this model can be computed two ways I ( p ) = − E[ ∂ 2 log L( p) 2 ] = E[ k 2 + n−k (1 − p) 2 p np n − np ] =[ + p 2 (1 − p )2 1 1 = n[ + ] p (1 − p) 2 ∂p = n[ p (1 − p)]−1 ] (A.5) The second way the Fisher information can also be computed is as the expectation of the square of the score function. To see this we take I ( p ) = E[ ∂ log L( p) 2 k n−k 2 ] = E[ − ] ∂p p (1 − p) k − kp − np + kp 2 = E[ ] p (1 − p) = E[ = (k − np )2 p(1 − p ) 2 Var (k ) 2 [ p(1 − p )] (A.6) ] = np (1 − p ) 2 [ p(1 − p)] = n p(1 − p ) To estimate the Fisher information we evaluate Eqs. A.5 and A.6 at p = pˆ ML . Plugging p = pˆ ML into either Eq. A.5 or A.6 gives the same estimate Eq. A.4. This is in general not the case because the expectations in Eqs. A.5 and A.6 can be difficult to compute. To summarize, the observed Fisher information is I ( pˆ ML ) = −[ ∂ 2 log L( p) ∂p 2 ] = pˆ ML n . pˆ ML (1 − pˆ ML ) (A.7) and it is the estimate of the Fisher information which can be computed two ways as − E[ ∂ 2 log f ( x | p ) ∂p 2 ] = E[ n log f ( x | p) 2 . ] = ∂p p (1 − p ) (A.8) Remember L( p) = f ( x | p). The general results (Eq. 9.38 and 9.39) are I (θ ) = − E[ ∫ ∂ 2 log f ( x | θ ) ∂θ 2 ] ∂ 2 log f ( x | θ ) ) f ( x | θ )dx ∂θ 2 ∂ log f ( x | θ ) 2 = E[( ) ] ∂θ ∂ log f ( x | θ ) 2 = ( ) f ( x | θ )dx. ∂θ =− ( (A.9) ∫ The estimate of the Fisher information is the observed Fisher information defined as ∂ 2 log f ( x | θ ) ] I (θˆML ) = −[ . ∂θ 2 θ =θˆ ML (A.10)

Addendum 1: The Observed Fisher Information and Fisher Information for the Binomial Model

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock