University:
University of Michigan
Course:
STATS 500 | DATASCI 500. Statistical Learning I: Regression
Academic year:

2021
Views:

326

Pages:

9
Author:

Marinees46

Scale parameters and quasi-likelihood Since the quasi-likelihood estimating equations are homogeneous, we can estimate the mean structure µ = exp(β 0 x) in a setting where the specified variance is off by a multiplicative constant. For example, these estimating equations can be used to consistently estimate β in a quasi-Poisson model where Var[yi |xi ] = φE [yi |xi ]. This is a quasi-likelihood estimator, because there is no single ”quasi-Poisson distribution”. There are many distributions that have this variance structure, but the solution to these estimating equations is not the MLE for a specific distribution. This can be viewed as a way to construct a consistent estimator for β that can be used for any distribution where the conditional variance has this structure. 44 / 52 Scale parameters and quasi-likelihood In a quasi-likelihood analysis, the scale parameter is usually estimated in a separate step, after the regression parameters (β) are estimated by solving the estimating equations. There are several related ways to estimate the scale parameter. A common approach is to use P φ̂ = − µ̂i )2 /v̂i . n−p i (yi 45 / 52 Overdispersion Under the Poisson model, var[y |x] = E [y |x]. A Poisson model results from using the Poisson GLM with the scale parameter φ fixed at 1. The quasi-Poisson model is the Poisson model with a scale parameter that may be any non-negative value. Under the quasi-Poisson model, var[y |x] ∝ E [y |x]. The negative binomial GLM allows the variance to be non-proportional to the mean. Any situation in which var[y |x] > E [y |x] is called overdispersion. Overdispersion is often seen in practice. One mechanism that may give rise to overdispersion is heterogeneity. Suppose we have a hierarchical model in which λ follows a Γ distribution, and y |λ is Poisson with mean parameter λ. Then marginally, y is negative binomial. 46 / 52 Shape and other auxiliary parameters We have seen that the scale parameter can be estimated intependently of the regression parameters (β). Some GLM’s (or quasi-GLM’s) contain additional parameters that cannot be estimated independently of β. Once example of this is the shape parameter α in the negative binomial GLM. The shape parameter can be estimated by maximum likelihood, together with β (using a profile likelihood technique), or can be selected using QIC. Gamma and beta GLM’s also have auxiliary parameters that are estimated in this way. 47 / 52 Model comparison for GLM’s If φ is held fixed across models, then twice the log-likelihood ratio between two nested models θ̂(1) and θ̂(2) is L≡2 X i (1) (yi θ̂i (1) − γ(θ̂i ))/φ − 2 X (2) (2) (yi θ̂i − γ(θ̂i ))/φ, i where θ̂(2) is nested within θ̂(1) , so L ≥ 0. This is called the scaled deviance. The statistic D = φL, which does not depend explicitly on φ, is called the deviance. 48 / 52 Model comparison for GLM’s Suppose that θ̂(1) is the saturated model, in which θi = yi . If the GLM is Gaussian and g (x) ≡ x, as discussed above, the deviance is D = 2 X X (2) (2) 2 (yi2 − yi2 /2) − 2 (yi θ̂i − θ̂i /2) i = X yi2 − (2) 2yi θ̂i + i (2) 2 θ̂i i = X (2) (yi − θ̂i )2 . i 49 / 52 Model comparison for GLM’s Thus in the Gaussian case, the deviance is the residual sum of squares for the smaller model (θ̂(2) ). In the Gaussian case, D/φ = L ∼ χ2n−p−1 . When φ is unknown, we can turn this around to produce an estimate of the scale parameter φ̂ = D . n−p−1 This is an unbiased estimate in the Gaussian case, but is useful for any GLM. 50 / 52 Model comparison for GLM’s Now suppose we want to compare two nested generalized linear models with deviances D1 < D2 . Let p1 > p2 be the number of covariates in each model. The likelihood ratio test statistic is L2 − L1 = D2 − D1 φ which asymptotically has a χ2p1 −p2 distribution. If φ is unknown, we can estimate it as described above (using the larger of the two models). The “plug-in” likelihood ratio statistic (D2 − D1 )/φ̂ is still asymptotically χ2p1 −p2 , as long as φ̂ is consistent. The finite sample distribution may be better approximated using D2 − D1 φ̂(p1 − p2 ) ≈ Fp1 −p2 ,n−p1 , 51 / 52 Model comparison for GLM’s We can compare any two fitted GLM’s using model selection statistics like AIC or BIC. AIC favors models having small values of Lopt − df, where Lopt is the maximized log-likelihood, and df is the degrees of freedom. Equivalently, the AIC can be expressed −D/2φ̂ − p − 1. The same φ̂ value should be used for all models being compared (i.e. by using the one from the largest model). 52 / 52

Generalized Linear Models-3

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock