GMM under Moderately High Dimensions

L9. GMM UNDER MODERATELY HIGH DIMENSIONS Abstract. Here we discuss the issues that arise in GMM under increasing dimension. This is not an easy material. 1. Introduction For the moment function g(θ) := Eg(X, θ), we assume that the true parameter value of interest θ0 ∈ Θ ⊂ Rdθ satisfes: g(θ0 ) = 0. We have data {Xi }ni=1 , which are identical copies of X, and, as a leading case we assume that they are also independent (i.i.d.). We form the empirical moment function: ĝ(θ) = En g(Xi , θ). Then our estimator θ̂ of θ0 is θ̂ ∈ arg min ĝ(θ)0 Âĝ(θ), θ∈Θ (GMM) where  is a positive-defnite matrix, possibly data-dependent, that converges to a nonstochastic positive-defnite matrix A. We claimed that under some conditions a θ̂ ∼ N (θ0 , V /n). To what extent does this result hold when the dimension of θ is high or when the number of moments considered is high? 2. GMM problems that separate in many low-dimensional GMM problems One case that is particularly nice is when the GMM problem can be disintegrated into many low dimensional problems, and we treat each in the limited information framework. For example, when we ran distribution regressions at multiple threshold values, the dimension of all the parameters stacked together was quite high. Was this a problem? No, it 1 2 turns out that we can handle very many low-dimensional problems in this way. Formally, the number of problems p could be very high but with the mild requirement that: (log p)7 /n → 0. (2.1) (θ10 , ..., θp0 )0 , Formally, suppose that θ = where each θj is a low dimensional parameter, namely dim θj is bounded as n → ∞, for a GMM problem indexed by j = 1, ..., p. Assume that the GMM estimator θ̂ = (θ̂10 , ..., θ̂p0 )0 obeys the asymptotic linearization property: n √ 1 X n(θ̂j − θj ) = √ n ϕj (Xi ) + rj , i=1 infuence function remainder where the remainder can be shown to be small max krj k = op (1/ log p). j≤p Conditions for such linearization are weak. Recall that for GMM, the infuence function is ϕj (Xi ) = −(G0j Aj Gj )−1 G0j Aj gj (Xi , θ0j ), where the index j signifes the dependence of the usual quantities on the j-th problem. Conditions on the remainder terms can also be established under mild conditions. Then it turns out that under some technical moment conditions and (2.1) we can still claim that √ a n(θ̂ − θ0 ) ∼ N (0, V ), (2.2) but in the sense that √ sup P( n(θ̂ − θ0 ) ∈ A) − P(N (0, V ) ∈ A) → 0, A∈R Pp where R is the collection of rectangular sets in R j=1 dim θj . The distinction that it holds for rectangular sets is important in very high dimensions, as the result is not true for elliptical regions or other convex regions. For inference purposes, we can also employ bootstrap or at least quick bootstrap, where we bootstrap the estimated infuence functions. These results follow from central limit theorems and bootstrap results for high dimensional vector means or approximate means developed in [3]. 3. GMM problems that do not separate into low-dimensional problems We study now the situation where the number of parameters or the number of moments is large. This case includes nonlinear least squares or M-estimation problems with many regressors, and the Arellano-Bond estimator for linear panel models with sequential exogeneity and large T . L9 3 We partition the parameter vector as: θ = (α0 , γ 0 )0 , θ0 = (α00 , γ00 )0 , where α is the target parameter and γ is the nuisance parameter, and α0 and γ0 are their respective true values. The dimension of α is low, the dimension of γ is high. We approximate this situation as p = dim(γ) → ∞ as n → ∞, while d = dim(α) is fxed. Also the number of moments used in GMM, m = dim(g(Xi , θ0 )), could be high, so we can approximate this situation as m → ∞ as n → ∞. Assertion 1 (GMM Inference with Moderately Many Parameters and Moments). There exist regularity conditions such that if the square of the dimensionality of the nuisance parameter and the number of moments is small compared to the sample size, namely that: (p + m)2 /n → 0 as n → ∞, then the approximate normality and consistency results continue to hold: √ a n(α̂ − α) ∼ N (0, V11 ), where V11 is the upper-left block of V in (2.2). Remark 1. Suÿcient conditions are given, for example, by [7] for GMM problems with m → ∞ and p is fxed; and by [5, 4] for nonlinear panel data models where m ∝ p → ∞. For exactly identifed exogeneous linear models, this condition can often be improved to requiring that the dimension is small compared to the sample size, p/n → 0. For exogenous linear models, very strong results were obtained in a sequence of papers by [6, 1, 2], which also cover the case where p/n → c > 0, which gives rise to additional terms in the variance formula.  The result above has a simple practical message: in nonlinear models or models with endogeneity, p2 and m2 should be small compared to n. For example, we saw that in Arellano-Bond approach the number of moments m = O(T 2 ), which could be too large relative to the sample size nT . This seems to be binding in the second empirical example that we considered where n = 147 and T = 19. This practical rule may be too rough as a guide in some applications. In such cases we may carry out Monte-Carlo experiments, using data-generating processes that mimic the empirical settings one is facing, to see how the methods perform. To understand Assertion 1, let us focus on the case where p = m. An asymptotic second order expansion of α̂ around α0 gives √ α̂ − α0 = Zn / n + b/n + rn , 4 a where Zn ∼ N (0, V11 ), b = O(p) is a frst order bias term, and rn is the higher order remainder such as rn = Op ((p/n)3/2 + p1/2 /n). Then, √ a n(α̂ − α0 ) ∼ N (0, V11 ) if both and √ nb/n → 0, √ nrn →P 0, i.e. p2 /n → 0, i.e. p3/2 /n → 0. Example 1 (A Contrived Example). Suppose that we are interested in the parameter α = γ 0 γ, and α0 = γ00 γ0 = 1. Suppose also that γ̂ − γ0 = Gn ∼ N (0, Ip /n). Then the α̂ = γ̂ 0 γ̂ obeys an exact Taylor expansion: √ p α̂ − α0 = 2γ00 Gn + G0n Gn = Zn / n + + rn , n where Zn ∼ N (0, 4), and nG0n Gn ∼ χp2 . Then, EG0n Gn = p/n, and rn = G0n Gn − p/n obeys p Ern2 . p1/2 /n, so that rn = OP (p1/2 /n) by Markov inequality.  This example shows that the bias is the bottleneck. If we remove the bias somehow, then we can improve the requirement from p2 /n → 0 to a weaker condition. There are several ways of removing the bias: a) Analytical bias correction, where we estimate b/n using analytical expressions for the bias and set α̌ = α̂ − b̂/n. In Example 1 above, the bias corrected estimator is just α̂ − p/n, but the expressions are much more complex in real problems. b) Split-sample bias correction, where we split the sample into two parts, compute the estimator on the two parts α̂(1) and α̂(2) , then set ᾱ = 12 α̂(1) + 12 α̂(2) , and then set α̌ = α̂ − (ᾱ − α̂) = 2α̂ − α. ¯ Here we can average over many splits to reduce variability, and it is also possible to use the bootstrap and leave-one-out methods for bias correction. Why does the sample-splitting method work? The frst order biases of α̂, α̂(1) , and α̂(2) are b b b , , , n n/2 n/2 L9 5 so that the frst order bias of α̌ is      b 1 b 1 b 2 − + = 0. n 2 n/2 2 n/2 With the bias correction, the resulting side conditions are weaker. Assertion 2 (Bias-Corrected GMM Inference with Moderately Many Parameters and Moments). There exist regularity conditions such that if the dimensionality and the number of moments is small compared to the sample size, namely that: (p + m)3/2 /n → 0 as n → ∞, then the approximate normality and consistency results for the bias-corrected GMM estimator continue to hold: √ a n(α̌ − α) ∼ N (0, V11 ), where V11 is the upper-left block of V in (2.2). References [1] Alexandre Belloni, Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Some new asymptotic theory for least squares series: Pointwise and uniform results. Journal of Econometrics, 186(2):345–366, 2015. [2] Matias Cattaneo, Michael Jansson, and Whitney Newey. Alternative asymptotics and the partially linear model with many regressors. CeMMAP working papers CWP36/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies, July 2015. [3] V. Chernozhukov, D. Chetverikov, and K. Kato. Central Limit Theorems and Bootstrap in High Dimensions. Annals of Probability, 2016. forthcoming. [4] Jinyong Hahn and Guido Kuersteiner. Bias reduction for dynamic nonlinear panel models with fxed e ects. Econometric Theory, 27(06):1152–1191, 2011. [5] Jinyong Hahn and Whitney Newey. Jackknife and analytical bias reduction for nonlinear panel models. Econometrica, 72(4):1295–1319, 2004. [6] Whitney K. Newey. Convergence rates and asymptotic normality for series estimators. Journal of Econometrics, 79:147–168, 1997. [7] Whitney K. Newey and Frank Windmeijer. Generalized method of moments with many weak moment conditions. Econometrica, 77(3):687–719, 2009.