Lecture 12 Heteroscedasticity 1

Heteroscedasticity 1 Heteroscedasticity • Assumption (A3) is violated in a particular way:  has unequal variances, but i and j are still not correlated with each other. Some observations (lower variance) are more informative than others (higher variance). f(y|x) . . x1 x2 x3 . E(y|x) = b0 + b1x x 2 1 Heteroscedasticity • Now, we have the CLM regression with hetero-(different) scedastic (variance) disturbances. (A1) DGP: 𝒚 = X  +  is correctly specified. (A2) E[|X] = 0 (A3’) Var[ ] = 2  ,  > 0. (CLM   = 1, for all 𝑖.) (A4) X has full column rank – rank(X)=𝑘–, where 𝑇 ≥ 𝑘. • Popular normalization: ∑  = 1. (A scaling, absorbed into 2.) • A characterization of the heteroscedasticity: Well defined estimators and methods for testing hypotheses will be obtainable if the heteroscedasticity is “well behaved” in the sense that  / ∑  → 0 as T → . -i.e., no single observation becomes dominant. (1/T) ∑  → some stable constant. (Not a plim!) GR Model and Testing • Implications for conventional OLS and hypothesis testing: 1. b is still unbiased. 2. Consistent? We need the more general proof. Not difficult. 3. If plim b = , then plim 𝑠 = 2 (with the normalization). 4. Under usual assumptions, we have asymptotic normality. • Two main problems with OLS estimation under heterocedasticity: (1) The usual standard errors are not correct. (They are biased!) (2) OLS is not BLUE. • Since the standard errors are biased, we cannot use the usual tstatistics or F–statistics or LM statistics for drawing inferences. This is a serious issue. 2 Heteroscedasticity: Inference Based on OLS • Q: But, what happens if we still use 𝑠 (XX)-1? A: It depends on XX – XX. If they are nearly the same, the OLS covariance matrix will give OK inferences. But, when will XX – XX be nearly the same? The answer is based on a property of weighted averages. Suppose i is randomly drawn from a distribution with E[ ] = 1. Then, ∑  𝑥 E[𝑥 ] –just like ∑ 𝑥 . • Remark: For the heteroscedasticity to be a significant issue for estimation and inference by OLS, the weights must be correlated with 𝒙 and/or 𝑥 . The higher correlation, heteroscedasticity becomes more important (b is more inefficient). Finding Heteroscedasticity • There are several theoretical reasons why the  may be related to some explanatory variables 𝑧 , . . . , 𝑧 and/or 𝑧 2,…, 𝑧 2. Examples: 1. Following the error-learning models, as people learn, their errors of behavior become smaller over time. Then,  is expected to decrease.6 2. As data collecting techniques improve,  is likely to decrease. Companies with sophisticated data processing techniques are likely to commit fewer errors in forecasting customer’s orders. 3. As companies grow, companies expand and tend to be more diversified and, thus, safer. Hence,  is likely to decrease with size. 4. Companies with larger profits tend to have greater variability in their dividend/buyback policies than companies with lower profits. 3 Finding Heteroscedasticity • Heteroscedasticity can also be the result of model misspecification. • It can arise as a result of the presence of outliers (either very small or very large). The inclusion/exclusion of an outlier, especially if T is small, can affect the results of regressions. • Violations of (A1) – model is correctly specified-–, can produce heteroscedasticity, due to omitted variables from the model. • Skewness in the distribution of one or more regressors included in the model can induce heteroscedasticity. Examples are economic variables such as income, wealth, and education. • David Hendry notes that heteroscedasticity can also arise because of – (1) incorrect data transformation (e.g., ratio or first difference). – (2) incorrect functional form (e.g., linear vs log–linear models). Finding Heteroscedasticity • Heteroscedasticity is usually modeled using one the following specifications: – H1 :  is a function of past ε and past  (ARCH models). – H2 :  increases monotonically with one (or several) exogenous variable(s) (𝑧 ,, . . . , 𝑧 ). – H3 :  decreases monotonically with Size (Market Cap). – H4 :  is the same within p subsets of the data but differs across the subsets (grouped heteroscedasticity). This specification allows for structural breaks. • These are the usual alternatives hypothesis (H1) in the heteroscedasticity tests. 4 Finding Heteroscedasticity • Visual test In a plot of residuals against dependent variable or other variable will often produce a fan shape. 180 160 140 120 100 Series1 80 60 40 20 0 0 50 100 150 9 Testing for Heteroscedasticity • We can use OLS along the White estimator for consistent inferences. Then, why do we want to test for heteroscedasticity? • The GLS/FGSL estimator has a lower asymptotic variance. • We want to test: H0: E( |x , x , …, x ) = 2 • H1 and the structure of the test depend on what we consider the drivers of  – i.e., in the previous examples: H1, H2, H3, H4, etc • The key is whether E[ ] =  is related to 𝑿 and/or 𝑿2. Suppose we suspect a particular independent variable, say 𝒙 , is driving  :  = f(𝒙 ) • Then, a simple test: Check the RSS for large values of 𝒙 , and the RSS for small values of 𝒙 . This is the Goldfeld-Quandt (GQ) test. 5 Testing for Heteroscedasticity • The Goldfeld-Quandt test GQ tests H0:  = 2 H1:  f𝒙 ) • Easy to compute: – Step 1. Arrange the data from small to large values of the independent variable suspected of causing heteroscedasticity, 𝒙 . – Step 2. Run two separate regressions, one for small values of 𝒙 and one for large values of 𝒙 , omitting d middle observations (d ≈ 20%). Get the RSS for each regression: RSS1 for small values of 𝒙 and RSS2 for large 𝒙 ’s. – Step 3. Calculate the F ratio RSS2 , ~ 𝐹 , , with 𝑑𝑓=[(T – d) – 2(𝑘 +1)]/2 (A5 holds) GQ = RSS1 Testing for Heteroscedasticity • The Goldfeld-Quandt test If (A5) does not hold, the F distribution becomes an approximation. Other tests may be preferred. Note: When we suspect more than one variable is driving the  ’s, this test is not very useful. • But, the GQ test is a popular to test for structural breaks (two regimes) in variance. For these tests, we rewrite step 3 to allow for a different sample size in the sub-samples 1 and 2. - Step 3. Calculate the F-test ratio GQ = [RSS2/ (T2 – k)]/[RSS1/ (T1 – k)] 6 Testing for Heteroscedasticity: GQ Test Example: We test if the 3-factor FF model for IBM and GE returns shows heteroscedasticity with a GQ test, using gqtest in package lmtest. • IBM returns > library(lmtest) > gqtest(ibm_x ~ Mkt_RF + SMB + HML, fraction = .20) Goldfeld-Quandt test data: ibm_x ~ Mkt_RF + SMB + HML GQ = 1.1006, df1 = 224, df2 = 223, p-value = 0.2371 alternative hypothesis: variance increases from segment 1 to 2  cannot reject H0 at 5% level. • GE returns gqtest(ge_x ~ Mkt_RF + SMB + HML, fraction = .20) Goldfeld-Quandt test data: ge_x ~ Mkt_RF + SMB + HML GQ = 2.744, df1 = 281, df2 = 281, p-value < 2.2e-16 alternative hypothesis: variance increases from segment 1 to 2  reject H0 at 5% level. 23 Testing for Heteroscedasticity: LR Test • The Likelihood Ratio Test Let’s define the likelihood function, assuming normality, for a general case, where we have g different variances: T ln L   ln 2  2 g Ti 1 ln i2  2 2 i 1  g 1  i 1 2 i ( yi  X i)( yi  X i) We have two models: (R) Restricted under H0: i2 = 2. From this model, we calculate ln L T T ln LR   [ln(2)  1]  ln(ˆ 2 ) 2 2 (U) Unrestricted. From this model, we calculate the log likelihood. T ln LU   [ln(2)  1]  2 g Ti  2 ln ˆ ; i 1 2 i ˆ i2  T1 ( yi  X i b)( yi  X i b) i 7 Testing for Heteroscedasticity: LR Test • Now, we can estimate the Likelihood Ratio (LR) test: LR  2(ln LU  ln LR )  T ln ˆ 2  g T ln ˆ i i 2 a    2g 1 i 1 Under the usual regularity conditions, LR is approximated by a χ . • Using specific functions for  , this test has been used by Rutemiller and Bowers (1968) and in Harvey’s (1976) groupwise heteroscedasticity paper. Testing for Heteroscedasticity • Score LM tests • We want to develop tests of H0: E( |x , x , …, x ) = 2 against an H1 with a general functional form. • Recall the central issue is whether E[ ] =  is related to 𝑿 and/or 𝑿2. Then, a simple strategy is to use OLS residuals to estimate disturbances and look for relationships between e and 𝑥 and/or x . • Suppose that the relationship between ε2 and X is linear:  = 𝒙 ′α + 𝑣 Then, we test: H0: α = 0 against H1: α ≠ 0. • We base the test on how the squared OLS residuals, e, correlate with X. 8 Testing for Heteroscedasticity • Popular heteroscedasticity LM tests: - Breusch and Pagan (1979)’s LM test (BP). - White (1980)’s general test. • Both tests are based on OLS residuals. That is, calculated under H0: No heteroscedasticity. • The BP test is an LM test, based on the score of the log likelihood function, calculated under normality. It is a general tests designed to detect any linear forms of heteroskedasticity. • The White test is an asymptotic Wald-type test, normality is not needed. It allows for nonlinearities by using squares and crossproducts of all the 𝑥’s in the auxiliary regression. Testing for Heteroscedasticity: BP Test • Let’s start with a general form of heteroscedasticity: hi(α0 + 𝑧 , α + 𝑧 , α + .... + 𝑧 , α ) =  • We want to test: H0: E( |𝑧 , 𝑧 ,…, 𝑧 ) = hi(𝒛 ′α) = 2 or H0: α = α2 = ... = α =  (m restrictions) • Assume normality. That is, the log likelihood function is: log L = constant + ½ Σ log  – ½ Σ  / Then, construct an LM test: LM = S(θR)’ I(θR)-1 S(θR) θ= (β,α) -2 -2 S(θ)=∂log L/∂θ’=[-Σi X’εi ;-½Σ(∂h/∂α)zii +½Σi-4 εi2(∂h/∂α)zi] I(θ) = E[- ∂2log L/∂θ∂θ’] • We have block diagonality, we can rewrite the LM test, under H0: LM = S(α0,0)’ [I22 – I21 I11 I21]-1 S(α0,0) 9 Testing for Heteroscedasticity: BP Test • We have block diagonality, we can rewrite the LM test, under H0: LM = S(α0,0)’ [I22 – I21 I11 I21]-1 S(α0,0) S(α0,0) = -½ Σi (∂h/∂α|α0,R,0)z′  = ½ 𝑒 / = ½ 𝑒 / + ½ Σi  𝑒 (∂h/∂α |α0,R,0)z′ (∂h/∂α |α0,R,0) ∑ 𝒛 (𝑒 / – 1) (∂h/∂α |α0,R,0) ∑ 𝒛 𝑤 𝑤 = 𝑒 / – 1 I22(α0,0) = E[- ∂2log L/∂ α∂ α’] = ½ [ (∂h/∂α |α0,R,0 )]2∑ I21(α0,0) = 0  = (1/T)∑ 𝑒 (MLE of under H0). 𝒛𝒛′ Then, LM = ½ (∑ 𝒛 𝑤 )′ [∑ 𝒛 𝒛 ′]-1 (∑ = ½ W′Z (Z′Z)-1 Z′W ~ χ 𝒛𝑤) Note: Recall R2 = [y′X (X′X)-1 X′y – T 𝑦 ]/[y′y – T 𝑦 ] = ESS/TSS Testing for Heteroscedasticity: BP Test • Since under H0: E[𝑤 ] = 0, E[ωi2] = 1. LM = ½ W′Z (Z′Z)-1 Z′W = ½ ESS ESS = Explained SS in regression of 𝑤 (= 𝑒 / – 1) against zi. • Under the usual regularity conditions, and under H0, √T (αML – α) N(0, 2 4 (Z′Z/T)-1) Then, LM-BP= (2  )-1 ESSe χ . ESSe = ESS in regression of 𝑒 against 𝒛 . Since  4  LM-BP χ 10 Testing for Heteroscedasticity: BP Test Note: Recall R2= [y′X (X′X)-1 X′y – T𝑦 ]/[y′y – T 𝑦 ] Under H0: E[ωi]=0, E[ωi2]=1, the LM test is equivalent to a T R2. (Think of 𝑦 0 & 𝒚′𝒚/T=1 above). • Variations: (1) Glesjer (1969) test. Use absolute values instead of 𝑒 to estimate the varying second moment. Following our previous example, |𝑒 |= α0 + α 𝑧 , + α 𝑧 , + .... + α 𝑧 , + 𝑣 (2) Harvey-Godfrey (1978) test. Use ln(𝑒 ). Then, the implied model for  is an exponential model. ln(𝑒 ) = α0 + α 𝑧 , + α 𝑧 , + .... + α 𝑧 , + 𝑣 Note: Implied model for  = exp{α0 + α 𝑧 , +.... + α 𝑧 , + 𝑣 }. Testing for Heteroscedasticity: BP Test • Variations: (3) Koenker’s (1981) studentized LM test. A usual problem with statistic LM is that it crucially depends on the assumption that ε is normal. Koenker (1981) proposed studentizing the statistic LM-BP by LM-S = (2  ) LM-BP/[(∑ (ε –  ))2/T] χ The studentized version of the test is asymptotically equivalent to a T*R2 test, where R2 is calculated from a regression of 𝑒 / on the variables Z. (Omitting  from the denominator is OK.) 11 Testing for Heteroscedasticity: BP Test • We have the following steps: - Step 1. Run OLS on DGP: y = X  + . – Keep 𝑒 and compute  = RSS/T - Step 2. (Auxiliary Regression). Run the regression of ei2 on the m explanatory variables, z. In our example, 𝑒 / = α + α 𝑧 , + .... + α 𝑧 , +𝑣 – Keep R2. - Step 3. Use the R2 from Step 2. Let’s call it 𝑅 . Calculate LM = T * 𝑅 χ . Testing for Heteroscedasticity: Example – IBM Example: We suspect that squared Mkt_RF (x1) –a measure of the overall market’s variance- drives heteroscedasticity. We do a studentized LM-BP test for IBM in the 3-factor FF model: fit_ibm_ff3 <- lm (ibm_x ~ Mkt_RF + SMB + HML) # Step 1 – OLS in DGP (3-factor FF model) e_ibm <- fit_ibm_ff3$residuals # Step 1 – keep residuals e_ibm2 <- e_ibm^2 # Step 1 – squared residuals Mkt_RF2 <- Mkt_RF^2 fit <- lm (e_ibm2 ~ Mkt_RF2) # Step 2 – Auxiliary regression Re_2 <- summary(fit_BP)$r.squared # Step 2 – keep R^2 LM_BP_test <- Re2 * T > LM_BP_test # Step 3 – Compute LM-BP test: R^2 * T [1] 0.25038 > p_val <- 1 - pchisq(LM_BP_test, df = 1) # p-value of LM_test > p_val [1] 0.6168019 LM-BP Test: 0.25028  cannot reject H0 at 5% level (χ2[1],.05≈3.84); 27 with a p-value= .6168. 12