Complementarity of Forecast Breakdown and Structural Change Tests

“Testing for Changes in Forecasting Performance” a) Comments about the complementarity of forecast breakdown and structural change tests We here expand on a comment made in the introduction about the complementarity of forecast breakdown and structural change tests. Since forecast breakdowns are often associated with changes in the parameters of the forecasting models, one may question the value-added of forecast breakdown tests over simply using a structural change test directly on the forecasting model. We expand upon an interesting case alluded to in the introduction, whereby it is shown that structural change and forecast breakdown tests complement each other; when one test has poor power for some parameter configurations, the other has high power. We consider data generated such that the relevant predictor changes from some to some at date  = , i.e., + ⎧ ⎨ + +  + = ⎩ + +  for  = 1    (A.1) for  =  + 1   −  + However,  is not accessible to the researcher, hence one uses the following misspecified model over the entire sample + =  +   + + for  = 1   −  . (A.2) We first theoretically illustrate how the forecast breakdown and structural change tests behave under this setting. For simplicity, let  ,  and  be independent scalar random variables with ( ) = ( ) =  and ( ) = 0. We also let  ( ) =  2 ,  ( ) =  2 and  ( ) = 2 . For the forecasting tests, assume that the fixed window scheme is applied (shown to be the preferred one in this paper) with an in-sample length smaller than or equal to  so that the coefficient  is consistently estimated. It is easy to show that the expected value of the limit of the quadratic forecasting loss changes from  2 to 2 + 2  2 + 2  2 before and after  =  . Hence, the magnitude of the change is  2  2 +  2  2  (A.3) For the structural change tests, we consider the Wald statistic (divided by  ) assuming, for simplicity, the break date to be known:  −1  = [( −  ) ][  ] The restricted (∼) and unrestricted residuals (ˆ) are then, respectively, ⎧ ⎨  −  ( e − ) − (e  − ) for  = 1   +   e+ = ⎩  + −  e  − ) for  =  + 1   −  +   − (e ⎧ ⎨  −  (̂ − ) − (̂ − ) for  = 1   +  (1) (1) ̂+ =  ⎩  +   −  ̂ − (̂ − ) for  =  + 1   −  +   (2) (2) 1   After some algebra, one can show that  e →  when ( ) = ( ) (otherwise,  e →     e  + [( ) + (1 − )( )]). Also ̂(1)  ̂(2) →   → , ̂ (1) →  and ̂ (2) → 0, where  = lim →∞   , so that the limit of ( − ) is (1−) 2  2 and that of   is  2 + (1 − ) 2  2 . Hence, the limit of the scaled Wald statistic is (1 − ) 2  2 [2 + (1 − ) 2  2 ] (A.4) Comparing (A.3) and (A.4) yields the following insight. When the variance of the unavailable predictor  2 increases, the change in the forecast loss increases and the change in forecast performance is easier to detect. However, a large variance would not help to detect a change in  associated with the predictor  via a structural change test because an increase in  2 lowers the value of test statistic. And vice-versa, when 2 is small, the change in the loss function is reduced while the value of the Wald test for a change is increased. To quantify the implications of this insight, we implement a simple Monte Carlo simulation. We set  =  = 1 and generate  ∼  (0 1) for  = 1   . The variables are generated by  ∼ (0 1) and  ∼ (0  2 ), which are independent of each other. Let  = 150  = 75 and  = 1, for the sake of illustration (the results remain qualitatively the same with other parameter values). For the forecasting tests, we consider the  test with  = 40, the  test with [02 ] ≤  ≤ [08 ], the  test with 0 = [02 ] and ̄ = 05, as well as the   and    tests described in the text. We only consider the fixed window scheme for reason discussed in the text. For the structural break tests, we consider the full-sample regression model (A.2) and test for variations in both ( ). Specifically, we use the sup  test of Andrews (1993) and the  max test of Bai and Perron (1998). The maximum number of breaks for the   max tests is five and the truncation parameter  = 01 is used. The number of replications is 1,000. Table S.1 shows the rejection frequencies when we vary   from 00 to 100. It illustrates a clear complementarity between the power of the forecasting and structural change tests. When  is small, the rejection frequencies of the forecasting tests are small for all tests considered. However, the rejection frequencies of the structural change tests are large because the denominator in (A.4) is small, which leads a large value of test statistic. The simulation results show that this feature continues to hold with more general tests with an unknown break date or with multiple breaks. Table S.1: Rejection frequencies sw 0.0 0.5 1.0 2.5 5.0 7.5 10.0 GR 0.123 0.252 0.564 0.999 1.000 1.000 1.000 SGR 0.253 0.424 0.832 1.000 1.000 1.000 1.000 DSW 0.137 0.189 0.340 0.885 0.966 0.983 0.979 TLSW 0.611 0.507 0.509 0.996 1.000 1.000 1.000 2 TLUD 0.677 0.595 0.592 0.997 1.000 1.000 1.000 SupF 0.995 0.995 0.959 0.483 0.141 0.095 0.073 UD max 0.996 0.995 0.960 0.496 0.171 0.125 0.091 b) Proof of Theorem 1 For the proof, we denote  () by  for simplicity. The Wald test for a constant mean © ª − versus one break at time  =  = 0 +  + b0 c for the series + = is   ( ) =  () − ( ) () ̂ ()  where  () , ( ) () and ̂ () are defined after (1). First, for a given , the restricted SSR is: ³P ´2 P − 2 1  −   −   () = = + = +  − −+1 ³ ´2 P − 2 0 −12 P −   =  = + − = +  − −+1 0 and the unrestricted SSR assuming a break at  =  is given by ³ ´2 P − 2 0 −12 P −  ( ) () =   −  + + = =  −  −  + 1 0 ´2 P − 0 ³ −12 P − 2  + =  −     = − +1 +  − +1  −  0 Hence, ´2 ³ 0 −12 P −   0 = +  − −+1 ³ ´2 0 −12 P −  +  0 + =  −  −  + 1 ³ ´ 2 0 −12 P −  0 +   + = − +1  −   () − ( ) () = − Let  = lim →∞ ( − 0 )0 . Then, using  = 0 + 0 +  − 1 and  = 0 +  + b0 c, we have 0 1 0 = →   − −+1 0 − ( − 0 ) 1− 0 1 0 = →   −  −  + 1 b0 c − ( − 0 ) + 1 − 0 0 1 = →   −  0 − b0 c − 1 1− Because + −1 =  for  =  + 1   −  + 1 and using Assumption 1 −12 P −  −12 P −2 +1 = 0 0 = + =− +1 + P −12 −12 P−  −2 +1 = 0 =0 − + − 0 =0 − + ⇒ Ω12 [ (1) −  ()]  3 −12 0 P − = + −12 = 0 P −2 +1 =− +1 + P −12 −12 P−  −2 +1 = 0 =0 − + − 0 =0 − + ⇒ Ω12 [ () −  ()]  −12 0 P − = − +1 + −12 = 0 P −2 +1 = −2 +2 + P −12 −12 P −2 +1  −2 +1 = 0 =0 − + − 0 =0 − + ⇒ Ω12 [ (1) −  ()]  Combining the above results yields  () − ( ) () ⇒ Ω[− [ (1) −  ()]2 [ () −  ()]2 [ (1) −  ()]2 + + ] 1− − 1−  and, under the null hypothesis, we have ̂ () → Ω. Note that  = lim →∞ ( − 0 )0 ≤ lim →∞ (1 − 0 )0 = ̄, so that  ∈ [0 ̄]. We also have for a trimming parameter ,  ∈ [ +  +   +  + (1 − )] ¸ ∙  − 0 −  ( − 0 ) +  ( − 0 ) + (1 − )  ∈  0 0 0 ¸ ∙  − 0 −  ( − 0 ) + (0 + 0 − ) ( − 0 ) + (1 − )(0 + 0 − )  ∈  0 0 0 Taking the limit implies  ∈ [ + (1 − ) 1 − (1 − )], and the result follows. 4 c) Table S.2: Additional critical values of the  test  = 005 ̄ 10% 5%  = 015 25% 1% ̄ 10% 5% 25% 1% 020 11796 13738 15306 17729 020 9627 025 12237 14085 15788 18148 025 10210 12048 13866 16249 030 12582 14384 16253 18356 030 10374 12100 14004 16448 035 12936 14662 16484 18631 035 10631 12248 14068 16729 040 13498 15356 17265 19328 040 11205 12946 14788 17317 045 13752 15561 17276 19364 045 11293 13104 14952 17478 050 14007 15808 17431 19385 050 11539 13338 15088 17540 055 14359 15985 17795 20052 055 11929 13534 15237 17760 060 14403 16166 17984 20247 060 12202 13887 15536 17626 065 14887 16589 18245 20343 065 12342 14243 15631 17812 070 15245 16873 18412 20772 070 12773 14434 16005 17998 075 15261 16890 18695 20837 075 12801 14509 16130 18159 080 15750 17412 19337 21097 080 13067 15008 17060 19548 5 11368 13037 15432 d) Table S.3: Additional simulation results for the size of the   and    tests for  = 150 a) static, non-robust SW UD fixed rolling recursive fixed rolling recursive c) dynamic, non-robust 10% 0.113 0.118 0.096 0.125 0.126 0.101 5% 0.062 0.064 0.051 0.064 0.072 0.057 1% 0.013 0.015 0.01 0.013 0.016 0.010 SW UD b) static, robust SW UD fixed rolling recursive fixed rolling recursive fixed rolling recursive fixed rolling recursive 10% 0.166 0.146 0.097 0.201 0.177 0.117 5% 0.093 0.083 0.055 0.114 0.096 0.063 1% 0.031 0.023 0.015 0.036 0.027 0.016 10% 0.154 0.146 0.108 0.200 0.194 0.149 5% 0.088 0.089 0.066 0.111 0.109 0.080 1% 0.025 0.028 0.017 0.026 0.035 0.018 d) dynamic, robust 10% 0.125 0.132 0.107 0.150 0.159 0.129 5% 0.069 0.072 0.061 0.081 0.088 0.071 1% 0.018 0.022 0.013 0.020 0.025 0.016 SW UD 6 fixed rolling recursive fixed rolling recursive e) Table S.4: Additional simulation results for the size of the  test for  = 150 a) static, non-robust mu_bar 0.25 0.5 0.75 m0 fixed rolling recursive fixed rolling recursive fixed rolling recursive 0.3T 0.097 0.101 0.089 0.114 0.118 0.113 0.174 0.174 0.164 10% 0.2T 0.086 0.075 0.084 0.121 0.114 0.117 0.165 0.160 0.152 0.1T 0.093 0.051 0.077 0.117 0.101 0.115 0.160 0.151 0.147 0.3T 0.065 0.072 0.063 0.075 0.081 0.078 0.116 0.125 0.123 5% 0.2T 0.048 0.049 0.052 0.071 0.074 0.076 0.112 0.112 0.119 0.1T 0.050 0.034 0.048 0.071 0.065 0.071 0.105 0.104 0.110 0.3T 0.033 0.032 0.031 0.028 0.031 0.029 0.055 0.056 0.052 1% 0.2T 0.025 0.018 0.022 0.026 0.025 0.025 0.049 0.052 0.049 0.1T 0.020 0.012 0.023 0.028 0.021 0.026 0.046 0.049 0.046 0.3T 0.131 0.128 0.124 0.161 0.170 0.167 0.278 0.269 0.270 10% 0.2T 0.122 0.090 0.112 0.160 0.155 0.156 0.254 0.244 0.247 0.1T 0.122 0.059 0.106 0.151 0.131 0.151 0.244 0.222 0.236 0.3T 0.092 0.092 0.089 0.112 0.109 0.111 0.218 0.217 0.218 5% 0.2T 0.083 0.056 0.076 0.107 0.100 0.103 0.193 0.191 0.191 0.1T 0.082 0.037 0.072 0.100 0.083 0.095 0.186 0.177 0.178 0.3T 0.044 0.045 0.044 0.050 0.055 0.053 0.108 0.117 0.109 1% 0.2T 0.035 0.031 0.032 0.041 0.043 0.044 0.096 0.106 0.101 0.1T 0.027 0.017 0.030 0.043 0.036 0.038 0.091 0.100 0.094 0.3T 0.109 0.108 0.103 0.124 0.124 0.121 0.189 0.193 0.192 10% 0.2T 0.099 0.078 0.097 0.137 0.115 0.128 0.183 0.177 0.177 0.1T 0.127 0.065 0.103 0.137 0.106 0.133 0.190 0.165 0.176 0.3T 0.078 0.072 0.070 0.086 0.088 0.090 0.132 0.136 0.131 5% 0.2T 0.071 0.052 0.062 0.086 0.078 0.090 0.131 0.128 0.129 0.1T 0.092 0.041 0.071 0.089 0.071 0.093 0.133 0.122 0.129 0.3T 0.036 0.035 0.034 0.033 0.038 0.033 0.061 0.073 0.061 1% 0.2T 0.032 0.024 0.026 0.033 0.033 0.031 0.058 0.064 0.060 0.1T 0.037 0.015 0.031 0.034 0.027 0.034 0.055 0.058 0.058 0.3T 0.135 0.126 0.132 0.167 0.168 0.176 0.286 0.280 0.282 10% 0.2T 0.124 0.098 0.123 0.169 0.154 0.173 0.261 0.256 0.257 0.1T 0.134 0.072 0.127 0.161 0.135 0.171 0.253 0.234 0.251 0.3T 0.095 0.085 0.088 0.119 0.116 0.119 0.226 0.225 0.234 5% 0.2T 0.081 0.057 0.075 0.115 0.103 0.114 0.203 0.202 0.209 0.1T 0.093 0.042 0.083 0.117 0.084 0.113 0.194 0.184 0.203 0.3T 0.044 0.047 0.043 0.053 0.062 0.055 0.119 0.129 0.119 1% 0.2T 0.037 0.027 0.035 0.049 0.051 0.051 0.108 0.115 0.110 0.1T 0.039 0.016 0.035 0.047 0.043 0.049 0.102 0.106 0.106 b) static, robust mu_bar 0.25 0.5 0.75 m0 fixed rolling recursive fixed rolling recursive fixed rolling recursive c) dynamic, non-robust mu_bar 0.25 0.5 0.75 m0 fixed rolling recursive fixed rolling recursive fixed rolling recursive d) dynamic, robust mu_bar 0.25 0.5 0.75 m0 fixed rolling recursive fixed rolling recursive fixed rolling recursive 7 f) Theoretical results about the limit of the loss function under the various DGPs with a single change We consider the asymptotic behavior of the loss sequence when there is a single coefficient or variance break. Our results pertain to the loss sequence in large samples. These are directly used when constructing the  ,   and    tests. The surprise losses are used when constructing the  and  tests. However, since the surprise losses series subtract the average of the in-sample losses, the dynamics of both sequences are similar. Hence, we consider only the loss sequence, namely the expected values of  lim →∞  = ∗ (), defined on the unit interval  ∈ [0 1] where  = lim →∞ ( ). The squared loss function is applied. For simplicity, and without substantive loss of generality, we consider a single break model generated by: + =    + +  for  = 1   −   (A.5) where   =  1 for  ≤ [ 0 ] and   =  2 for   [ 0 ]. Again for simplicity, the predictor  is a scalar that satisfies (2 ) =  2 and ( − ) =   . We also assume that  is a white noise with mean 0 and variance  2 , where  2 =  21 for  ≤ [ 0 ] and  2 =  22 for   [ 0 ]. Let the in-sample length be  = [ ] with  chosen so that  ≤ 0 . In the following, we consider the following two cases: a coefficient change, i.e.,  1 = 0 and  2 = ∆ with  21 =  22 =  2 ; a variance change, i.e.,  21 =  2 and  22 =  2 + ∆2 with  1 =  2 = . Suppose we use the static regression model of + on  to produce a  -period ahead forecast at time . For the out-of-sample procedure, we use the estimate of the coefficient  obtained from the in-sample information given in  ∈ [1  ], labelled as ̂ [1] . We consider the three window schemes and estimate the coefficient using OLS for the sample period [1  −  ] with the fixed scheme, [ −  + 1  −  ] with the rolling scheme, and [1  −  ] with the recursive scheme. Hence, ̂ [1] with the same  can be different depending on the window scheme. When the static regression is used, the expected value of ∗ () is such that i h (A.6)  [∗ ()] =   lim (+ −  ̂ [1] )2 = 2 +  2 (  −  ∗[0] )2   →∞ where we denote  ∗[0] =  lim →∞ ̂ [1] . We also denote the true value of   and  2 by   and  2 . Next, we consider a dynamic regression, i.e., the regression of + on  and  . Including  as a predictor while the true model is (A.5) is inconsequential under the null hypothesis, because the true value for  is zero. Things are quite different when instabilities are present. When the dynamic model is used, with ∗[0] =  lim →∞ ̂[1] , the expected value of ∗ () is: i h  [∗ ()] =   lim (+ − ̂[1]  −  ̂ [1] )2  = (1 +  →∞ ∗2 ∗2 [0] )  +  2 [(1 − ∗[0] )  −  ∗[0] ]2  (A.7) The case of a coefficient change. Consider the case of coefficient change, i.e.,  1 6=  2 with  21 =  22 =  2 . For the static model, we obtain the following results for the limit of the 8 coefficient estimates. With the fixed scheme,  ∗[0] =  1 , for 0 ≤  ≤ 1. With the rolling scheme, ⎧ ⎪ ⎪  for 0 ≤  ≤ 0  ⎪ ⎨ 1 0 −+ 0  ∗[0] =  1 + − 2 for 0   ≤ 0 +  ,   ⎪ ⎪ ⎪ ⎩  for 0 +    ≤ 1 2 With the recursive scheme,  ∗[0] ⎧ ⎨  1 = ⎩ 0  +  1 −0 2  for 0 ≤  ≤ 0  for 0   ≤ 1 Note that the coefficient estimate is biased when the rolling and the recursive schemes are used. When the dynamic model is used, the results are more complex, so we focus on the case with  1 = 0 and  2 = ∆ , without loss of generality With the fixed scheme, ⎡ ⎤ ⎡ ⎤−1 ⎡ ⎤ P− 2 P− P− −1 −1 −1  ̂   =1  =1   ⎦ =1  + ⎦ ⎣ [1] ⎦ = ⎣ ⎣  P P P − − 2 −1 ̂ [1] −1 =1   −1 −      +  =1 =1 P P P P     2 →  2  −1 −   → 0 −1 − 2 →  2  −1 −  + → 0 Using −1 − =1 =1 =1 =1 P  and −1 − =1  + → 0, ⎤ ⎡ ⎤−1 ⎡ ⎤ ⎡ ⎤ ⎡ ∗ 2 0 0   0 ⎦ ⎣ ⎦ = ⎣ ⎦ for 0 ≤  ≤ 1 ⎣ [0] ⎦ = ⎣ 0  2  ∗[0] 0 0 For the rolling scheme, let  = ( − 0 ). After some algebra, we obtain: ⎧ ⎡ ⎤ ⎪ 0 ⎪ ⎪ ⎣ ⎦ ⎪ for 0 ≤  ≤ 0  ⎪ ⎪ ⎪ ⎪ 0 ⎪ ⎡ 2  (1−)∆2 ⎤ ⎤ ⎪ ⎡ ⎪ ⎪    ⎨ ∗[0]  2 2 + 2 (1−)∆2 ⎣ ⎦= ⎦ for 0   ≤ 0 +  ⎣   2 ∆ ∗ ⎪ ⎪  [0] ⎪ 2 2 +2 (1−)∆2 ⎪ ⎡ ⎤ ⎪ ⎪ ⎪ ⎪ 0 ⎪ ⎪ ⎣ ⎦ ⎪ for 0 +    ≤ 1 ⎪ ⎩ ∆ For the recursive scheme, let  = ( − 0 ). After some algebra, we obtain: ⎧ ⎡ ⎤ ⎪ ⎪ ⎣0⎦ ⎪ ⎡ ⎤ ⎪ for 0 ≤  ≤ 0  ⎪ ⎪ ∗ ⎨ 0  ⎣ [0] ⎦ = ⎡ 2  (1−)∆2 ⎤    ∗ ⎪ ⎪  [0] 2  2 + 2 (1−)∆2  ⎪   ⎦ ⎪ ⎣ for 0   ≤ 1 ⎪ ⎪    2 ∆ ⎩ 2  2 +2 (1−)∆2 9 As in (A.6), when the static model is used,  [∗ ()] =  2 + 2 Γ(), where Γ() = (  − ∗[0] )2 . Therefore, i) with the fixed scheme, ⎧ ⎨ 0 for 0   ≤   0 Γ() = ⎩ ∆2 for    ≤ 1 0  ii) With the rolling scheme, ⎧ ⎪ ⎪ 0 ⎪ ⎨ ¡ 0 −+ Γ() = 1 +  ⎪ ⎪ ⎪ ⎩ 0 −0 2  iii) With the recursive scheme, ⎧ ⎨ 0 Γ() = ⎩ ¡ 0  +  1 − 2 −0 2  ¢2 = (0 −+)2 2 ∆ 2 for 0 ≤  ≤ 0  for 0   ≤ 0 +  for 0 +    ≤ 1 − 2 ¢2 = 20 2 ∆2 for 0 ≤  ≤ 0  for 0   ≤ 1 When the dynamic model is used, we have: i) with the fixed scheme, ⎧ ⎨ 2 for 0 ≤  ≤ 0  ∗  [ ()] = ⎩  2 +  2 ∆2 for 0   ≤ 1   ii) With the rolling scheme, ⎧ ⎪ ⎪ 2 for 0 ≤  ≤ 0  ⎪ ⎨ ∗  [ ()] =  (  ∆       ) for 0   ≤ 0 +  ⎪ ⎪ ⎪ ⎩ 2 for 0 +    ≤ 1 where  (  ∆       ) ≡  (  ∆       )[2  2 +  2 (1 − )∆2 ]2 , with  (  ∆       ) =  4  6 + [2 2 (1 − ) + ( 2 −   )2 ] 2  4 ∆2 +[ 2 (1 − ) + 2( 2 −   )(1 −   ) + (1 − )](1 − ) 4  2 ∆4 +(1 −   )2 2 (1 − )2  6 ∆4  and iii) with the recursive scheme, ⎧ ⎨ 2 for 0 ≤  ≤ 0  ∗  [ ()] = ⎩  (  ∆     ) for    ≤ 1     0 10 where  (  ∆       ) ≡  (  ∆       )[2  2 +  2 (1 − )∆2 ]2 with  (  ∆       ) =  4  6 + [2 2 (1 − ) + ( 2 −   )2 ] 2  4 ∆2 +[ 2 (1 − ) + 2( 2 −   )(1 −  ) + (1 − )](1 − )4  2 ∆4 +(1 −   )2 2 (1 − )2  6 ∆4  The case of a variance change. We now consider the case of variance change, i.e.,  21 =  2 and  22 =  2 +∆2 with  1 =  2 = . It is relatively easy to derive the limit of the coefficient estimate and of the loss sequence. Under any window schemes, we can show that: i) when the static model is used  ∗[0] = , for 0 ≤  ≤ 1, and when the dynamic model is used, ⎤ ⎡ ⎤ ⎡ ∗  0 ⎣ [0] ⎦ = ⎣ ⎦ for 0 ≤  ≤ 1  ∗[0]  Therefore, under any window schemes, when the static model is used, (A.6) yields  [∗ ()] =  2 +  2 (  −  ∗[0] )2 =  2 ⎧ ⎨ 2 = ⎩ 2 + ∆ 2  for 0 ≤  ≤ 0  for 0   ≤ 1 and when the dynamic model is used, ∗ 2 2 ∗ 2  [∗ ()] = (1 + ∗2 [0] )  +   [(1 − [0] )  −  [0] ] =  2 ⎧ ⎨ 2 = ⎩ 2 + ∆ 2  for 0 ≤  ≤ 0  for 0   ≤ 1 Numerical illustration. We compute the numerical values of [∗ ()] for 0 ≤  ≤ 1. We set  2 = 1,  = 01 and 0 = 05 and a small break (∆  ∆2 = 1) and a large break (∆  ∆2 = 5) to investigate how the magnitude of the break affects the shape of the loss sequence. Figures 1 and 2 present [∗ ()] for the coefficient change case for the static and dynamic regressions, respectively. The upper (lower) panels report the case of a small (large) break. When the static model is used (Figure 1), we see a stepwise change in the loss sequences when the fixed scheme is used. However, the same change in coefficient translates into a spiked shape with the rolling scheme and a triangular shape with the recursive scheme. This explains why the fixed scheme is to be preferred and why using the rolling or recursive scheme induces a loss of power. Note, however, that the break magnitude only changes the height of the change in the loss sequence, not the shapes of the loss sequences. Hence, increasing the magnitude of the break size should still increase power under all schemes. 11 Things are different when considering the dynamic model, We still have the same general shapes for the loss sequences. However, the spikes and the triangular shape for the rolling and the recursive schemes become more narrow and closer to an outlier as the break becomes larger. This explains why, when using a dynamic model, the use of a rolling or recursive scheme leads to a non-monotonic power function, i.e., the power decreases as the break magnitude increases. Figures S.1 and S.2 present [∗ ()] for the variance change case when the static and dynamic regressions are used, respectively. Here, the results are simple. The limit of the loss sequence always yields a stepwise change, whose magnitude depends on the change in variance. Hence, all tests should have similar power functions that are monotonically increasing in the magnitude of the variance break. 12 Figure S.1. Shapes of [∗ ()] for a variance break in a static regression a) Small break (∆2 = 1): fixed 2.5 rolling 2.5 2 2 2 1.5 1.5 1.5 1 1 0 0.2 0.4 0.6 0.8 1 0 0.2 recursive 2.5 0.4 0.6 0.8 1 1 0 0.2 0.4 0.6 0.8 1 0.8 1 b) Large break (∆2 = 5): fixed 7 rolling 7 6 6 6 5 5 5 4 4 4 3 3 3 2 2 2 1 1 0 0.2 0.4 0.6 0.8 1 recursive 7 1 0 0.2 0.4 0.6 0.8 0 1 0.2 0.4 0.6 Figure S.2. Shapes of [∗ ()] for a variance break in a dynamic regression a) Small break (∆2 = 1): fixed 2.5 rolling 2.5 2 2 2 1.5 1.5 1.5 1 1 0 0.2 0.4 0.6 0.8 1 0 0.2 recursive 2.5 0.4 0.6 0.8 1 1 0 0.2 0.4 0.6 0.8 1 0.8 1 b) Large break (∆2 = 5): fixed 7 rolling 7 6 6 6 5 5 5 4 4 4 3 3 3 2 2 2 1 0 0.2 0.4 0.6 0.8 1 recursive 7 1 1 0 0.2 0.4 13 0.6 0.8 1 0 0.2 0.4 0.6 g) Theoretical results for the iterated forecast method We assess the properties of the loss sequence when the iterated forecasting method is used. We consider the case of a coefficient change with no extra predictors  , i.e., an unconditional forecast, to avoid the potential complication of forecasting  . The data are generated by  =   +   for  = 1   where ⎧ ⎨ 0 for  ≤ [ 0 ]  = ⎩ ∆ for   [  ]  0 and  is a white noise with mean 0 and variance  2 . Suppose we use an AR1 model, in which the true value of  is zero for simplicity and without loss of generality, so that the regression is: +1 =  +  +  After we obtain the OLS coefficient estimate ̂[1] and ̂ [1] , the iterative method constructs a  −period ahead forecast as follows ̂+ = ̂ [1] + ̂[1] + −1  where ̂[1] ≡ becomes P =0 = ̂ [1] (1 + ̂[1] ) + ̂2[1] + −2  .. . = ̂ [1] ̂[1] + ̂[1]   ̂[1] . Because  is asymptotically small, the expected value of ∗ () ∗ h  [ ()] =   lim (̂+ − ̂ [1] ̂[1] +  →∞ ∗2 ∗2 [0] )  ̂[1]  )2 i  ∗ (A.8) ∗ 2 + [(1 − ∗ [0] )  −  [0] ̂[0] ]  P ∗ where ∗[0] =  lim →∞ ̂[1] ,  ∗[0] =  lim →∞ ̂ [1] and ̂[0] =  lim →∞ =0 ∗ [0] . Note ∗ that we now have an expression for  [ ()] influenced by  , which did not appear in the direct forecast counterparts; see (A.7). Hence, it is interesting to assess the effects of  . The limit of the coefficient estimate is the same as that for the dynamic regression model with  = 1 for all  since  is fixed and, hence, small in large samples. With the fixed scheme, ⎡ ⎤ ⎡ ⎤ P−1 2 P−1 ⎤−1 ⎡ −1 P−1 −1 −1 ̂   =1   =1  ⎦ =1  + ⎦ ⎣ [1] ⎦ = ⎣ ⎣  P P P −1 −1 −1 −1 ̂ [1] −1 −1   1   =1  =1 =1 + P P P P     2 →  2 , −1 −1  → 0, −1 −1 1 → 1, −1 −1 We have −1 −1 =1 =1 =1 =1  +1 → 0 P  and −1 −1 =1 +1 → 0 so that ⎤ ⎡ ⎤−1 ⎡ ⎤ ⎡ ⎤ ⎡ 2 0 0 0 ∗[0] ⎦=⎣ ⎦ ⎣ ⎦ = ⎣ ⎦ for 0 ≤  ≤ 1 ⎣ 0  2  ∗[0] 0 0 = (1 + 14 With the rolling scheme, we let  = ( − 0 ) and obtain ⎧ ⎡ ⎤ ⎪ 0 ⎪ ⎪ ⎣ ⎦ ⎪ for 0 ≤  ≤ 0  ⎪ ⎪ ⎪ 0 ⎪ ⎪ ⎡ (1−)∆2 ⎤ ⎡ ⎤ ⎪ ⎪ ⎪  ∗ ⎨ [0] 2 +(1−)∆2 ⎣ ⎣ ⎦= ⎦ for 0   ≤ 0 +  2 ∆ ⎪ ⎪  ∗[0] 2 2 ⎪ ⎪  ⎡  +(1−)∆ ⎤ ⎪ ⎪ ⎪ ⎪ 0 ⎪ ⎪ ⎣ ⎦ ⎪ for 0 +    ≤ 1 ⎪ ⎩ ∆ With the recursive scheme, we let  = ( − 0 ) and obtain ⎧ ⎡ ⎤ ⎪ ⎪ ⎪ ⎣0⎦ ⎤ ⎪ ⎡ for 0 ≤  ≤ 0  ⎪ ⎪ ⎨ 0 ∗[0] ⎦= ⎣ ⎡ (1−)∆2 ⎤  ∗ ⎪ ⎪  [0]  2 +(1−)∆2 ⎪ ⎪ ⎣ ⎦ for 0   ≤ 1 ⎪ ⎪ 2 ∆ ⎩  2 +(1−)∆2 We can then obtain  [∗ ()] by plugging ∗[0] and  ∗[0] into (A.8). With the fixed scheme, ⎧ ⎨ 2  [∗ ()] = ⎩  2 + ∆2 for 0 ≤  ≤ 0  for 0   ≤ 1  With the rolling scheme,  [∗ ()] = ⎧ ⎪ ⎪ 2 ⎪ ⎨ for 0 ≤  ≤ 0   (  ∆ ) for 0   ≤ 0 +  ⎪ ⎪ ⎪ ⎩ 2 for 0 +    ≤ 1 ∗ ∗ 2 ∗ 2 ∗ where  (  ∆ ) ≡ (1 + ∗2 [0] ) + [(1 − [0] )∆ −  [0] ̂[0] ] with [0] = [(1 − )∆2 ][2 + (1 − )∆2 ] and  ∗[0] = [ 2 ∆ ][ 2 + (1 − )∆2 ]. With the recursive scheme, ⎧ ⎨ 2 for 0 ≤  ≤ 0   [∗ ()] = ⎩  (  ∆ ) for    ≤ 1   0 ∗ ∗ 2 ∗ 2 ∗ where  (  ∆ ) ≡ (1 + ∗2 [0] ) + [(1 − [0] )∆ −  [0] ̂[0] ] with [0] = [(1 − )∆2 ][2 + (1 − )∆2 ] and  ∗[0] = [ 2 ∆ ][ 2 + (1 − )∆2 ]. Since the expressions 15 above are not quite intuitive, we compute the numerical values of [∗ ()] for 0 ≤  ≤ 1. In particular, we set the parameter values  2 = 1,  = 01 and 0 = 05. We consider the forecasting horizons  = 3 and  = 12. For each value of  , we consider a small break (∆ = 1) and a large break (∆ = 5) to investigate how the magnitude of the break affects the shape of the loss sequence. Figures S.3 presents [∗ ()] for the coefficient change case when  = 3. The upper panel reports the case of a small break and the lower panel that of a large break. The pattern of the loss sequence is very similar to what was obtained using the direct forecasting (dynamic regression) in that the change in the coefficient translates into a spike when the rolling or the recursive schemes are used. For the direct forecast method,  does not appear in the results as it is assumed small relative to  . Figure S.4 presents [∗ ()] for the coefficient change case when  = 12 and the results are very similar to those of  = 3, except that the effect of increasing the break magnitude on the shapes of [∗ ()] for the rolling and the recursive schemes is somewhat weaker when  is larger. Otherwise, all qualitative results reported in the text for the “direct forecast method” continue to hold if one uses the “indirect forecast method”. 16 Figure S.3. Shapes of [∗ ()]: Coefficient break with iterated forecasts ( = 3) a) Small break (∆ = 1) fixed 2.2 rolling 2.2 2 2 2 1.8 1.8 1.8 1.6 1.6 1.6 1.4 1.4 1.4 1.2 1.2 1.2 1 1 0 0.2 0.4 0.6 0.8 1 recursive 2.2 1 0 0.2 0.4 0.6 0.8 0 1 0.2 0.4 0.6 0.8 1 0.8 1 b) Large break (∆ = 5) fixed 31 rolling 31 26 26 26 21 21 21 16 16 16 11 11 11 6 6 6 1 1 0 0.2 0.4 0.6 0.8 1 recursive 31 1 0 0.2 0.4 0.6 0.8 0 1 0.2 0.4 0.6 Figure S.4. Shapes of [∗ ()]: Coefficient break with iterated forecasts ( = 12) a) Small break (∆ = 1) fixed 2.2 rolling 2.2 2 2 2 1.8 1.8 1.8 1.6 1.6 1.6 1.4 1.4 1.4 1.2 1.2 1.2 1 1 0 0.2 0.4 0.6 0.8 1 recursive 2.2 1 0 0.2 0.4 0.6 0.8 0 1 0.2 0.4 0.6 0.8 1 0.8 1 b) Large break (∆ = 5) fixed 31 rolling 31 26 26 26 21 21 21 16 16 16 11 11 11 6 6 6 1 1 1 0 0.2 0.4 0.6 0.8 1 recursive 31 0 0.2 0.4 17 0.6 0.8 1 0 0.2 0.4 0.6 h) A “typical” realization of the loss sequences Figures S.5 and S.6 present a “typical” realization of the loss sequences for DGPs 3-5 under the three forecasting schemes for the tests 2,  2 and   (the results using 1,  1 and   are, respectively, almost equivalent and, hence, omitted). Because the loss sequence is generated for every , we present the one for which the test statistic is maximized, say ∗ , whose value is indicated in parenthesis above each path. Figure S.5: A realization of loss sequences: static model SGR2 DGP3 fixed (75) rolling (45) recursive (75) 60 60 60 50 50 50 40 40 40 30 30 30 20 20 20 10 10 10 0 0 DGP4 fixed (50) 0 rolling (45) recursive (50) 250 250 250 200 200 200 150 150 150 100 100 100 50 50 50 0 DGP5 0 fixed (74) rolling (75) 600 600 400 400 200 200 0 0 0 600 500 400 300 200 100 0 18 recursive (74) DSW2 DGP3 fixed (30) rolling (75) recursive (30) 60 60 60 50 50 50 40 40 40 30 30 30 20 20 20 10 10 10 0 0 0 DGP4 fixed (50) rolling (50) 250 250 250 200 200 200 150 150 150 100 100 100 50 50 50 0 DGP5 0 0 fixed (41) recursive (50) rolling (57) recursive (55) 600 600 600 500 500 500 400 400 400 300 300 300 200 200 200 100 100 100 0 0 0 19 TLSW DGP3 fixed (37) rolling (90) recursive (73) 60 60 60 40 40 40 20 20 20 0 0 DGP4 250 fixed (43) 250 0 rolling (75) 250 200 200 200 150 150 150 100 100 100 50 50 50 0 0 DGP5 fixed (72) recursive (56) 0 rolling (87) recursive (79) 600 600 600 400 400 400 200 200 200 0 0 0 20 Figure S.6: A realization of loss sequences: dynamic model SGR2 DGP3 fixed (75) rolling (64) recursive (64) 60 60 60 40 40 40 20 20 20 0 0 DGP4 fixed (50) 0 rolling (45) recursive (49) 250 250 250 200 200 200 150 150 150 100 100 100 50 50 50 0 0 DGP5 fixed (77) 0 rolling (77) recursive (77) 600 600 600 400 400 400 200 200 200 0 0 0 21 DSW2 DGP3 fixed (30) rolling (75) recursive (75) 60 60 60 50 50 50 40 40 40 30 30 30 20 20 20 10 10 10 0 0 DGP4 fixed (51) 0 rolling (46) recursive (48) 250 250 250 200 200 200 150 150 150 100 100 100 50 50 50 0 0 DGP5 fixed (37) 0 rolling (69) recursive (36) 600 600 500 500 500 400 400 400 300 300 200 200 100 100 100 0 0 0 300 200 22 TLSW DGP3 fixed (33) rolling (76) recursive (76) 60 60 60 50 50 50 40 40 40 30 30 30 20 20 20 10 10 10 0 0 DGP4 fixed (51) 0 rolling (48) recursive (49) 250 250 250 200 200 200 150 150 150 100 100 100 50 50 50 0 DGP5 0 0 fixed (35) rolling (38) recursive (38) 600 600 600 500 500 500 400 400 400 300 300 300 200 200 200 100 100 100 0 0 0 23