Time Series Stationarity, AR(P) & Ma(Q) 3

RS – EC2 - Lecture 13 AR Process – Bootstrap • So far, we constructed the bootstrap sample by randomly resampling from the data values (yt,xt). This created an i.i.d bootstrap sample. • This is inappropriate for time-series, since we have dependence. • There are two popular methods to bootstrap time series. (1) Model-Based (Parametric) Bootstrap (2) Block Resampling Bootstrap AR Process – Bootstrap (1) Model-Based (Parametric) Bootstrap 1. Estimate b and residuals e: 2. Fix an initial condition {yt-k+1, yt-k+2 , yt-k+3, ..., y0} 3. Simulate i.i.d. draws e* from the empirical distribution of the residuals {e1, e2 , e3, ..., eT}. 4. Create the bootstrap series yt by the recursive formula y t *  ˆ  ˆ 1 y t 1 *  ˆ 2 y t  2 * ....  ˆ p y t  p *   t * Pros: Simple. Similar to the usual bootstrap. Cons: This construction imposes homoskedasticity on the errors e* ; which may be different than the properties of the actual e. It also imposes the AR(p) as the DGP. 21 RS – EC2 - Lecture 13 AR Process – Bootstrap (2) Block Resampling 1. Divide the sample into T/m blocks of length m. 2. Resample complete blocks. For each simulated sample, draw T/m blocks. 3. Paste the blocks together to create the bootstrap time-series yt*. Pros: It allows for arbitrary stationary serial correlation, heteroskedasticity, and for model misspecification. Cons: It may be sensitive to the block length, and the way that the data are partitioned into blocks. May not work well in small samples. Moving Average Process • An MA process models E[yt|Ft-1] with lagged error terms. An MA(q) model involves q lags. • We keep the white noise assumption for εt. Example: A linear MA(q) model: q yt     t  1 t 1   2 t  2  ...   q t  q      i t i   t i 1 q yt      i Li t   t     ( L) t i 1  ( L)  1  1 L   2 L2  ...   q Lq • Q: Is yt stationary? Check the moments. 22 RS – EC2 - Lecture 13 Moving Average Process – Stationarity • Q: Is yt stationary? Check the moments. WLOG, assume μ=0. E[ yt ]  0 Var [ yt ]  (1  12   22  ...   q2 ) 2  (t  j )  E[ yt yt  j ]  E[( 1 t 1 yt  j   2 t  2 yt  j  ....   q t  q yt  j   t yt  j )]   2 [  j 1 j j |k | ] q | k | q; otherwise  (t  j )  0. • It is easy to verify that the sums are finite  MA(q) is stationary. • Note that an MA(q) process can generate an AR process. (L)1 yt  *t yt    (L)t • We have an infinite sum polynomial on θL. That is, an AR(∞).   j 0  j ( L) yt   * t MA Process – Invertibility • We need to make sure that θ(L)-1 is defined: We require θ(L) ≠ 0. When this condition is met, we can write εt as a causal function of yt. We say the MA is invertible. For this to hold, we require:   j0 |  j (L) |   Definition: A linear process {yt} is invertible strictly speaking, an invertible function of {εt}, if there is a  ( L )  1   1 L   2 L 2  ... with  with  t  (L) yt .  j0 |  j (L) |   23 RS – EC2 - Lecture 13 MA Process – Example: MA(1) • Example: MA(1) process: y t     ( L ) t  ( L )  1  1 L - Moments E Yt     0  Var  yt    2  12  2 1  E[ yt , yt 1 ]  1 2  k  E[ yt , yt  k ]  0, k 1 Note: The autocovariance function is zero after lag 1. – Invertibility: If |θ1|<1, we can write (1+ θ1 L)-1 yt + μ* = εt (1  1 L  1 2 L2  ...  1 j L j  ...) y t  *   *      ( L) y i t  t i 1 MA Process – Example: MA(2) • Example: MA(2) process: y t     ( L ) t  ( L )  1  1 L   2 L2 - Moments E Yt       2 1  12   22 ,  2   1 1   2 , k     2 2 ,   0,  k 0 k 1 k 2 k 2 Note: the autocovariance function is zero after lag 2. 24 RS – EC2 - Lecture 13 MA Process – Example: MA(2) - Inveritbility: The roots of 2  1   2  0 all lie inside the unit circle. It can be shown the invertibility condition for MA(2) process is: 1   2  1  2  1  1 1  2  1 MA Process – Estimation • MA are more complicated to estimate. In particular, there are nonlinearities. Consider an MA(1): yt = εt + θ εt-1 The auto-correlation is ρ1 = θ/(1+θ2). Then, MM estimate of θ satisfies: 2 ˆ ˆ  1  1  4 r1 r1    2 r1 (1  ˆ 2 ) • A nonlinear solution and difficult to solve. • Alternatively, if |θ|< 1, we can try a ∈(-1; 1),  t ( a )  yt  a yt 1  a 2 yt  2  ... and look (numerically) for the least-square estimator ˆ  arg a min{ S T ( a )   t 1  t 2 ( a )} T 25 RS – EC2 - Lecture 13 The Wold Decomposition Theorem - Wold (1938). Any covariance stationary {yt} has infinite order, moving-average representation: y t   j  0 j L j  t   t ,   0  1, where  t : deterministic term (perfectly forecastable). Say,  t      j2   j 0  t ~ WN (0,  2 ) • yt is a linear combination of innovations over time. • A stationary process can be represented as an MA(∞) plus a deterministic “trend.” The Wold Decomposition Example: Let xt= yt – κt. Then, check moments: E[ xt ]  E[ yt   t ]  E[ xt 2 ]    j 0   j 0  j E[ t  j ]  0.  2j E[ t2 j ]   2   j 0  2j  . E[ xt , xt  j ]  E[( t  1  t 1   2  t  2  ...)( t  j  1  t  j 1   2  t  j  2  ...)   2 ( j  1  j 1   2  j  2  ...)   2   k 0  k  k 2 Xt is a covariance stationary process. 26 RS – EC2 - Lecture 13 ARMA Process • A combination of AR(p) and MA(q) processes produces an ARMA(p, q) process: yt    1 yt 1  2 yt  2  ....   p yt  p   t  1 t 1   2 t  2  ...   q t  q p q i 1 i 1     i yt i    i Li t   t   ( L ) yt     ( L ) t • Usually, we insist that (L)≠0, θ(L)≠0 & that the polynomials (L), θ(L) have no common factors. This implies it is not a lower order ARMA model. ARMA Process Example: Common factors. Suppose we have the following ARMA(2,3) model ( L) yt  ( L) t with ( L)  1  .6 L  .3L2 ( L)  1  1.4 L  .9 L2  .3L3  (1  .6 L  .3L2 )(1  L) This model simplifies to: yt  (1 L)t • Pure AR Representation:  an MA(1) process.  L  y t     a t   L   • Pure MA Representation:  yt      L at   L   • Special ARMA(p,q) cases:  p L   q L   q L   p L  – p = 0: MA(q) – q = 0: AR(p). 27 RS – EC2 - Lecture 13 ARMA: Stationarity, Causality and Invertibility Theorem: If (L) and θ(L) have no common factors, a (unique) stationary solution to ( L) yt  ( L) t exists if and only if | z | 1   ( z )  1  1 z   2 z 2  ...   p z p  0 . This ARMA(p, q) model is causal if and only if | z | 1   ( z )  1  1 z   2 z 2  ...   p z p  0 . This ARMA(p, q) model is invertible if and only if | z | 1   ( z )  1  1 z   2 z 2  ...   p z p  0 . • Note: Real data cannot be exactly modeled using a finite number of parameters. We choose p, q to create a good approximated model. ARMA Process – SDE Representation • Consider the ARMA(p,q) model:  ( L )( yt   )   ( L ) t Let xt  yt   and wt   ( L ) t . Then, x t  1 x t 1   2 x t  2  ....   p x t  p  w t (SDE).  xt is a p-th-order linear stochastic difference equation Example: 1st-order SDE (AR(1)): xt   xt 1   t  Recursive solution (Wold form): x t   t  1 x 1    t  t  i   t  1 x 1  i0   i0 i  ti where x 1 is an initial condition. 28 RS – EC2 - Lecture 13 ARMA Process – Dynamic Multiplier • The dynamic multiplier measurers the effect of εt on subsequent values of xt: That is, the first derivative on the Wold representation: δxt+j/δεt = δxj/δε0 = ψj. For an AR(1) process: δxt+j/δεt = δxj/δε0 =  j. • That is, the dynamic multiplier for any linear SDE depends only on the length of time j, not on time t. ARMA Process – Impulse Response Function • The impulse-response function (IRF) a sequence of dynamic multipliers as a function of time from the one time change in the innovation, εt. • Usually, IRF are represented with a graph, that measures the effect of the innovation, εt, on yt over time: δyt+j/δεt+ δyt+j+1/δεt + δyt+j+21/δεt+...= ψj + ψj+1+ ψj+2+... • Once we estimate the ARMA coefficients, it is easy to draw an IRF. 29 RS – EC2 - Lecture 13 ARMA Process – Addition • Q: We add two ARMA process, what order do we get? • Adding MA processes x t  A ( L ) t z t  C ( L )u t y t  x t  z t  A ( L ) t  C ( L ) u t - Under independence:  y ( j )  E [ y t y t  j ]  E [( x t  z t )( x t  j  z t  j )]  E [( x t x t  j  z t z t  j )]   x ( j )   z ( j ) - Then, γ(j) = 0 for j > Max(qx, qz,)  yt is ARMA(0, max(qx, qz)) - Implication: MA(2) + MA(1) = MA(2) ARMA Process – Addition • Q: We add two ARMA process, what order do we get? • Adding AR processes (1  A ( L )) x t   t (1  C ( L )) z t  u t y t  xt  z t  ? - Rewrite system as: (1  C( L))(1  A( L))xt  (1  C(L)) t (1  A(L))(1  C( L))zt  (1  A( L))ut (1  A(L))(1  C( L)) yt  (1  C(L)) t  (1  A( L))ut   t  ut  [C(L) t  A(L)ut ] - Then, yt is ARMA(px+pz , max(px, pz)) 30