Lecture Notes 1: Stationarity, Lag Operator, ARMA, and Covariance Structure

Introduction 1 14.384 Time Series Analysis, Fall 2007 Professor Anna Mikusheva Paul Schrimpf, scribe September 6, 2007 Lecture 1 revised on September 9, 2009 Stationarity, Lag Operator, ARMA, and Covariance Structure Introduction History – popular in early 90s, making comeback now. The main difference between time series econometrics and cross-section is in dependence structure. Crosssection econometrics mainly deals with i.i.d. observations, while in time series each new arriving observation is stochastically depending on the previously observed. The dependence is our best friend and a great enemy. On one side, the dependence screw up your inferences: the regular (ordinary) Central Limit Theorem should be corrected to hold for dependent observations. That bring us to the task of correcting our procedures for dependence. On the other side, the dependence allow us to do more by exploiting it. For example, we can make forecasts (which are almost non-sense in cross-section). Some topics may sounds counter-intuitive for you at first (if you are cross-section minded). For example, if you are working with very persistent time series, your estimates can be severely biased even if the exclusion restriction is satisfied. But time-to-time you can recover a stochastic trend super-consistently even when the exclusion restriction is not exactly satisfied. I personally find it amusing:) Can roughly divide time series into macro and finance related stuff. Macro Time series mostly focuses on means. Macro limited by small number of observations available over long horizon. A typical data set has at best 20 years of monthly or 40 years of quarterly data, which sum up to less than 300 observations. This allows us to study linear relations between variables or model means. Financial data usually high-frequency over short period of time. This allows us to model volatility and higher moments. Outline Can divide course into two main parts: 1. Classics Univariate Multivariate stationary ARMA VARMA nonstationary unit root cointegration 2. DSGE • simulated GMM • ML • Bayesian Goals The main objective of this course is to develop the skills needed to do empirical research in fields operating with time series data sets. The course aims to provide students with techniques and receipts for estimation and assessment of quality of economic models with time series data. Special attention will be placed on limitations and pitfalls of different methods and their potential fixes. The course will also emphasize recent developments in Time Series Analysis and will present some open questions and areas of ongoing research. Problem Sets 2 Problem Sets Will have an empirical part – requires programming. Use whatever language you prefer. I recommend Matlab and discourage Stata. There will be a session devoted to Intro to MatLab. You need not write your programs from scratch. You can freely download programs from the web, but make sure you use them correctly and cite them. Working in groups is encouraged, but you should write your own solutions. The final exam will be in a take-home format. ARMA Processes Stationarity We need what we have observed to be stable, in some sense, so that we can make inferences and statements about the future. Definition 1. White noise is a sequence of random variables {et } such that Eet = 0, Eet es = 0, Ee2t = σ 2 Definition 2. A process, {yt }, is strictly stationary if for each k, t and n, the distribution of {yt , ..., yt+k } is the same as the distribution of {yt+n , ..., yt+k+n } Definition 3. {yt }, is 2nd order stationary(or weakly stationary, or simply stationary) if Eyt , Eyt2 , and cov(yt , yt+k ) do not depend on t Examples of non-stationary Example 4. Break: ( yt = β + et β + λ + et t≤k t>k Example 5. Random Walk (also known as unit root process) yt = yt−1 + et One can notice that V ar(yt ) = V ar(yt−1 ) + V ar(et ) > V ar(yt−1 ) Lag operator and operations with it Definition 6. Lag operator Denoted L. Lyt = yt−1 . 1) The lag operator can be raised to powers, e.g. L2 yt = yt−2 . We can also form polynomials of it a(L) = a0 + a1 L + a2 L2 + ... + ap Lp a(L)yt = a0 yt + a1 yt−1 + a2 yt−2 + ... + ap yt−p 3) Lag polynomials can be multiplied. Multiplication is commutative, a(L)b(L) = b(L)a(L). 4) Some lag polynomials can be inverted. We define (1 − ρL)−1 by the following equality: (1 − ρL)(1 − ρL)−1 ≡1 The statement is: If |ρ| < 1, then (1 − ρL)−1 = ∞ X i=0 ρi Li . (1) Simple Processes 3 First we need to check that the infinite sum is a valid definition of an operator from weakly stationary sequences (in L2 sense) to weakly stationary sequences. This mean that if xt is weakly stationary, then PJ for all t a sequence ( i=0 ρi Li )xt converges to a limit in L2 space as J → ∞ and the result is a weakly PJ stationary sequence. If |ρ| < 1, one can check that yt,J = ( i=0 ρi Li )xt is a Cauchy sequence in L2 . That is, E(yt,j − yt,J )2 → 0 as min(j, J) → ∞. Given that L2 is a complete space (all Cauchy sequences converge), 2 there exists a limit, call it zt : yt,J →L zt . Then we have to check that zt is weakly stationary(which is trivial). Now we have to check the equality (1): (1 − ρL) ∞ X i i ρL = ∞ X i=0 i i ρL − i=0 ∞ X ρi Li i=1 =ρ0 L0 = 1 For higher order polynomials, we can invert them by factoring, using the formula for (1−ρL)−1 (assuming that the roots are outside the unit circle), and then rearranging, for example: 1 − a1 L − a2 L2 =(1 − λ1 L)(1 − λ2 L) , |λi | < 1 (1 − a1 L − a2 L2 )−1 =(1 − λ1 L)−1 (1 − λ2 L)−1 ∞ ∞ X X =( λi1 Li )( λi2 Li ) i=0 = ∞ X à L j j=0 i=0 j X ! λk1 λj2−k k=0 Another (perhaps more easy) way to approach the same problem is do a partial fraction decomposition a b 1 = + (1 − λ1 x)(1 − λ2 x) 1 − λ1 x 1 − λ2 x ⇒ λ2 λ1 ,b= a= λ1 − λ2 λ2 − λ1 ∞ ∞ X X a−1 (L) = a λi1 Li + b λi2 Li i=0 i=0 This trick only works when the λi are unique. The formula is slightly different otherwise. Note: the λi are the inverse of the roots of the lag polynomials. To invert a polynomial, we needed |λi | < 1, i.e., the roots of the polynomial are outside of unit circle. Simple Processes Autoregressive (AR) AR(1): yt = ρyt−1 + et , |ρ| < 1 (1 − ρL)yt = et AR(p): a(L)yt = et , where a(L) is order p Moving average (MA) M A(1): yt = et + θet−1 yt = (1 + θL)et M A(q): yt = b(L)et , where b(L) is order q Covariance structure 4 ARMA ARM A(p, q): a(L)yt = b(L)et , where a(L) is order p and b(L) is order q, and a(L) and b(L) are relatively prime. An ARMA representation is not unique. For example, an AR(1) (with |ρ| < 1) is equal to an M A(∞), as we saw above. We can see it from the formula for inversion: yt = ∞ X ρj et−j =0 Aside: you can also get this formula by repeatedly using the definition of AR(1): yt = ρyt−1 + et = ρ(ρyt−2 + et−1 ) + et = ... = k−1 X ρj et−j + ρk yt−k =0 2 and noticing that ρk yt−k →L 0 as k → ∞ In fact, this is more generally true. Any AR(p) with roots outside the unit circle has an M A representation. These processes are called stationary ( because there is a weakly stationary version of them). Any MA process with roots outside unit circle can be written as AR(∞), such processes called invertible. If yt = b(L)et ia an invertible MA process, then et = b(L)−1 yt . That is, the “errors” are laying in a space of observations and can be recovered from y’s (another name for this: errors are fundamental ). Covariance structure Definition 7. auto-covariance γk ≡ cov(yt , yt+k ) Definition 8. auto-correlation ρk ≡ γk γ0 AR(1) example yt = ρyt−1 + et Observe V ar(yt ) = ρ2 V ar(yt−1 ) + σ 2 , and V ar(yt ) = V ar(yt−1 ) = γ0 , so γ0 = σ2 1−ρ2 . ρk σ 2 1−ρ2 . by induction that γk = Another way to see this is from the MA representation: yt = ∞ X ρi e i i=0 ⇒ γ0 = ∞ ∞ X X ρj et+k−j ) = γk = cov( ρi et−i , i=0 j=0 X ∞ X i,j:i=j−k 2 ρ2i σ 2 = σ 1 − ρ2 ρi+j σ 2 = ρk σ 2 1 − ρ2 Also, it is easy to see Covariance function More generally, if yt = 5 P∞ i=0 ci et−i then cov(yt , yt+k ) =cov Ã∞ X ci et−i , i=0 =σ 2 ∞ X ∞ X ! ci et+k−i i=0 cj cj+k j=0 P∞ MA representation and cov ariance stationarity yt = i=0 ci et−i so yt has finite variance, and in P ∞ 2 fact is covariance stationary, if j=0 cj < ∞. It is often easier to prove things with the stronger assumption P∞ P∞ of absolute summability, j=0 |cj | < ∞ (or stronger still j=0 j|cj | < ∞). Covariance function Definition 9. covariance function γ(ξ) = P∞ γi ξ i , where ξ is a complex number. i=−∞ Lemma 10. Covariance function of MA For an MA, yt = c(L)et , γ(ξ) = σ 2 c(ξ)c(ξ −1 ). Proof. c(ξ)c(ξ −1 ) =( ∞ X i ci ξ )( i=0 = = ∞ X ∞ X ci ξ −i ) i=0 cj cl ξ j−l j,l=0 ∞ X k=−∞ ξ k ∞ X cj cj+k j=0 −1 b(ξ)b(ξ ) Lemma 11. Covariance function of ARMA For an ARMA, a(L)yt = b(L)et , γ(ξ) = σ 2 a(ξ)a(ξ −1 ) The last statement will be very helpful for spectrum.