General Linear Process. Autocovariance and Spectrum

Transcript

1 STAT 520 Linear Stationary and Nonstationary Models 1 General Linear Process Slide 1 Consider a general linear process of the form z t = a t + j=1 ψ ja t j = (1+ j=1 ψ jb j )a t = ψ(b)a t, where a t is a white noise process with var[a t ] = σ 2 a, B is the backward shift operator, Bz t = z t 1, B j z t = z t j, and ψ(b) is called the transfer function. Alternatively, one may write (1 j=1 π jb j )z t = π(b)z t = a t, where the current value of z t is regressed on the past values z t j. It is easily seen that π(b)ψ(b) = 1. It is known that any zero-mean stationary Gaussian process can be written in the MA form z t = ψ(b)a t with j=1 ψ j <. The transfer function ψ(b) defines a linear filter that transforms the input a t to output z t. The filter is stable with j=1 ψ j <. Autocovariance and Spectrum Slide 2 Set ψ 0 = 1 and ψ h = 0, h < 0. It is easy to calculate γ k = σa 2 j=0 ψ jψ j+k. Write γ(b) = k= γ kb k as the autocovariance generating function. It follows that γ(b) = σa 2 k j ψ jψ j+k B k = σa 2 j ψ jb j k ψ j+kb j+k = σaψ(b 2 1 )ψ(b), where B 1 = F is the forward shift operator. Recall the definition of the power spectrum, p(ω) = k= γ ke i2πkω. Substituting e i2πω for B in ψ(b), one has p(ω) = σ 2 aψ(e i2πω )ψ(e i2πω ) = σ 2 a ψ(e i2πω ) 2.

2 STAT 520 Linear Stationary and Nonstationary Models 2 Stationarity and Invertibility Slide 3 For the linear process z t = ψ(b)a t to be a valid stationary process, ψ(e i2πω ) = j=0 ψ je i2πωj must be convergent, i.e., ψ(b) be convergent for B 1. It suffices to have j=0 ψ j <. A process is invertible if π(b) is convergent for B 1. It suffices to have j=0 π j <. To illustrate the idea, consider the MA(1) process z t = (1 θb)a t. Since k 0 xj (1 x) = 1 x k+1, one has z t = k j=1 θj z t j +a t θ k+1 a t k 1. For θ < 1, one may let k and invert the process into an AR( ) process, with π j = θ j dying out as j 0. For θ 1, θ k+1 a t k 1 0. Also note that ρ 1 = 1/(θ +θ 1 ), so θ = b ±1 are not identifiable. Invertibility removes the ambiguity and assures practical sensibility. AR(p) Process: Stationarity Slide 4 An autoregressive process of order p (i.e., AR(p)) is defined by z t = φ 1 z t 1 + +φ p z t p +a t, or (1 φ 1 B φ p B p )z t = φ(b)z t = a t. The transfer function is given by ψ(b) = φ 1 (B). AR(p) is invertible by definition. Write φ(b) = p j=1 (1 G jb), where G 1 j are the roots of φ(b) = 0. One has (assuming distinctive roots), ψ(b) = p j=1 1 1 G j B = p j=1 K j 1 G j B, so one must have G i < 1 for ψ(b) to be convergent for all B 1. In other words, one needs the roots of φ(b) to lie outside of the unit circle for z t = φ 1 (B)a t to be stationary. To get the roots of 1+.6x+.5x 2, use polyroot(c(1,.6,.5)) in R.

3 STAT 520 Linear Stationary and Nonstationary Models 3 Examples: AR(1) and AR(2) Stationarity condition For AR(1), one needs φ 1 < 1. For AR(2), one needs g j < 1 in the expression Slide 5 φ(b) = (1 g 1 B)(1 g 2 B) = 1 (g 1 +g 2 )B ( g 1 g 2 )B 2. With g j real, (φ 1,φ 2 ) = (g 1 +g 2, g 1 g 2 ) over g 1,g 2 ( 1,1). With g j a conjugate pair Ae ±i2πω, one has (φ 1,φ 2 ) = (2Acos2πω, A 2 ) over ω ( 1/2,1/2), A (0,1). Autocorrelation For AR(1), ρ k = φ k 1, k 0. For AR(2), ρ k = φ 1 ρ k 1 +φ 2 ρ k 2, k > 0; ρ 0 = 1, ρ 1 = φ 1 /(1 φ 2 ). Examples: AR(1) and AR(2) Variance For AR(1), γ 0 = φ 1 γ 1 +σ 2 a, so γ 0 = σ 2 a/(1 φ 1 ρ 1 ) = σ 2 a/(1 φ 2 1). For AR(2), γ 0 = φ 1 γ 1 +φ 2 γ 2 +σ 2 a, so Slide 6 γ 0 = Power spectrum σ 2 a 1 φ 1 ρ 1 φ 2 ρ 2 = 1 φ 2 1+φ 2 σ 2 a {(1 φ 2 ) 2 φ 2 1 }. For AR(1), p(ω) = σ 2 a/ 1 φ 1 e i2πω 2 = σ 2 a/(1+φ 2 1 2φ 1 cos2πω). For AR(2), p(ω) = σ 2 a 1 φ 1 e i2πω φ 2 e i4πω 2 = σ 2 a 1+φ 2 1 +φ2 2 2φ 1(1 φ 2 )cos2πω 2φ 2 cos4πω.

4 STAT 520 Linear Stationary and Nonstationary Models 4 AR(p) Process: Yule-Walker Equations Taking expectations of the expression, z t k z t = φ 1 z t k z t 1 + +φ p z t k z t p +z t k a t, one has, after dividing by γ 0, Slide 7 ρ k = φ 1 ρ k 1 + +φ p ρ k p, k > 0. Substituting k = 1,...,p, one obtains the Yule-Walker equations 1 ρ 1 ρ 2... ρ p 1 φ 1 ρ 1 ρ 1 1 ρ 1... ρ p 2 φ 2 ρ =.,. ρ p 1 ρ p 2 ρ p φ p ρ p or P p φ = ρ p, and φ = P 1 p ρ p expresses φ j s in terms of ACF s. Partial Autocorrelation Function Consider a Gaussian stationary process. The partial autocorrelation function at lag k is defined by α k = corr[(z k E[z k z 1,...,z k 1 ]),(z 0 E[z 0 z 1,...,z k 1 ])]. Slide 8 For a general process, replace E[y z 1,...,z k 1 ] by the linear projection of y in {z 1,...,z k 1 }, ŷ = b 0 +b 1 z 1 + +b k 1 z k 1, with b j s chosen to minimize E(y ŷ) 2. It can be shown that α k equals the kth element of φ k = P 1 k ρ k, φ kk. Replacing ρ v by r v in the Yule-Walker equations, one gets the sample PACF ˆφ kk as the kth element of ˆφ 1 k = ˆP k ˆρ k. For AR(p) processes at lag k > p, one has φ kk = 0, and it can be shown that, asymptotically, ˆφ kk N(0, 1 N ). Sample PACF s can be calculated and plotted in R using acf with type="partial".

5 STAT 520 Linear Stationary and Nonstationary Models 5 Recursive Yule-Walker Solutions and PACF Slide 9 Let h = k 1. Partition P k = ( P h ρ h ρ T ), where ρ h 1 h is ρ h in reverse order, and write d = 1 ρ T h P 1. One has P 1 k = h ρ h = 1 ρt h P 1 h ρ h = 1 φt h ρ h ( P 1 h +d 1 P 1 h ρ h ρt h P 1 h d 1 ρ T h P 1 h d 1 P 1 h ρ h Write φ h = P 1 h ρ h. Straightforward algebra yields, φ k = P 1 k ρ h = ρ k φ h d 1 (ρ k φ T h ρ h ) φ h d 1 (ρ k φ T h ρ h ) d 1 = which gives the recursive formulas for Yule-Walker solutions. ). φ h φ kk φ h φ kk, Consider Gaussian process with σa 2 = 1. The conditional covariance matrix of (z 0,z k ) (z 1,...,z k 1 ) is given by ( ) 1 ρk ρ T ) ρ k 1 ( h P 1 ρ T h (ρ h, ρ h ). h It follows that α k = (ρ k ρ T h P 1 h ρ h )/(1 ρt h P 1 h ρ h ) = φ kk. Yule-Walker (Moment) Estimates for AR(p) Recall the Yule-Walker equations P p φ = ρ p and note the fact that γ 0 = p j=1 φ jγ j +σa. 2 Substituting the moment estimates ˆρ j = r j for ρ j, j = 1,...,p and ˆγ 0 = c 0 for γ 0, one has the Yule-Walker estimates ˆφ 1 p = ˆP p ˆρ p and ˆσ a 2 = c 0 (1 ˆφ T p ˆρ p) = ˆv p. Slide 10 Recall the recursive Yule-Walker solutions, one has φ kk = ρ k ρ T k 1 φ k 1 1 φ T k 1ρ k 1 = γ 0 (ρ k ρ T k 1 φ k 1 )/v k 1, φ k,k 1 = φ k 1 φ kk φk 1, v k = γ 0 (1 φ T kρ k ) = v k 1 (1 φ 2 kk), where φ T k = (φ T k,k 1,φ kk ). Putting hats on the parameters and starting with ˆφ 11 = r 1 and ˆv 1 = c 0 (1 r1), 2 one obtains the Durbin-Levinson algorithm for fitting AR models.

6 STAT 520 Linear Stationary and Nonstationary Models 6 Examples: AR(1), AR(2), and AR(3) Slide 11 The Y-W equations for AR(1), AR(2), and AR(3) are φ 1 = ρ 1, ( )( ) ( ) 1 ρ 1 ρ 2 φ 1 ρ 1 1 ρ 1 φ 1 ρ 1 =, and ρ 1 1 ρ 1 φ 2 = ρ 2. ρ 1 1 φ 2 ρ 2 ρ 2 ρ 1 1 φ 3 ρ 3 The Durbin-Levinson algorithm proceeds as follows: 1. φ 11 = r 1 ; v 1 = c 0 (1 r 2 1). 2. φ 22 = c 0 (r 2 ρ 1 φ 11 )/v 1 ; φ 21 = φ 11 φ 22 φ 11 ; v 2 = v 1 (1 φ 2 22). 3. φ 33 = c 0 (r 3 ρ 1 φ 22 ρ 2 φ 21 )/v 2 ; (φ 31,φ 32 ) = (φ 21,φ 22 ) φ 33 (φ 22,φ 21 ); v 3 = v 2 (1 φ 2 33). MA(q) Process: Invertibility Slide 12 A moving average process of order q (i.e., MA(q)) is defined by z t = a t θ 1 a t 1 θ q a t q, or z t = (1 θ 1 B θ q B q )a t = θ(b)a t. The transfer function is given by ψ(b) = θ(b). MA(q) is stationary by definition. Similar to the stationarity condition for AR(p), one needs the roots of θ(b) to lie outside of the unit circle for z t = θ(b)a t to be invertible. Let G 1 j be the roots of θ(b) and consider the spectrum p(ω) = q j=1 1 G je i2πω 2. For G j real, 1 G j e i2πω 2 (G j +G 1 j 2cos2πω), so G ±1 j are exchangeable. Similar arguments can be made for conjugate pairs of complex roots. Hence, MA(q) models come in 2 q -plet, of which only one is invertible, barring G j = 1.

7 STAT 520 Linear Stationary and Nonstationary Models 7 Examples: MA(1) and MA(2) Invertibility condition The invertibility of MA(1) and MA(2) is dual to the stationarity of AR(1) and AR(2). Slide 13 Variance and autocorrelation For MA(1), γ 0 = σ 2 a(1+θ 2 1); ρ 1 = θ 1 /(1+θ 2 1), ρ k = 0, k > 1. For MA(2), γ 0 = σ 2 a(1+θ 2 1 +θ 2 2); ρ 1 = θ 1(1 θ 2 ) 1+θ1 2, ρ 2 = +θ2 2 θ 2 1+θ1 2, ρ k = 0, k > 2. +θ2 2 Power spectrum Replacing φ 1 by θ 1 and φ 2 by θ 2 in the power spectrums of AR(1) and AR(2), and move the denominators to the numerators, one gets the power spectrums of MA(1) and MA(2). Multiplicity: MA(1) and MA(2) Consider z t = (1 2B)a t, which has the same autocorrelation function as the invertible z t = (1 0.5B)a t. Slide 14 Consider z t = (1 B B 2 )a t = ( B)( B)a t, which has the same autocorrelation function as the invertible z t = ( B 2 )a t = ( B)( B)a t, where 1/1.618 = The other two members of the family are z t = ( B 2 )a t = ( B)( B)a t and z t = (1+B B 2 )a t = ( B)( B)a t. The a t in different expressions are independent but may have different variances.

8 STAT 520 Linear Stationary and Nonstationary Models 8 ARMA(p, q) Process An ARMA(p,q) model is of the form z t φ 1 z t 1 φ p z t p = a t θ 1 a t 1 θ q z t q, Slide 15 or φ(b)z t = θ(b)a t, where φ(b) and θ(b) are polynomials of degree p and q in B. The stationarity and invertibility are governed by the roots of φ(b) and θ(b). For k > q, since φ(b)z t = θ(b)a t is uncorrelated with z t k, one has cov[φ(b)z t,z t k ] = φ(b)ρ k = 0, or more explicitly, ρ k = φ 1 ρ k 1 + +φ p ρ k p, k > q. With the transfer function ψ(b) = φ 1 (B)θ(B), the power spectrum of ARMA(p,q) is seen to be p(ω) = σ 2 a θ(e i2πω ) 2 / φ(e i2πω ) 2. Example: ARMA(1,1) Stationarity and invertibility condition For stationarity, one needs φ 1 < 1, for invertibility, θ 1 < 1. Slide 16 Variance and autocorrelation Note that E[z t a t ] = E[(φ 1 z t 1 +a t θ 1 a t 1 )a t ] = σ 2 a, one has γ 0 = E[(φ 1 z t 1 +a t θ 1 a t 1 ) 2 ] = φ 2 1γ 0 +σ 2 a +θ 2 1σ 2 a 2φ 1 θ 1 σ 2 a, so γ 0 = a 2 t(1+θ 2 1 2φ 1 θ 1 )/(1 φ 2 1). Similarly, one has ρ 1 = φ 1 θ 1 σ 2 a/γ 0 = (φ 1 θ 1 )(1 2φ 1 θ 1 ) 1+θ 2 1 2φ 1θ 1, ρ k = φ k 1 1 ρ 1, k > 1. Power spectrum p(ω) = σ 2 1 θ 1 e i2πω 2 a 1 φ 1 e i2πω 2 = σ2 a 1+θ 2 1 2θ 1 cos2πω 1+φ 2 1 2φ 1cos2πω.

9 STAT 520 Linear Stationary and Nonstationary Models 9 Moment Estimates for MA(q) and ARMA(p,q) Slide 17 For an MA(q) model, one has γ 0 = σa(1+ 2 q j=1 θ2 j ) and γ k = σa( θ 2 k + q k j=1 θ jθ j+k ), k 1. The moment estimates of σa, 2 θ q,..., θ 1 can be obtained through a simple iteration, σ 2 a = c 0 /(1+ q j=1 θ2 j), θ k = (c k /σ 2 a q k j=1 θ jθ j+k ), k = q,...,1. Remember that the solutions of θ j and σ 2 a are not unique. For an ARMA(p,q) model, one needs to use c j, j = 0,...,p+q. One can solve φ j from the equations, γ k = φ 1 γ k 1 + +φ p γ k p, k = q +1,...,q +p. Note that w t = φ(b)z t = θ(b)a t, and the ACF of w t is γ k = φt Γ k φ, where φ T = (1, φ 1,..., φ p ) and Γ k has (i,j)th entry γ k+j i. Use c k = ˆφ TˆΓkˆφ in the MA iteration above to get θj. Moment Estimates: ARMA(1,1) When applied to an ARMA(1,1) process (1 φb)z t = (1 θb)a t, the algorithm for moment estimates proceeds as follows: Slide Solve φ from r 2 = φr Calculate ( )( ) ( )( ) c c 0 c = (1, φ), c c 1 c = (1, φ). c 1 c 0 φ c 0 c 1 φ 3. Solve θ, σ 2 a from equations σa 2 = c 0 1+θ 2, θ = c 1 σa 2.

10 STAT 520 Linear Stationary and Nonstationary Models 10 Estimation of Mean Slide 19 Consider φ(b)z t = µ+θ(b)a t. It is easily seen that E[z t ] = µ/(1 φ 1 φ p ). Recall the large sample variance of the sample mean z, var[ z] = 1 n k= γ k = γ(1) n = σ2 a n ψ2 (1) = σ2 a θ 2 (1) n φ 2 (1) = p(0) n, where γ(b) = σ 2 aψ(b)ψ(b 1 ) is the covariance generating function and p(ω) is the power spectrum. The moment estimate of µ is thus ˆµ = ˆφ(1) z with approximate standard error ˆσ a ˆθ(1) / n. Fitting an ARMA(1,1) model to Series A, one has ˆφ =.8683, ˆθ =.4804, and ˆσ a 2 = Further, z = with s.e.[ z] ˆp(0)/n =.0882, and ˆµ = (1.8683)(17.06) = 2.25 with s.e.[ˆµ] = ˆσ a (1 ˆθ)/ n = Linear Difference Equation and ACF From the linear difference equation φ(b)ρ k = 0, k > q, one can obtain a general expression for ρ k. Slide 20 Write φ(b) = p j=1 (1 G jb), where G 1 j are the roots of φ(b). It is easy to verify that (1 G j B)G t j = 0, so ρ k has a term A j G k j. For a double root G 1 j, one also has (1 G j B) 2 (tg t j ) = 0, so ρ k has terms (A j,0 +A j,1 k)g k j. In general, a root G 1 j of multiplicity m contributes terms m 1 v=0 A j,vk v G k j. For pairs of conjugate complex roots G j 1 e ±iγ j, one has terms G j k (A j e iγ jk +Āje iγ jk ) = 2 G j k A j cos(kγ j +α j ). Assuming distinct roots, one has ρ k = p j=1 A jg k j, where A j s are determined by the initial values ρ q,...,ρ q p+1.

11 STAT 520 Linear Stationary and Nonstationary Models 11 Examples: AR(2) and ARMA(2,1) For (1 0.4B 0.21B 2 )z t = (1 0.7B)(1+0.3B)z t = a t, ρ k = A k +A 2 ( 0.3) k, k > 0, as φ(b)ρ k = 0, k > 0. A 1 and A 2 can be fixed via ρ 0 = 1 and ρ 1 = ρ 1 = φ 1 /(1 φ 2 ). Slide 21 For (1 0.8e iπ/3 B)(1 0.8e iπ/3 B)z t = (1 0.5B)a t, ρ k = A(0.8e iπ/3 ) k +Ā(0.8e iπ/3 ) k = A (0.8) k e i(α+kπ/3) + A (0.8) k e i(α+kπ/3) = (0.8) k 2 A cos(kπ/3+α) = (0.8) k {Bcos(kπ/3)+Csin(kπ/3)}, k > 1, where B and C can be fixed from ρ 0 = 1 and ρ 1 ; ρ 1 and κ = σa/γ 2 0 satisfy equations 1 = φ 2 1 +φ φ 1 φ 2 ρ 1 +(1+θ 2 2φ 1 θ)κ and ρ 1 = φ 1 +φ 2 ρ 1 θκ, where φ 1 = 0.8, φ 2 = 0.64, and θ = 0.5. Reverse Time Stationary Models A stationary process is characterized by its autocovariance and mean, independent of the time direction. In particular, models assuming forward or reverse time are mathematically equivalent. Slide 22 Recall the autocovariance generating function of z t = ψ(b)a t, γ(b) = σaψ(b)ψ(b 2 1 ). It is clear that z t = ψ(f)a t has the same autocovariance, where F = B 1 is the forward shift operator. For ARMA(p,q), let G 1 j be the roots of θ(b) and H 1 j those of φ(b). The same autocovariance is shared by all processes of the form p j=1 (1 H jb ±1 )z t = q j=1 (1 G jb ±1 )a t. Consider an MA(1) process z t = a t θa t 1. For θ > 1, one has z t = a t θa t 1 = ( θ)( θ 1 a t +a t 1 ) = ã t θ 1 ã t+1, an invertible reverse time MA(1) model, where ã t = θa t 1.

12 STAT 520 Linear Stationary and Nonstationary Models 12 Model Identification via ACF/PACF For k > q with an MA(q) process, ρ k = 0, E[r k ] 0, and var[r k ] (1+2 q j=1 ρ2 j )/N. Slide 23 For k > p with an AR(p) process, φ kk = 0, E[ˆφ kk ] 0, and var[ˆφ kk ] 1/N, where ˆφ kk is the Yule-Walker estimate of φ kk. For an stationary ARMA(p,q) process, ρ k damps out exponentially. If φ(b) has a near unit root G 1 i = (1 δ i ) 1, ρ k has a term A i (1 δ i ) k A i (1 kδ i ), damping out at a much slower linear rate. A slowly damping ρ k signifies nonstationarity. In practice, one inspect r k for stationarity, take differences if nonstationary, and repeat the process. The order identification of mixed ARMA model is not as straightforward. Model Selection via AIC or BIC To each observed series, one usually can fit several different models with similar goodness-of-fit. For example, suppose the ARMA(1,1) model (1+.2B)z t = (1.8B)a t is a good fit to the data. Since (1+.2B) 1 (1.8B) = 1 B +.2B 2.04B B +.2B 2, Slide 24 so an MA(2) fit z t = (1 B +.2B 2 )a t is also likely a good fit. AIC and BIC can be of assistance in the selection of competing models. Let l(γ z) be the log likelihood of the model and ˆγ be the MLE of γ, where γ consists of all model parameters including φ j, θ k, and σ 2 a. AIC and BIC are defined by AIC = 2l(ˆγ z)+2r, BIC = 2l(ˆγ z)+rlogn, where r is the number of parameters and n is the sample size. Models with smaller AIC or BIC are considered better ones.

13 STAT 520 Linear Stationary and Nonstationary Models 13 ARIMA(p,d,q) Processes To model nonstationary yet nonexplosive series, a popular device is the autoregressive integrated moving average (ARIMA) model, φ(b) d z t = ϕ(b)z t = θ(b)a t, Slide 25 where ϕ(b) = φ(b) d is a generalized AR operator. Note that d = (1 B) d has roots on the unit circle. A process with roots of ϕ(b) inside the unit circle is explosive. Assume stationarity and invertibility for d z t. An ARIMA model can be written in the AR( ) form π(b)z t = a t, where π(b) = 1 j=1 π jb j = θ 1 (B)φ(B)(1 B) d. For d > 0, since π(1) = 0, one has j=1 π j = 1. MA form of ARIMA Processes Symbolically, an ARIMA process can be written in a MA( ) form z t = ψ(b)a t, ψ(b) = 1+ i=1 ψ ib i, although {z t } is nonstationary and the filter unstable. From ϕ(b)ψ(b) = θ(b), one has Slide 26 ψ j = ϕ 1 ψ j 1 + +ϕ p+d ψ j p d θ j, j > 0, where ψ 0 = 1, ψ j = 0, j < 0. For j > q, ϕ(b)ψ j = 0. Take a time origin k < t and write z t = I k (t k)+c k (t k), where I k (t k) = t k 1 j=0 ψ j a t j. For t k > q, ϕ(b)i k (t k) = θ(b)a t, so ϕ(b)c k (t k) = 0. C k (t k) is called the complementary function, and is seen to be determined by the history up to time k. It follows that E[z t z k,z k 1,...] = C k (t k). Note that C k (t k) = C k 1 (t (k 1))+ψ t k a k.

14 STAT 520 Linear Stationary and Nonstationary Models 14 Example: ARIMA(1,1,1) Consider p = d = q = 1 with φ, θ < 1. ϕ(b) = (1 φb)(1 B). Since ϕ(b)ψ j = 0, j > 1, one has ψ j = A 0 +A 1 φ j, where A 0 = (1 θ)/(1 φ) and A 1 = (θ φ)/(1 φ) are determined from A 0 +A 1 = ψ 0 = 1 and A 0 +A 1 φ = ψ 1 = ϕ 1 θ = 1+φ θ. Slide 27 Since C k (t k) = b (k) 0 +b (k) 1 φt k for t k > 1, one has z t = t k 1 j=0 (A 0 +A 1 φ j )a t j +(b (k) 0 +b (k) 1 φt k ), where b (k) 0, b(k) 1 satisfy the initial conditions b (k) 0 +b (k) 1 = z k and b (k) 0 +b (k) 1 φ+a k+1 = z k+1 = (1+φ)z k φz k 1 +a k+1 θa k. Solving for b (k) 0, b(k) 1 from the equations, one has b (k) 0 = (z k φz k 1 θa k )/(1 φ), b (k) 1 = ( φ(z k z k 1 )+θa k )/(1 φ). With π(b) = (1 θb) 1 (1 φb)(1 B), it is easy to verify that π 1 = 1+φ θ, π j = (1 θ)(θ φ)θ j 2, j > 1. Example: IMA(0,2,2) Consider p = 0, d = q = 2 with θ(b) invertible. ϕ(b) = (1 B) 2. Since ϕ(b)ψ j = 0, j > 2, one has ψ j = A 0 +A 1 j, where A 0 = 1+θ 2 and A 1 = 1 θ 1 θ 2 are solved from A 0 +A 1 = ψ 1 = ϕ 1 θ 1 = 2 θ 1 and A 0 +2A 1 = ψ 2 = ϕ 1 ψ 1 +ϕ 2 θ 2 = 2(2 θ 1 ) (1+θ 2 ). Slide 28 Since C k (t k) = b (k) 0 +b (k) 1 (t k) for t k > 2, one has z t = a t + t k 1 j=1 (A 0 +A 1 j)a t j +(b (k) 0 +b (k) 1 (t k)), where b (k) 0, b(k) 1 satisfy the initial conditions b (k) 0 +b (k) 1 = z k+1 a k+1 and b (k) 0 +2b (k) 1 = z k+2 a k+2 ψ 1 a k+1. It follows that b (k) 1 = z k+2 z k+1 a k+2 (1 θ 1 )a k+1 = z k z k 1 (θ 1 +θ 2 )a k θ 2 a k 1 and b (k) 0 = z k+1 a k+1 b (k) 1 = z k +θ 2 a k. Note that C k (0) = z k b (k) 0. Since θ(b)π(b) = ϕ(b), one has π 1 = 2 θ 1, π 2 = π 1 θ 1 (1+θ 2 ) = θ 1 (2 θ 1 ) (1+θ 2 ), and θ(b)π j = 0, j > 2.

15 STAT 520 Linear Stationary and Nonstationary Models 15 ARIMA Processes with Added Noise The sum of independent MA processes of orders q and q 1 is itself an MA process of order max(q,q 1 ). Slide 29 Suppose one observes Z t = z t +b t, where φ(b) d z t = θ(b)a t and φ 1 (B)b t = θ 1 (B)α t with a t, α t being two independent white noise processes. It follows that φ 1 (B)φ(B) d Z t = φ 1 (B)θ(B)a t +φ(b)θ 1 (B) d α t, so Z t is of order (p 1 +p,d,max(p 1 +q,p+d+q 1 )). In particular, an IMA process with added white noise is of order (0,d,max(q,d)). If φ(b) and φ 1 (B) share some common roots, the orders will be lower. In general, an ARIMA model of form ϕ(b)z t = θ(b)a t is over-parameterized if ϕ(b) and θ(b) have common roots. Example: IMA(0,1,1) and Random Walk Consider Z t = z t +b t, where z t = a t θa t 1 and a t, b t are independent white noise with variances σ 2 a, σ 2 b. For the autocovariance of Z t = (1 θb)a t +(1 B)b t, one has Slide 30 γ 0 = σ 2 a(1+θ 2 )+2σ 2 b, γ 1 = θσ 2 a σ 2 b, γ k = 0, k > 1. Write Z t = u t Θu t 1 and equate γ 0 = σ 2 u(1+θ 2 ), γ 1 = Θσ 2 u, Θ = r(1+θ2 )+2 4r(1 θ) 2 +r 2 (1 θ 2 ) 2 2(1+rθ), σu 2 = θσ2 a +σ2 b Θ, where r = σa/σ 2 b 2. Consider a random walk with θ = 0. One has Θ = (r+2 4r+r 2 )/2, σ 2 u = σ 2 b /Θ. Hence, an IMA(0,1,1) process with Θ > 0 is seen to be a random walk buried in a white noise.

16 STAT 520 Linear Stationary and Nonstationary Models 16 Testing for Unit Root Slide 31 Consider an AR(1) process z t = φz t 1 +a t. Observing z 0,...,z n and minimizing the LS criterion n t=1 (z t φz t 1 ) 2, one has ˆφ = n t=1 z tz t 1 / n t=2 z2 t 1 = φ+ n t=1 z t 1a t / n t=1 z2 t 1. It can be shown through conditioning arguments that E[ n t=1 z t 1a t ] = 0, var[ n t=1 z t 1a t ] = σ 2 ae[ n t=1 z2 t 1]. For φ < 1, z t is stationary with γ 0 = var[z t ] = σ 2 a/(1 φ 2 ), so n/(1 φ2 )(ˆφ φ) = O p (1). For φ = 1, E[ n t=1 z2 t 1] = σ 2 an(n+1)/2, thus n(ˆφ 1) = O p (1). A test based on the t-statistic, ˆτ = (ˆφ 1)/ s 2 / n t=1 z2 t 1, where s 2 = n t=1 (z t ˆφz t 1 ) 2 /(n 1), was proposed by Dickey and Fuller, who derived its asymptotic null distribution under φ = 1. Testing for Unit Root Allowing for a constant, a linear trend, and possibly dependent but stationary innovations u t with autocovariance γ k, one has z t = β 0 +β 1 (t n/2)+φz t 1 +u t. Slide 32 The asymptotic distribution of the t-statistic, ˆτ = (ˆφ 1)/s.e.[ˆφ], was derived by Phillips and Perron under φ = 1, which depends on γ 0 and σ 2 = p u (0) = k= γ k. Consistent estimates of γ 0 and σ 2 are ˆγ 0 = n t=1û2 t/(n 3) and the Newey-West estimate, ˆσ 2 = n 1 n t=1û2 t +2n 1 l s=1 w sl n t=s+1ûtû t s, where û t are the residuals from the LS fit, w sl = 1 s/(l+1), and l, l 4 /n 0 as n. The test is implemented in PP.test. For φ(b) z t =θ(b)a t, z t =z t 1 + p j=1 φ jw t j +θ(b)a t =z t 1 +u t, where w t = z t. The process {u t } is stationary when {w t } is.