Stationary ARMA Processes

Transcript

1 Stationary ARMA Processes Eduardo Rossi University of Pavia October 2013 Rossi Stationary ARMA Financial Econometrics / 45

2 Moving Average of order 1 (MA(1)) Y t = µ + ɛ t + θɛ t 1 t = 1,..., T ɛ t WN(0, σ 2 ) E(ɛ t ) = 0 E(ɛ 2 t ) = σ 2 E(ɛ t ɛ t j ) = 0 j 0 E(Y t ) = E(µ + ɛ t + θɛ t 1 ) E(Y t ) = µ + E(ɛ t ) + θe(ɛ t 1 ) E(Y t ) = µ E(Y t µ) 2 = E(ɛ t + θɛ t 1 ) 2 = E(ɛ 2 t + θ 2 ɛ 2 t 1 + 2θɛ t 1 ɛ t ) = σ 2 + θ 2 σ = (1 + θ 2 )σ 2 Rossi Stationary ARMA Financial Econometrics / 45

3 Moving Average of order 1 (MA(1)) First Autocovariance γ(1) = E(Y t µ)(y t 1 µ) = E(ɛ + θɛ t 1 )(ɛ t 1 + θɛ t 2 ) = E(ɛ t ɛ t 1 + θɛ 2 t 1 + θɛ t ɛ t 2 + θ 2 ɛ t 1 ɛ t 2 ) = 0 + θσ = θσ 2 Higher Autocovariances are all zero γ(j) = E[(Y t µ)(y t j µ)] = 0 Rossi Stationary ARMA Financial Econometrics / 45

4 Moving Average of order 1 (MA(1)) MA(1) is covariance stationary regardless the value of θ. γ(j) = (1 + θ 2 )σ 2 + θσ 2 < j=0 If ɛ t is a Gaussian White Noise, then MA(1) is ergodic for all moments. Autocorrelation function ρ(j) γ(j) γ(0) ρ(j) 1 θσ 2 ρ(1) = (1 + θ 2 )σ 2 = θ 1 + θ 2 ρ(j) = 0 j > 0 Rossi Stationary ARMA Financial Econometrics / 45

5 Moving Average of order 1 (MA(1)) The largest possible value for ρ(1) is 0.5. This occurs if θ = 1. The smallest is 0.5, θ = 1. For 0.5 < ρ(1) < 0.5 there are two different values of θ that could produce that autocorrelation. θ 1+θ 2 is unchanged if θ is replaced by 1/θ. Rossi Stationary ARMA Financial Econometrics / 45

6 Invertibility MA(1): Y t = µ + ɛ t + θ 1 ɛ t 1 Autocorrelation function: Y t = µ + (1 + θ 1 L)ɛ t ρ(1) γ(1) γ(0) = θ 1 + θ 2 Replacing θ by 1/θ and assuming that the (unobserved) shock process has a variance of θ 2 σ 2 instead of σ 2 yields a process with the same autocovariance structure as the original process. Rossi Stationary ARMA Financial Econometrics / 45

7 Invertibility The invertibility of (1 + θ 1 z) depends on the roots of 1 + θ 1 z = 0 invertibility requires θ < 1; if θ 1 the infinite sequence (1 θl + θ 2 L 2 θ 3 L ) would not be well defined. For a MA(q) there are 2 q representations of the process having the same correlogram. Identification problem. To overcome this problem we impose the invertibility condition. AR( ) representation: θ(l) 1 Y t = θ(1) 1 µ + ɛ t Rossi Stationary ARMA Financial Econometrics / 45

8 Moving average of order q (MA(q)) Y t = µ + θ(l)ɛ t t = 1,..., T ɛ t WN(0, σ 2 ) θ(l) = 1 + θ 1 L θ q L q E(Y t ) = µ E[(Y t µ) 2 ] = E[(θ(L)ɛ t ) 2 ] = E(ɛ 2 t + θ 2 1ɛ 2 t θ 2 qɛ 2 t q) = (1 + θ θ 2 q)σ 2 Rossi Stationary ARMA Financial Econometrics / 45

9 Moving average of order q (MA(q)) because E(θ i θ j ɛ t i ɛ t j ) = 0 i j i, j = 0,..., q θ 0 = 1 The autocovariance function γ(j) = E[(θ(L)ɛ t )(θ(l)ɛ t j )] = E[(ɛ t θ j ɛ t j θ q ɛ t q )(ɛ t j + θ 1 ɛ t j θ q ɛ t q j )] = E(θ j ɛ 2 t j + θ 1 θ j+1 ɛ 2 t j θ q θ q j ɛ 2 t q) = (θ j + θ 1 θ j θ q θ q j )σ 2 j = 1,..., q γ(j) = 0 j > q Rossi Stationary ARMA Financial Econometrics / 45

10 Example: MA(2) Y t = µ + ɛ t + θ 1 ɛ t 1 + θ 2 ɛ t 2 γ(0) = (1 + θ θ 2 2)σ 2 γ(1) = E[(ɛ t + θ 1 ɛ t 1 + θ 2 ɛ t 2 )(ɛ t 1 + θ 1 ɛ t 2 + θ 2 ɛ t 3 )] = θ 1 E(ɛ 2 t 1) + θ 1 θ 2 E(ɛ 2 t 2) = (θ 1 + θ 1 θ 2 )σ 2 γ(j) = 0 j = 3, 4,... Rossi Stationary ARMA Financial Econometrics / 45

11 The Infinite-Order Moving Average Process Y t = µ + ψ j ɛ t j j=0 ɛ t WN(0, σ 2 ) The infinite sequence generates a c.s. process provided that square summability holds < j=0 a slightly stronger condition is the absolute summability ψ 2 j ψ j < j=0 Rossi Stationary ARMA Financial Econometrics / 45

12 The Infinite-Order Moving Average Process E(Y t ) = µ γ(0) = E(Y t µ) 2 = lim T E(ψ 0ɛ t + ψɛ t ψ T ɛ t T ) = lim T (ψ2 0 + ψ ψ 2 T )σ 2 γ(j) = E[(Y t µ)(y t j µ)] = σ 2 (ψ j ψ 0 + ψ j+1 ψ ) Rossi Stationary ARMA Financial Econometrics / 45

13 The Infinite-Order Moving Average Process An MA( ) with absolutely summable coefficients has absolutely summable covariances γ(j) < An MA( ) absolutely summable is ergodic for the mean. If ɛ t i.i.d.n(0, σ 2 ) then the process is ergodic for all moments. j=0 Rossi Stationary ARMA Financial Econometrics / 45

14 The Autoregressive process of order 1 (AR(1)) Y t is c.s. if φ < 1. Y t = c + φy t 1 + ɛ t ɛ t WN(0, σ 2 ) T 1 T 1 Y t = c φ j + φ T Y t T + φ j ɛ t j j=0 j=0 Rossi Stationary ARMA Financial Econometrics / 45

15 The Autoregressive process of order 1 (AR(1)) First Order Difference Equation If φ < 1 then backward solution: If φ > 1 forward solution based on (1 φl)y t = w t y t = w t + φw t (1 φl) 1 = φ 1 L 1 1 φ 1 L 1 = φ 1 L 1 [1 + φ 1 L 1 + φ 2 L ] (1 φl)(1 φl) 1 = 1 when it is applied to a bounded sequence {w t } t= the result is another bounded sequence. Applying (1 φl) 1 we are implicitly imposing a boundedness assumption. Rossi Stationary ARMA Financial Econometrics / 45

16 The Autoregressive process of order 1 (AR(1)) Premultiplying by [1 + φ 1 L φ (T 1) L (T 1) ][ φ 1 L 1 ] the limit of this operator exists and is (1 φl) 1 when φ > 1 (1 φl) 1 = [ φ 1 L 1 ][1 + φ 1 L ] Applying this operator amounts to solving the difference equation forward. Rossi Stationary ARMA Financial Econometrics / 45

17 The Autoregressive process of order 1 (AR(1)) For a AR(1) process with φ > 1: Y t = (1 φl) 1 ɛ t = [ φ 1 L 1 ][1 + φ 1 L ]ɛ t Y t = [ φ 1 L 1 ][ɛ t + φ 1 ɛ t ] = φ j ɛ t+j this is the unique stationary solution. This is regarded as unnatural since Y t is correlated with {ɛ s, s > t} a property not shared by the solution obtained when φ < 1. It is customary when modelling stationary time series to restrict attention to AR(1) processes with φ < 1 for which Y t has the representation in terms of {ɛ s, s t}. If φ = 1 there is no stationary solution. j=1 Rossi Stationary ARMA Financial Econometrics / 45

18 Stationary solution AR(p) Stationary solution A (unique) stationary solution to φ(l)y t = ɛ t exists if and only the roots of φ(z) avoid the unit circle: z = 1 φ(z) = 1 φ 1 z... φ p z p 0 Causal AR(p) This AR(p) process is causal if and only if the roots of φ(z) are outside the unit circle: z 1 φ(z) = 1 φ 1 z... φ p z p 0 Rossi Stationary ARMA Financial Econometrics / 45

19 The Autoregressive process of order 1 (AR(1)) When Y t is c.s. we can write: Y t = (c + ɛ t ) + φ(c + ɛ t 1 ) + φ 2 (c + ɛ t 2 ) +... = c + cφ + cφ ɛ t + φɛ t c = 1 φ + ɛ t + φɛ t 1 + φ 2 ɛ 2 t }{{} MA( ) ψj =φ j when φ < 1 ψ j = φ j 1 = 1 φ j=0 j=0 this ensures that the MA( ) representation exists. The AR(1) process is ergodic for the mean. Rossi Stationary ARMA Financial Econometrics / 45

20 The Autoregressive process of order 1 (AR(1)) µ E(Y t ) = c 1 φ γ(0) = E[(Y t µ) 2 ] = E[(ɛ t + φɛ t 1 + φɛ t ) 2 ] = (1 + φ 2 + φ )σ 2 = σ 2 /(1 φ 2 ) γ(j) = E[(Y t µ)(y t j µ)] = E[(ɛ t + φɛ t 1 + φ 2 ɛ t )(ɛ t j + φɛ t j 1 + φ 2 ɛ t j )] = (φ j + φ j+2 + φ j )σ 2 = φ j (1 + φ 2 + φ )σ 2 = σ 2 (φ j /(1 φ 2 )) Rossi Stationary ARMA Financial Econometrics / 45

21 The Autoregressive process of order 1 (AR(1)) Autocorrelation function geometric decay. solution: ρ(j) = γ(j) γ(0) = φj γ(j) = φγ(j 1) γ(j) = φ j γ(0) Rossi Stationary ARMA Financial Econometrics / 45

22 The AR(2) process Y t = c + φ 1 Y t 1 + φ 2 Y t 2 + ɛ t ɛ t WN(0, σ 2 ) (1 φ 1 L φ 2 L 2 )Y t = c + ɛ t The difference equation is stable provided that the roots of 1 φ 1 z φ 2 z 2 = 0 lie outside the unit circle. ψ(l) = (1 φ 1 L φ 2 L 2 ) 1 = ψ 0 + ψ 1 L + ψ 2 L Rossi Stationary ARMA Financial Econometrics / 45

23 The AR(2) process The autocovariances c µ = 1 φ 1 φ 2 Y t = µ(1 φ 1 φ 2 ) + φ 1 Y t 1 + φ 2 Y t 2 + ɛ t Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) + ɛ t multiplying both sides by (Y t j µ) and taking expectations produces E[(Y t µ)(y t j µ)] = φ 1 E[(Y t 1 µ)(y t j µ)] +φ 2 E[(Y t 2 µ)(y t j µ)] + E[ɛ t (Y t j µ)] γ(j) = φ 1 γ(j 1) + φ 2 γ(j 2) j = 1, 2,... Rossi Stationary ARMA Financial Econometrics / 45

24 The AR(2) process The autocovariances follow the same the second-order difference equation as does the process for Y t. The autocorrelations ρ(j) = φ 1 ρ(j 1) + φ 2 ρ(j 2) j = 1, 2,... Setting j = 1 For j = 2 The variance of a c.s. AR(2) ρ(1) = φ 1 + φ 2 ρ(1) ρ(1) = φ 1 1 φ 2 ρ(2) = φ 1 ρ(1) + φ 2 E[(Y t µ) 2 ] = φ 1 E[(Y t 1 µ)(y t µ)]+φ 2 E[(Y t 2 µ)(y t µ)]+e[(ɛ t )(Y t µ)] Rossi Stationary ARMA Financial Econometrics / 45

25 The AR(2) process E(ɛ t )(Y t µ) = E(ɛ t )[φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) + ɛ t ] = φ φ σ 2 γ(0) = φ 1 γ(1) + φ 2 γ(2) + σ 2 γ(0) = φ 1 ρ(1)γ(0) + φ 2 ρ(2)γ(0) + σ 2 Substituting ρ(1) and ρ(2) [ ] φ 2 γ(0) = 1 + φ 2 (φ 1 ρ(1) + φ 2 ) γ(0) + σ 2 1 φ 2 [ φ 2 = 1 + φ 2φ 2 ] 1 + φ 2 2 γ(0) + σ 2 1 φ 2 1 φ 2 Rossi Stationary ARMA Financial Econometrics / 45

26 The AR(2) process γ(0) = = = = = [ 1 φ2 1 φ 2φ 2 ] 1 1 φ 2 2 σ 2 1 φ 2 1 φ 2 [ 1 φ2 φ 2 1 φ 2φ 2 1 φ2 2 (1 φ ] 1 2) σ 2 1 φ 2 (1 φ 2 )σ 2 1 φ 2 φ 2 1 φ 2φ 2 1 φ2 2 (1 φ 2) (1 φ 2 )σ 2 1 φ 2 φ 2 1 φ 2φ 2 1 φ2 2 (1 φ 2) (1 φ 2 )σ 2 (1 + φ 2 )[(1 φ 2 ) 2 φ 2 1 ] Rossi Stationary ARMA Financial Econometrics / 45

27 The AR(p) process provided that the roots of Y t = c + φ 1 Y t 1 + φ 2 Y t φ p Y t p + ɛ t all lie the unit circle. Covariance-stationary representation: ɛ t WN(0, σ 2 ) φ(z) = 1 φ 1 z... φ p z p = 0 Y t = µ + ψ(l)ɛ t = c φ 1 φ 2... φ p 1 φ 1 L... φ p L p ɛ t Rossi Stationary ARMA Financial Econometrics / 45

28 The AR(p) process where and The mean is ψ(z) = (1 φ 1 z... φ p z p ) 1 = φ(z) 1 ψ j < j=0 c µ = E(Y t ) = 1 φ 1... φ p Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + ɛ t Autocovariances are found by multiplying both sides by (Y t j µ) and taking expectations Rossi Stationary ARMA Financial Econometrics / 45

29 The AR(p) process The autocovariance function { φ1 γ(j 1) + φ γ(j) = 2 γ(j 2) φ p γ(j p) j = 1, 2,... φ 1 γ(1) φ p γ p + σ 2 j = 0 Rossi Stationary ARMA Financial Econometrics / 45

30 The AR(p) process Dividing the autocovariance function by γ 0 we obtain the Yule-Walker equations: ρ j = φ 1 ρ j 1 + φ 2 ρ j φ p ρ j p j = 1, 2,... Thus the autocovariances and autocorrelations follow the same p-th order difference equation as does the process itself. For distinct roots, their solutions take the form γ(j) = g 1 λ j 1 + g 2λ j g pλ j p where the eigenvalues (λ 1,..., λ p ) are the solutions to λ p φ 1 λ p 1... φ p = 0 Rossi Stationary ARMA Financial Econometrics / 45

31 The Autoregressive Moving Average process (ARMA(p,q)) Y t = c + φ 1 Y t φ p Y t p + ɛ t + θ 1 ɛ t θ q ɛ t q (1 φ 1 L... φ p L p )Y t = c + (1 + θ θ q L q )ɛ t φ(l)y t = c + θ(l)ɛ t where φ(l) = 1 φ 1 L... φ p L p θ(l) = 1 + θ θ q L q the stationarity depends on the roots of 1 φ 1 z... φ p z p = 0 Rossi Stationary ARMA Financial Econometrics / 45

32 Common factors in ARMA(p,q) There is a potential for redundant parameterization with ARMA processes. Consider a simple white noise process Y t = ɛ t Suppose both sides are multiplyied by (1 ρl): (1 ρl)y t = (1 ρl)ɛ t Both are valid representations, thus the latter might be described as an ARMA(1,1) process, with φ 1 = ρ and θ 1 = ρ. Since any value of ρ describes the data equally well, we will get into trouble trying to estimate the parameter ρ by maximum likelihood. Rossi Stationary ARMA Financial Econometrics / 45

33 Common factors in ARMA(p,q) A related overparameterization can arise with an ARMA(p,q) model. Consider the factorization of the lag polynomial operators: (1 λ 1 L)(1 λ 2 L)... (1 λ p L)(Y t µ) = (1 η 1 L)... (1 η q L)ɛ t Assume that λ i < 1 for all i, so that the process is c.s.. If φ(l) and θ(l) have any roots in common, λ i = η j for some i and j, then both sides can be divided by (1 λ i L) p q (1 λ k L)(Y t µ) = (1 η k L)ɛ t k=1,k i k=1,k j Rossi Stationary ARMA Financial Econometrics / 45

34 Common factors in ARMA(p,q) (1 φ 1L... φ p 1L p 1 )(Y t µ) = (1 + θ 1L θ q 1L q 1 )ɛ t where p (1 φ 1L... φ p 1L p 1 ) (1 λ k L) k=1,k i (1 + θ 1L θ q 1L q 1 ) q k=1,k j (1 η k L) The stationary process ARMA(p,q) process is clearly identical to the stationary ARMA(p-1,q-1) process. Rossi Stationary ARMA Financial Econometrics / 45

35 Common factors: example Consider a white noise process ɛ t. We can write Y t = ɛ t Y t Y t Y t 2 = ɛ t ɛ t ɛ t 2 (1 L L 2 )Y t = (1 L L 2 )ɛ t This is in the form of an ARMA(2,2) process, with But φ(l) = (1 L L 2 ), θ(l) = (1 L L 2 ) Y t = θ(l) φ(l) ɛ t = ɛ t Rossi Stationary ARMA Financial Econometrics / 45

36 Stationarity of ARMA(p,q) Stationary solution If φ(z) and θ(z) have no common factors, a unique stationary solution to φ(l)y t = c + θ(l)ɛ t exists if and only the roots of φ(z) avoid the unit circle: z = 1 φ(z) = 1 φ 1 z... φ p z p 0 Rossi Stationary ARMA Financial Econometrics / 45

37 Causality and invertibility of ARMA(p,q) Causal ARMA(p,q) This ARMA(p,q) process is causal if and only if the roots of φ(z) are outside the unit circle: z 1 φ(z) = 1 φ 1 z... φ p z p 0 invertible if and only if the roots of θ(z) are outside the unit circle: z 1 θ(z) = 1 + θ 1 z θ q z q 0 Rossi Stationary ARMA Financial Econometrics / 45

38 Causality and invertibility of ARMA(p,q): example (1 1.5L)Y t = ( L)ɛ t φ(z) = 1 1.5z θ(z) = z φ(z) and θ(z) have no common factors, and {Y t } is an ARMA(1,1). The root of φ(z) is hence the process is not causal. φ(z) = 0 z = 2 3 < 1 θ(z) = 0 z = 5 which is outside the unit circle, so {Y t } is invertible. Rossi Stationary ARMA Financial Econometrics / 45

39 The Autoregressive Moving Average process (ARMA(p,q)) If the roots are outside the unit circle then the inverse of φ(z) exists, then dividing by φ(l) both sides Y t = µ + ψ(l)ɛ t ψ(l) = θ(l) φ(l) c µ = 1 φ 1... φ p ψ j < j=0 Rossi Stationary ARMA Financial Econometrics / 45

40 The Autoregressive Moving Average process (ARMA(p,q)) c = µ(1 φ 1... φ p ) Y t = µ(1 φ 1... φ p ) + φ 1 Y t φ p Y t p + ɛ t θ q ɛ t q Y t µ = φ 1 (Y t 1 µ) φ p (Y t p µ) + ɛ t + θ 1 ɛ t θ q ɛ t q The variance E[(Y t µ) 2 ] = φ 1 E[(Y t 1 µ)(y t µ)] φ p E[(Y t p µ)(y t µ)] +E[ɛ t (Y t µ)] + θ 1 E[ɛ t 1 (Y t µ)] θ q E[ɛ t q (Y t µ)] E[(Y t µ) 2 ] = φ 1 [σ 2 (ψ 1 ψ 0 + ψ 2 ψ )] φ p [σ 2 (ψ p ψ 0 + ψ p+1 ψ )] +E[ψ 0 ɛ 2 t ] + θ 1 E[ψ 1 ɛ 2 t 1] θ q E[ψ q ɛ 2 t q] Rossi Stationary ARMA Financial Econometrics / 45

41 The Autoregressive Moving Average process (ARMA(p,q)) E[(Y t µ) 2 ] = φ 1 [σ 2 (ψ 1 ψ 0 + ψ 2 ψ )] φ p [σ 2 (ψ p ψ 0 + ψ p+1 ψ )] +ψ 0 σ 2 + θ 1 ψ 1 σ θ q ψ q σ 2 An ARMA(p,q) process will have more complicated autocovariances for lags 1 through q than would the corresponding AR(p) process γ(j) = E[(Y t µ)(y t j µ)] = φ 1 E[(Y t 1 µ)(y t j µ)] φ p E[(Y t p µ)(y t j µ)] +E[ɛ t (Y t j µ)] + θ 1 E[ɛ t 1 (Y t j µ)] θ q E[ɛ t q (Y t j µ)] Rossi Stationary ARMA Financial Econometrics / 45

42 For j > q autocovariances are given by E[(Y t µ)(y t j µ)] = φ 1 E[(Y t 1 µ)(y t j µ)] φ p E[(Y t p µ)(y t j µ)] +E[ɛ t (Y t j µ)] + θ 1 E[ɛ t 1 (Y t j µ)] θ q E[ɛ t q (Y t j µ)] E[ɛ t q (Y t j µ)] = E[ɛ t q (ψ(l)ɛ t j )] = E[ɛ t q (ψ 0 ɛ t j + ψ 1 ɛ t j )] = ψ 0 E[ɛ t q ɛ t j ] + ψ 1 E[ɛ t q ɛ t j 1 ] +... = 0 then E[(Y t µ)(y t j µ)] = φ 1 E[(Y t 1 µ)(y t j µ)] φ p E[(Y t p µ)(y t j µ)] Rossi Stationary ARMA Financial Econometrics / 45

43 The Autoregressive Moving Average process (ARMA(p,q)) γ(j) = φ 1 γ(j 1) + φ 2 γ(j 2) φ p γ(j p) j = q + 1, q + 2,... Thus after q lags the autocovariance function follow the p-th order difference equation governed by the autoregressive parameters. Rossi Stationary ARMA Financial Econometrics / 45

44 ARMA(1,1) Y t = c + φ 1 Y t 1 + ɛ t + θ 1 ɛ t 1 (1 φ 1 L)Y t = c + (1 + θ 1 L)ɛ t φ 1 < 1 ψ(l) = 1 + θ 1L 1 φ 1 L = (1 + θ 1L)(1 + φ 1 L + φ 2 1L ) ψ 0 + ψ 1 L + ψ 2 L = (1 + θ 1 L)(1 + φ 1 L + φ 2 1L ) Rossi Stationary ARMA Financial Econometrics / 45

45 ARMA(1,1) ψ 0 = 1 ψ 1 = θ 1 + ψ 1 ψ 2 = φ φ 1 θ 1... =... γ(0) = φ 1 E[(Y t 1 µ)(y t µ)] + E[ɛ t (Y t µ)] + θ 1 E[ɛ t 1 (Y t µ)] γ(0) = φ 1 [σ 2 (ψ 1 ψ 0 + ψ 2 ψ )] + σ 2 + θ 1 ψ 1 σ 2 γ(1) = φ 1 γ(0) + θ 1 σ 2 γ(2) = φ 1 γ(1) Rossi Stationary ARMA Financial Econometrics / 45