ARMA Models: I VIII 1

Transcript

1 ARMA Models: I autoregressive moving-average (ARMA) processes play a key role in time series analysis for any positive integer p & any purely nondeterministic process {X t } with ACVF {γ X (h)}, there is an AR(p) process {Y t } with ACVF {γ Y (h)} such that γ Y (h) = γ X (h) for h p corresponding statement does not hold for MA(q) processes (cf. AR(1) and MA(1) processes), but adding MA component to form ARMA processes increases flexibility by defining potentially useful models with small number of parameters will now extend notions introduced for ARMA(1,1) model to higher order ARMA models VIII 1

2 ARMA Models: II {X t } is said to be an ARMA(p, q) process if it is stationary and if, for t Z, X t φ 1 X t 1 φ p X t p = Z t + θ 1 Z t θ q Z t q, where {Z t } WN(0, σ 2 ), and the polynomials 1 φ 1 z φ p z p and 1 + θ 1 z + + θ q z q have no common roots (factors) in above z is a complex-valued variable note: ARMA model sometimes written in 3 other ways: X t φ 1 X t 1 φ p X t p = Z t θ 1 Z t 1 θ q Z t q X t + φ 1 X t φ p X t p = Z t + θ 1 Z t θ q Z t q X t + φ 1 X t φ p X t p = Z t θ 1 Z t 1 θ q Z t q BD 83, CC 77, SS 92 VIII 2

3 ARMA Models: III polynomial condition is sometimes stated in terms of 1 φ 1 z 1 φ p z p and 1 + θ 1 z θ q z q having no common roots (as will be noted later, this formulation has one distinct advantage) to see why no common root is stipulated, recall ARMA(1,1) process X t = φx t 1 + Z t + θz t 1, for which φ + θ 0 was stipulated reason for this stipulation became clear when we considered causal stationary solution X t = Z t + (φ + θ) φ j 1 Z t j ; j=1 note that {X t } degenerates into WN model when φ + θ = 0 BD 83, CC 78, SS 93 VIII 3

4 ARMA Models: IV ARMA(1,1) polynomial condition says 1 φz & 1 + θz should not have a common root 1 φz = 0 & 1 + θz = 0 yield roots of 1/φ & 1/θ, and 1/φ 1/θ is equivalent to φ + θ 0 can write ARMA(p, q) model more compactly as φ(b)x t = θ(b)z t, with φ(z) = 1 φ 1 z φ p z p & θ(z) = 1+θ 1 z + +θ q z q (as before, B is the backward shift operator) needed conditions φ < 1 and θ < 1 on ARMA(1,1) parameters for process to be stationary, causal and invertible similarly, need conditions on φ j s and θ k s for ARMA(p,q) process to be such these can be stated as conditions on polynomials φ(z) and θ(z) BD 84, CC 78, 80, SS 94, 95 VIII 4

5 ARMA Models: V 1. there is a (unique) stationary solution to φ(b)x t = θ(b)z t if and only if φ(z) 0 for all z = 1 2. ARMA(p, q) process is causal, meaning that, for t Z, X t = ψ j Z t j = ψ(b)z t with ψ(b) = ψ j B j & ψ j <, j=0 if φ(z) 0 for all z 1 3. ARMA(p, q) process is invertible, meaning that, for t Z, Z t = π j X t j = π(b)x t with π(b) = π j B j & π j <, j=0 if θ(z) 0 for all z 1 j=0 j=0 j=0 j=0 BD 84, 85, 86, CC 78, 80, SS 94, 95 VIII 5

6 ARMA Models: VI for complex variable z = x + iy, where i 1, unit circle defined to be set of all z s such that z 2 x 2 + y 2 = 1 unit circle handily described by e iω cos (ω) + i sin (ω) as ω varies from 0 to 2π (note that e iω 2 = cos 2 (ω) + sin 2 (ω) = 1) conditions can be restated in terms of roots of φ(z) and θ(z), i.e., values z l and z m such that φ(z l ) = 0 and θ(z m ) = 0 1. stationarity: requires all roots z l of φ(z) be off the unit circle; i.e., must have z l 1 2. causality: requires all roots z l of φ(z) to be outside the unit circle; i.e., must have z l > 1 3. invertibility: requires all roots z m of θ(z) to be outside the unit circle; i.e., must have z m > 1 BD 84, 85, 86, CC 78, 80, SS 94, 95 VIII 6

7 Unit Circle, Some Roots z and Their Reciprocals 1/z y * * x VIII 7

8 ARMA Models: VII causality condition on φ(z) implies existence of inverse of filter φ(b), namely, 1 φ(b) = χ j B j = χ(b), where χ j < j=0 j=0 likewise, invertibility condition on θ(z) implies existence of inverse of filter θ(b), namely, 1 θ(b) = π j B j = π(b), where π j < j=0 j=0 BD 85, 86, CC 78, 80, SS 94, 95 VIII 8

9 ARMA Models: VIII 1. definition of ARMA process says φ(b)x t = θ(b)z t 2. causality of ARMA process says X t = ψ(b)z t 3. multiplication of above by φ(b) says φ(b)x t = φ(b)ψ(b)z t comparison of 3 & 1 says φ(b)ψ(b) = θ(b) and hence (1 φ 1 B φ p B p )(ψ 0 + ψ 1 B + ) = 1 + θ 1 B + + θ q B q ( ) expanding out left-hand side (LHS) of ( ) yields ψ 0 + ψ 1 B + ψ 2 B 2 + ψ 3 B 3 + φ 1 ψ 0 B φ 1 ψ 1 B 2 φ 1 ψ 2 B 3 φ 2 ψ 0 B 2 φ 2 ψ 1 B 3 φ 3 ψ 0 B 3 BD 85, CC 79, SS 101 VIII 9

10 now take ARMA Models: IX ψ 0 + ψ 1 B + ψ 2 B 2 + ψ 3 B 3 + φ 1 ψ 0 B φ 1 ψ 1 B 2 φ 1 ψ 2 B 3 φ 2 ψ 0 B 2 φ 2 ψ 1 B 3 φ 3 ψ 0 B 3 collect together coefficients for B, B 2, B 3,... to get ψ 0 + (ψ 1 φ 1 ψ 0 )B + (ψ 2 φ 1 ψ 1 φ 2 ψ 0 )B 2 + (ψ 3 φ 1 ψ 2 φ 2 ψ 1 φ 3 ψ 0 )B 3 + and equate with 1 + θ 1 B + θ 2 B 2 + θ 3 B 3 + (RHS of ( )): 1 = ψ 0 θ 1 = ψ 1 φ 1 ψ 0 θ 2 = ψ 2 φ 1 ψ 1 φ 2 ψ 0 θ 3 = ψ 3 φ 1 ψ 2 φ 2 ψ 1 φ 3 ψ 0 BD 85, CC 79, SS 101 VIII 10

11 ARMA Models: X rewrite as 1 = ψ 0 θ 1 = ψ 1 φ 1 ψ 0 θ 2 = ψ 2 φ 1 ψ 1 φ 2 ψ 0 θ 3 = ψ 3 φ 1 ψ 2 φ 2 ψ 1 φ 3 ψ 0. ψ 0 = 1 ψ 1 = φ 1 ψ 0 + θ 1 ψ 2 = φ 1 ψ 1 + φ 2 ψ 0 + θ 2 ψ 3 = φ 1 ψ 2 + φ 2 ψ 1 + φ 3 ψ 0 + θ 3. BD 85, CC 79, SS 101 VIII 11

12 stare at ARMA Models: XI ψ 0 = 1 ψ 1 = φ 1 ψ 0 + θ 1 ψ 2 = φ 1 ψ 1 + φ 2 ψ 0 + θ 2 ψ 3 = φ 1 ψ 2 + φ 2 ψ 1 + φ 3 ψ 0 + θ 3. to see recursive scheme for computing ψ j s: p ψ j = φ k ψ j k + θ j, j = 0, 1, 2,..., k=1 for which we need to define θ 0 = 1, θ j = 0 for j > q and ψ j = 0 for j < 0 (also take p k=1 φ kψ j k to be 0 if p = 0) BD 85, CC 79, SS 101 VIII 12

13 ARMA Models: XII now start with 1. definition of ARMA process: θ(b)z t = φ(b)x t 2. invertibility of ARMA process: Z t = π(b)x t 3. multiplication of above by θ(b): θ(b)z t = θ(b)π(b)x t comparison of 3 & 1 says θ(b)π(b) = φ(b) and hence (1 + θ 1 B + + θ q B q )(π 0 + π 1 B + ) = 1 φ 1 B φ p B p same argument as before (with φ k replaced by θ k and with θ j replaced by φ j ) leads to scheme for computing π j s: q π j = θ k π j k φ j, j = 0, 1, 2,..., k=1 where φ 0 1, φ j 0 for j > p and π j 0 for j < 0 BD 86, CC 79 VIII 13

14 Example ARMA(1,1) Process: I note: already considered in overheads VII 21 to VII 28 process takes the form X t φx t 1 = Z t + θz t 1 here φ(z) = 1 φz and θ(z) = 1 + θz roots of φ(z) = 0 and θ(z) = 0 are 1/φ and 1/θ stationary, causal & invertible if 1/φ > 1 & 1/θ > 1, i.e., φ < 1 and θ < 1 (easily checked!) have already noted ψ 0 = 1 and ψ j = (φ + θ)φ j 1 for j 1 also have π 0 = 1 and π j = (φ + θ)( θ) j 1 for j 1 next overheads show (1) plot of roots and their reciprocals and (2) one realization for specific ARMA(1,1) model X t 0.5X t 1 = Z t + 0.4Z t 1, {Z t } Gaussian WN(0, 1) BD 86, 87, CC 77, 78, SS 95, 96 VIII 14

15 Root Plot ( / = AR/MA; red/blue = root/root 1 ) y * * x VIII 15

16 Realization of ARMA(1,1) Process t x t VIII 16

17 Example B&D s AR(2) Process: I AR(2) process takes form X t φ 1 X t 1 φ 2 X t 2 = Z t invertibility trivially true: Z t = X t φ 1 X t 1 φ 2 X t 2 here φ(z) = 1 φ 1 z φ 2 z 2 (note: π 1 = φ 1 and π 2 = φ 2 ) need to find roots z 1 and z 2 to see if {X t } is stationary & causal B&D consider X t = 0.7X t 1 0.1X t 1 + Z t, for which φ(z) = 1 0.7z + 0.1z 2 = (1 0.5z)(1 0.2z) roots are thus z 1 = 2 and z 2 = 5 both z 1 and z 2 are outside the unit circle process is thus stationary & causal next overheads show plots of roots and one realization, for which {Z t } Gaussian WN(0, 1) BD 87 VIII 17

18 Root Plot (red/blue = root/root 1 ) y x VIII 18

19 Realization of B&D s AR(2) Process x t t VIII 19

20 Example B&D s AR(2) Process: II for AR(2) processes, recursive scheme for computing ψ j s, namely, 2 ψ j = φ k ψ j k + θ j, j = 0, 1, 2,..., k=1 leads to ψ 0 = 1, ψ 1 = φ 1 & ( ) ψ j = φ 1 ψ j 1 + φ 2 ψ j 2, j 2 theory of homogeneous linear difference equations says that, if roots z 1 and z 2 are distinct, have ψ j = α 1 z j 1 + α 2 z j 2, j 2 since ( ) says ψ 2 = φ φ 2 and ψ 3 = φ φ 1φ 2, can solve for α l s using ψ 2 = α 1 z1 2 + α 2 z2 2 and ψ 3 = α 1 z1 3 + α 2 z2 3 for B&D AR(2) process, get ψ j = j j, j 2 BD 87 VIII 20

21 ψ j s for B&D s AR(2) Process ψ j j VIII 21

22 Example Second AR(2) Process: I now consider X t = 0.75X t 1 0.5X t 2 + Z t, for which φ(z) = z+0.5z 2 z = 1 z i i roots z 1 and z 2 are 3 4 ± 23 4 i (complex conjugates) here z 1 = z 2 = 2, so roots are outside the unit circle process is thus stationary & causal can reexpress roots as 2e ±iω, where ω = radians (58.0 ) realizations will tend to fluctuate roughly with period 2π. ω = 6.2 next overheads show plots of roots and one realization, for which {Z t } Gaussian WN(0, 1) VIII 22

23 Root Plot (red/blue = root/root 1 ) y x VIII 23

24 Realization of Second AR(2) Process t x t VIII 24

25 Example Second AR(2) Process: II as before, ψ 0 = 1 & ψ 1 = φ 1, while φ j s for j 2 satisfy ψ j = αz j 1 + α z j 2 = α e iϕ z 1 j e iωj + α e iϕ z 1 j e iωj = 2 α cos (ωj ϕ)/ z 1 j (note: α is complex conjugate of complex variable α α e iϕ ) letting α x + iy, can also write ψ j = 2[x cos (ωj) + y sin (ωj)]/ z 1 j as before, can get ψ 2 and ψ 3 from recursive scheme, yielding two equations to solve to get x and y (and hence α and ϕ): ψ 2 = 2[x cos (2ω)+y sin (2ω)]/ z 1 2 & ψ 3 = 2[x cos (3ω)+y sin (3ω)]/ z 1 3 here get x = 0.5 & y. = and hence α = 0.59 & ϕ. = VIII 25

26 ψ j s for Second AR(2) Process ψ j j VIII 26

27 Example AR(4) Process now consider AR(4) process X t = X t X t X t X t 4 +Z t, where {Z t } Gaussian WN(0, 1) thus φ(z) = z z z z 4 polyroot function in R calculates roots as ± 0.786i and ± 0.650i, with corresponding magnitudes and thus {X t } is stationary and causal getting closed form expression for ψ j s is tedious, so opt to just compute them using recursive scheme VIII 27

28 Root Plot (red/blue = root/root 1 ) y x VIII 28

29 Realization of AR(4) Process t x t VIII 29

30 ψ j s for AR(4) Process ψ j j VIII 30

31 Aside Harmonic Processes: I reconsider stationary process of Problem 2(b): X t = Z 1 cos (ωt) + Z 2 sin (ωt), where Z 1 and Z 2 are independent N (0, 1) RVs above is an example of a harmonic process realizations of harmonic processes are qualitatively very different from those for ARMA processes (see next overhead) homework exercise: given X 1 and X 2, can write X t = 2 cos (ω)x t 1 X t 2, above resembles AR(2) process t Z Y t = φ 1 Y t 1 + φ 2 Y t 2 + Z t if we set φ 1 = 2 cos (ω), φ 2 = 1 and Z t = 0 (can achieve by stipulating {Z t } WN(0, 0)) VIII 31

32 Three Realizations of Harmonic Process (ω = π/12) x t t VIII 32

33 Aside Harmonic Processes: II since X t = 2 cos (ω)x t 1 X t 2, can perfectly predict X t given X t 1 & X t 2 note: {X t } is example of a deterministic stationary process regarding {X t } as an AR(2) process with {Z t } WN(0, 0), have φ(z) = 1 φ 1 z φ 2 z 2 = 1 2 cos (ω)z + z 2, which has roots e ±iω since φ(e iω ) = 1 2 cos (ω)e iω + e i2ω = 1 (e iω + e iω )e iω + e i2ω = 0, where we have made use of 2 cos (ω) = e iω + e iω since e ±iω 2 = cos 2 (ω) + sin 2 (ω) = 1, roots are on unit circle reconsider example ω = π/12, which has period 2π ω = 24 VIII 33

34 Root Plot for Harmonic Process y * * x VIII 34