84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this course. The special case, ARMA(,), is defined by linear difference equations with constant coefficients as follows. Definition 4.8. A TS {X t } is an ARMA(,) process if it is stationary and it satisfies X t φx t = Z t + θz t for every t, (4.3) where {Z t } WN(0, σ 2 ) and φ + θ 0. Such a model may be viewed as a generalization of the two previously introduced models: AR() and MA(). Compare AR(): X t = φx t + Z t MA(): X t = Z t + θz t ARMA(,): X t φx t = Z t + θz t Hence, when φ = 0 then ARMA(,) MA() and we denote such a process as ARMA(0,). Similarly, when θ = 0 then ARMA(,) AR() and we denote such process as ARMA(,0). Here, as in the MA and AR models, we can use the backshift operator to write the ARMA model more concisely as where φ(b) and θ(b) are the linear filters: φ(b)x t = θ(b)z t, (4.32) φ(b) = φb, θ(b) = + θb. 4.6. Causality and invertibility of ARMA(,) For what values of the parameters φ and θ does the stationary ARMA(,) exist and is useful? To answer this question we will look at the two properties of TS, causality and invertibility. The solution to 4.3, or to 4.32, can be written as X t = φ(b) θ(b)z t.
4.6. AUTOREGRESSIVE MOVING AVERAGE MODEL ARMA(,) 85 However, for φ < we have (see Remark 4.3) φ(b) θ(b) = ( + φb + φ2 B 2 + φ 3 B 3 +...)( + θb) = + φb + φ 2 B 2 + φ 3 B 3 +... + θb + φθb 2 + φ 2 θb 3 + φ 3 θb 4 +... = + (φ + θ)b + (φ 2 + φθ)b 2 + (φ 3 + φ 2 θ)b 3 +... = + (φ + θ)b + (φ + θ)φb 2 + (φ + θ)φ 2 B 3 +... = ψ j B j, where ψ 0 = and ψ j = (φ + θ)φ j for j =, 2,.... Thus, we can write the solution to 4.32 in the form of an MA() model, i.e., X t = Z t + (φ + θ) This is a stationary unique process. φ j Z t j. (4.33) Now, suppose that φ >. Then, by similar arguments as in the AR() model, it can be shown that X t = θφ Z t (φ + θ) φ j Z t+j. Here too, we obtained a noncausal process which depends on future noise values, hence of no practical value. If φ = then there is no stationary solution to 4.32. While causality means that the process {X t } is expressible in terms of past values of {Z t }, the dual property of invertibility means that the process {Z t } is expressible in the past values of {X t }. Is ARMA(,) invertible? ARMA(,) model is φ(b)x t = θ(b)z t and so writing the solution for Z t we have Z t = θ(b) φ(b)x t = + θb ( φb)x t. (4.34)
86 CHAPTER 4. STATIONARY TS MODELS Now, if θ < then the power series expansion of the function f(x) = +θx is f(x) = + θx = ( θ) j x j, what in terms of the backshift operator B can be written as Applied to 4.34 it gives Z t = + θb = ( θ) j B j. ( θ) j B j ( φb)x t = X t (φ + θ) ( θ) j X t j. The conclusion is that ARMA(,) is invertible if θ <. Otherwise it is noninvertible. The two properties, causality and invertibility, determine the admissible region for the values of parameters φ and θ, which is the square < φ < < θ <. 4.6.2 ACVF and of ARMA(,) The fact that we can express ARMA(,) as a linear process of the form X t = ψ j Z t j, where Z t is a white noise, is very helpful in deriving the ACVF and of the process. By Corollary 4. we have For ARMA(,) the coefficients ψ j are ψ 0 = γ(τ) = σ 2 ψ j ψ j+τ. ψ j = (φ + θ)φ j for j =, 2,...
4.6. AUTOREGRESSIVE MOVING AVERAGE MODEL ARMA(,) 87 and we can easily derive expressions for γ(0) and γ(). and γ() = σ 2 ψ j ψ j+ γ(0) = σ 2 = σ 2 [ = σ 2 [ ψ 2 j = σ 2 [ + + (φ + θ) 2 φ 2(j ) + (φ + θ) 2 φ 2j (φ + θ)2 φ 2. = σ [ 2 (φ + θ) + (φ + θ)(φ + θ)φ + (φ + θ)φ(φ + θ)φ 2 + (φ + θ)φ 2 (φ + θ)φ 3 +... = σ [ 2 (φ + θ) + (φ + θ) 2 φ( + φ 2 + φ 4 +...) = σ [(φ 2 + θ) + (φ + θ) 2 φ φ 2j [ = σ 2 (φ + θ) + (φ + θ)2 φ φ 2 Similar derivations for τ 2 give γ(τ) = φ τ γ(). (4.35) Hence, we can calculate the autocorrelation function ρ(τ). For τ = we obtain and for τ 2 we have ρ() = γ() (φ + θ)( + φθ) = (4.36) γ(0) + 2φθ + θ 2 ρ(τ) = φ τ ρ(). (4.37) From these formulae we can see that when φ = θ the ρ(τ) = 0 for τ =, 2,... and the process is just a white noise. Graph 4.4 shows the admissible region for the parameters φ and θ and indicates the regions when we have special cases of ARMA(,), which are white noise, AR() and MA().
88 CHAPTER 4. STATIONARY TS MODELS θ MA() AR() - φ - φ = θ WN Figure 4.4: Admissible parameter region for ARMA(,)
4.6. AUTOREGRESSIVE MOVING AVERAGE MODEL ARMA(,) 89 30 80 30 80 phi = -0.9, theta = 0.5 phi = 0.9, theta = 0.5 0 5 0-5 0 phi = -0.9, theta = -0.5 phi = 0.9, theta = -0.5-0 5 0-5 -0 30 80 30 80 Figure 4.5: ARMA(,) for various values of the parameters φ and θ.
90 CHAPTER 4. STATIONARY TS MODELS Series : SimulatedARMA$V2 Series : SimulatedARMA$V -0.5 0.0 0.5.0 0.0 0.2 0.4 0.6 0.8.0 Series : SimulatedARMA$V3 Series : SimulatedARMA$V4 -.0-0.5 0.0 0.5.0-0.2 0.0 0.2 0.4 0.6 0.8.0 Figure 4.6: of the ARMA(,) processes with the parameter values as in Figure 4.5, respectively.