122 CHAPTER 6. ARMA MODELS 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss a linear function of X (X n,x n 1,...,X 1 T predicting a future value of X n+m for m 1, 2,... We call a function f (n (X β 0 + β 1 X n +... + β n X 1 β 0 + β i X n+1 i the best linear predictor (BLP of X n+m if it minimizes the prediction error S(β E[X n+m f (n (X] 2, where β is the vector of the coefficients β i and X is the vector of variables X n+1 i. Since S(β is a quadratic function of β and is bounded below by zero there is at least one value of β that minimizes S(β. It satisfies the equations S(β β i 0, i 0, 1,...,n. Evaluation of the derivatives gives so called prediction equations S(β β 0 E[X n+m β 0 β i X n+1 i ] 0 (6.18 S(β β j E[(X n+m β 0 Assuming that E(X t µ the first equation can be written as β i X n+1 i X n+1 j ] 0 (6.19 µ β 0 β i µ 0, which gives β 0 µ(1 β i.
6.3. FORECASTING ARMA PROCESSES 123 The set of equations (6.20 gives 0 E(X n+m X n+1 j β 0 µ β i E(X n+1 i X n+1 j E(X n+m X n+1 j µ 2 (1 γ(m (1 j β i β i γ(i j β i E(X n+1 i X n+1 j That is we obtain the following form of the prediction equations (6.20. γ(m 1 + j β i γ(i j, j 1,...,n. (6.20 We obtain the same set of equations when E(X t 0. Hence, we assume further that the TS is a zero-mean stationary process. Then β 0 0 too. 6.3.1 One-step-ahead Prediction Given {X 1,...,X n } we want to forecast the value of X n+1. The BLP of X n+1 is f (n β i X n+1 i. The coefficients β i satisfy (6.21, where m 1, that is β i γ(i j γ(j, j 1, 2,...,n. A convenient way of writing these equations is using matrix notation. We have where Γ n β n γ n, (6.21 Γ n {γ(i j} j,,2,...,n β n (β 1,...,β n T γ n (γ(1,...,γ(n T. If Γ n is nonsingular than the unique solution to (6.22 exists and is equal to β n Γ 1 n γ n. (6.22
124 CHAPTER 6. ARMA MODELS Then the forecast of X n+1 based on X (X n,...,x 1 T can be written as X (n n+1 (Γ 1 n γ n T X. (6.23 The mean square one-step-ahead prediction error denoted by P (n n+1 is P (n n+1 E(X n+1 X (n n+1 2 E(X n+1 γnγ T 1 n X 2 E(Xn+1 2 2γnΓ T 1 n XX n+1 + γnγ T 1 n XX T Γ 1 n γ n γ(0 2γnΓ T 1 n γ n + γnγ T 1 n Γ n Γ 1 n γ n γ(0 γnγ T 1 n γ n. (6.24 Example 6.6. Prediction for an AR(2 Let X t φ 1 X t 1 + φ 2 X t 2 + Z t be a causal AR(2 process. Suppose we have one observation of X 1. Then onestep-ahead prediction function is f (1 β 1 X 1, where β 1 Γ 1 1 γ 1 γ(1 γ(0 ρ(1 φ 11 and we obtain X (1 2 ρ(1x 1 φ 11 X 1. To predict X 3 based on X 2 and X 1 we need to calculate β 1 and β 2 in the prediction function f (2 β 1 X 2 + β 2 X 1.
6.3. FORECASTING ARMA PROCESSES 125 These can be obtained from (6.23 as ( ( 1 ( β1 γ(0 γ(1 γ(1 γ(1 γ(0 γ(2 β 2 1 γ 2 (0 γ 2 (1 ( γ(0 γ(1 γ(1 γ(0 ( γ(1 γ(2 ( 1 γ(0γ(1 γ(1γ(2 γ 2 (0 γ 2 (1 γ 2 (1 + γ(0γ(2 γ(1(γ(0 γ(2 γ 2 (0 γ 2 (1 γ(0γ(2 γ 2 (1 γ 2 (0 γ 2 (1 ρ(1(1 ρ(2 1 ρ 2 (1 ρ(2 ρ 2 (1 1 ρ 2 (1. From the difference equations (6.17 calculated in Example 6.4 we know that That is It finally gives ρ(1 φ 1 1 φ 2 ρ(2 φ 1 ρ(1 φ 2 ρ(0 0 ρ(2 φ 1 ρ(1 + φ 2. ( β1 β 2 ( φ1 φ 2. In fact, we can obtain this result directly from the model taking X (2 3 φ 1 X 2 + φ 2 X 1 which satisfies the prediction equations, namely In general, for n 2, we have i.e., β j 0 for j 3,...,n. E[(X 3 φ 1 X 2 φ 2 X 1 X 1 ] E[Z 3 X 1 ] 0 E[(X 3 φ 1 X 2 φ 2 X 1 X 2 ] E[Z 3 X 2 ] 0. X (n n+1 φ 1 X n + φ 2 X n 1, (6.25
126 CHAPTER 6. ARMA MODELS Similarly, it can be shown that a one-step-ahead prediction for AR(p is X (n n+1 φ 1 X n + φ 2 X n 1 +... + φ p X n p+1, for n p. (6.26 Remark 6.8. An interesting connection between the PACF and vector β n is that in fact φ nn β n, the last element of the vector. For this reason, the vector β n is usually denoted by φ n in the following way β n β 1 β 2. β n φ n1 φ n2. φ nn φ n. The prediction equation (6.22 for a general ARMA(p,q model is more difficult to calculate, particularly for large values of n when we would have to calculate an inverse of matrix Γ n of large dimension. Hence some recursive solutions to calculate the predictor (6.24 and the mean square error (6.25 were proposed, one of them by Levinson in 1947 and by Durbin in 1960. The method is known as the Durbin-Levinson Algorithm. Its steps are following: Step 1 Put φ 00 0, P (0 1 γ(0. Step 2 For n 1 calculate where, for n 2 φ nn ρ(n n 1 k1 φ n 1,kρ(n k 1 n 1 k1 φ n 1,kρ(k (6.27 Step 3 For n 1 calculate φ nk φ n 1,k φ nn φ n 1,n k, k 1, 2,...,n 1. P (n n+1 P (n 1 n (1 φ 2 nn. (6.28 Remark 6.9. Note, that the Durbin-Levinson algorithm gives an iterative method to calculate the PACF of a stationary process.
6.3. FORECASTING ARMA PROCESSES 127 Remark 6.10. When we predict a value of the TS based only on one preceding datum, that is n 1, we obtain φ 11 ρ(1, and hence the predictor X (1 2 ρ(1x 1, or in general X (1 n+1 ρ(1x n and its mean square error P (1 2 γ(0(1 φ 2 11. When we predict X n+1 based on two preceding values, that is n 2, we obtain φ 22 ρ(2 φ 11ρ(1 1 φ 11 ρ(1 ρ(2 ρ2 (1 1 ρ 2 (1 which we have also obtained solving the matrix equation (6.22 for β 2, φ 21 φ 11 φ 22 φ 11 ρ(1(1 φ 22. Then the predictor is and its mean square error X (2 n+1 φ 21 X n + φ 22 X n 1 P (2 3 γ(0(1 φ 2 11(1 φ 2 22. We could continue these steps for n 3, 4,... Example 6.7. Prediction for an AR(2, continued
128 CHAPTER 6. ARMA MODELS Using the Durbin-Levinson algorithm for AR(2 we obtain φ 11 ρ(1 φ 1 1 φ 2 φ 22 ρ(2 ρ2 (1 1 ρ 2 (1 φ 2 φ 21 ρ(1(1 φ 22 φ 1 φ 33 ρ(3 φ 1ρ(2 φ 2 ρ(1 1 φ 1 ρ(1 φ 2 ρ(2 0 φ 31 φ 21 φ 33 φ 22 φ 1 φ 32 φ 22 φ 33 φ 21 φ 2 φ 44 ρ(4 φ 1ρ(3 φ 2 ρ(2 1 φ 1 ρ(1 φ 2 ρ(2 0 The results for φ 33 and φ 44 come from the fact that in the numerator we have the difference which is zero (difference equation. Hence, one-step-ahead predictor for AR(2 is based only on two preceding values, as there are only two nonzero coefficients in the prediction function. As before, we obtain the result X (2 n+1 φ 1 X n + φ 2 X n 1. Remark 6.11. The PACF for AR(2 is φ 11 φ 1 1 φ 2 φ 22 φ 2 φ ττ 0 for τ 3. (6.29 6.3.2 m-step-ahead Prediction Given values of variables {X 1,...,X n } the m-steps-ahead predictor is X (n n+m φ (m n1 X n + φ (m n2 X n 1 +... + φ (m nn X 1, (6.30
6.3. FORECASTING ARMA PROCESSES 129 where φ (m nj prediction equations are where and β j satisfy the prediction equations (6.21. In matrix notation the γ (m n Γ n φ (m n γ (m n, (6.31 (γ(m,γ(m + 1,...,γ(m + n 1 T φ (m n (φ (m n1,φ (m n2,...,φ (m nn T. The mean square m-step-ahead prediction error is P (n n+m E[X n+m X n+m] (n 2 γ(0 (γ (m T Γ 1 n n γ (m n. (6.32 The mean square prediction error assesses the precision of the forecast and it is used to calculate so called prediction interval (PI. When the process is Gaussian the the PI is X (n n+m ± u α P n+m, (n (6.33 where u α is such that P( U < u α 1 α, where U is a standard normal r.v. For α 0.5 we have u α 1.96 and the 95% prediction interval boundaries are ( X (n n+m 1.96 P n+m, (n X(n n+m + 1.96 P (n n+m. Here we have used the hat notation as usually we do not know the values of the model parameters and we have to use their estimators. We will discuss the model parameter estimation in the next section.