Introduction to the ML Estimation of ARMA processes

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Introduction to the ML Estimation of ARMA processes"

Πυθις Διαμαντόπουλος
6 χρόνια πριν
Προβολές:

1 Introduction to the ML Estimation of ARMA processes Eduardo Rossi University of Pavia October 2013 Rossi ARMA Estimation Financial Econometrics / 1

2 We consider the AR(p) model: Y t = c + φ 1 Y t φ p Y t p + ε t t = 1,..., T ε t WN(0, σ 2 ) where Y 0, Y 1,..., Y 1 p are given. Notation as a regression model Y t = z tθ + ε t with θ = (c, φ 1,..., φ p ) and z t = (1, Y t 1,..., Y t p ) : c Y 1 1 Y 0... Y 1 p θ 1. = Y T 1 Y T 1... Y T p θ p ε 1. ε T Rossi ARMA Estimation Financial Econometrics / 1

3 OLS Estimation of AR(p) The model is: The OLS estimator: y = Zθ + ε θ = (Z Z) 1 Z y = (Z Z) 1 Z (Zθ + ε) = θ + (Z Z) 1 Z ε ( ) 1 ( ) 1 1 = θ + T Z Z T Z ε OLS is no longer linear in y. Hence cannot be BLUE. In general OLS in no more unbiased. Small sample properties are analytically difficult to derive. Rossi ARMA Estimation Financial Econometrics / 1

4 OLS Estimation of AR(p) If Y t is a stable AR(p) process and ε t is a standard white noise, then the following results hold (Mann and Wald, 1943): 1 T (Z Z) Γ p ( ) 1 T T Z d ε N(0, σ 2 Γ) then consistency and asymptotic normality follows from Cramer s theorem: T ( θ θ) d N(0, σ 2 Γ 1 ) Rossi ARMA Estimation Financial Econometrics / 1

5 Impact of autocorrelation on regression results Necessary condition for the consistency of OLS estimator with stochastic (but stationary) regressors is that z t is asymptotically uncorrelated with ε t, i.e. plim ( 1 T Z ε t ) = 0: ( 1 plim θ θ = plim = Γ 1 plim T Z Z ) 1 ( 1 plim ( ) 1 T Z ε ) T Z ε OLS is no longer consistent under autocorrelation of the regression error as ( ) 1 plim T Z ε 0 Rossi ARMA Estimation Financial Econometrics / 1

6 OLS Estimation - Example Consider an AR(1) model with first-order autocorrelation of its errors Y t = φy t 1 + u t u t = ρu t 1 + ε t ε t WN(0, σ 2 ) such that Z = [Y 0,..., Y T 1 ]. Then E [ ] [ 1 1 T Z u = E T ] Y t 1 u t = 1 T t=1 E[Y t 1 (ρ(y t 1 φy t 2 ) + ε t )] t=1 since u t = ρ(y t 1 φy t 2 ) + ε t Rossi ARMA Estimation Financial Econometrics / 1

7 OLS Estimation - Example E [ ] 1 T Z u ( ) ( ) 1 = ρ E[Y 2 1 T t 1] φρ E[Y t 1 Y t 2 ] + T t=1 t=1 ( ) 1 E[Y t 1 ε t ] T t=1 = ρ [γ y (0) φγ y (1)] where γ y (h) is the autocovariance function of {Y t } which can be represented as an AR(2) process. Rossi ARMA Estimation Financial Econometrics / 1

8 MLE AR(1) For the Gaussian AR(1) process, Y t = c + φy t 1 + ε t φ < 1 the joint distribution of is the observations ε t NID(0, σ 2 ) Y T = (Y 1,..., Y T ) Y T N(µ, Σ) y (Y 1, Y 2,..., Y T ) are the single realization of Y T Rossi ARMA Estimation Financial Econometrics / 1

9 MLE AR(1) µ = µ. µ Y 1. Y T Σ = N (µ, Σ) γ 0... γ T γ T 1... γ 0 Rossi ARMA Estimation Financial Econometrics / 1

10 The p.d.f. of the sample y = (Y 1, Y 2,..., Y T ) is given by the multivariate normal density f Y (y; µ, Σ) = (2π) T 2 Σ 1 2 exp { 1 } 2 (y µ) Σ 1 (y µ) Denoting Σ = σ 2 y Ω with Ω ij = φ i j γ γ 0... γ T T 1 γ 0 Σ = = γ γ γ T 1... γ T 1 0 γ ρ(t 1) Σ = σy 2 Ω = σy ρ(t 1)... 1 Rossi ARMA Estimation Financial Econometrics / 1

11 ρ(j) = φ j Collecting the parameters of the model in θ = (c, φ, σ 2 ), the joint p.d.f. becomes: { f Y (y; θ) = (2πσy 2 ) T 2 Ω 1 2 exp 1 } 2σy 2 (y µ) Ω 1 (y µ) Collecting the parameters of the model in θ = (c, φ, σ 2 ), the sample log-likelihood function is given by L(θ) = T 2 log(2π) T 2 log(σ2 y ) 1 2 log( Ω ) 1 2σy 2 (y µ) Ω 1 (y µ) Rossi ARMA Estimation Financial Econometrics / 1

12 Sequential Factorization The prediction-error decomposition uses the fact that the ε t are independent, identically distributed: T f (ε 2,..., ε T ) = f ε (ε t ). and: t=2 [ T ] g Y (y T,..., y 1 ) = g Yt Y t 1 (y t y t 1 ) g Y1 (y 1 ) t=2 by the Markov property. We assume that the marginal density of Y 1 is that of ε 1 with E[Y 1 ] = µ = E[(Y 1 µ) 2 ] = σ 2 y = c 1 φ σ2 1 φ 2 Rossi ARMA Estimation Financial Econometrics / 1

13 Since then ε t = Y t (c + φy t 1 ) g Yt Y t 1 (y t y t 1 ) = f ε (y t y t 1 ) = f ε (ε t ) t = 2,..., T Rossi ARMA Estimation Financial Econometrics / 1

14 Hence: {[ T ] } g Y (y T,..., y 1 ) = f ε (y t y t 1 ) f ε (y 1 ) t=2 For ε t NID(0, σ 2 ), the log-likelihood is given by: L(θ) = log L(θ) = log f ε (y t y t 1 ; θ) + log f y (y 1 ; θ) t=2 { = T 2 log (2π) T 1 2 log (σ 2 ) + 1 2σ 2 { 1 2 log (σ2 y ) + 1 } 2σy 2 (y 1 µ) 2 } ε t t=2 Rossi ARMA Estimation Financial Econometrics / 1

15 where Y 1 N(µ, σy 2 ) with µ = c 1 φ and σ 2 2 y = σ2 1 φ. 2 Maximization of the exact log likelihood for an AR(1) process must be accomplished numerically. Rossi ARMA Estimation Financial Econometrics / 1

16 MLE AR(p) Gaussian AR(p): Y t = c + φ 1 Y t φ p Y t p + ε t ε t NID(0, σ 2 ) θ = (c, φ 1,..., φ p, σ 2 ) Exact MLE Using the prediction-error decomposition, the joint p.d.f is given by: [ T ] f Y (y 1, y 2,..., y T ; θ) = t=p+1 f ε (y t y t 1 ; θ) f Y1,...,Y p (y 1,..., y p ; θ) but only the p most recent observations matter f ε (y t y t 1 ; θ) = f ε (y t y t 1,..., y t p ; θ) Rossi ARMA Estimation Financial Econometrics / 1

17 MLE AR(p) The likelihood function for the complete sample is: [ T ] f Y (y 1, y 2,..., y T ; θ) = f ε (y t y t 1,..., y t p ; θ) f y (y 1,..., y p ; θ) t=p+1 With ε t NID(0, σ 2 ) [ 1 f ε (y t y t 1,..., y t p ; θ) = exp (yt c φ 1 y t 1... φ p y t p ) 2 ] 2πσ 2 2σ 2. The first p observations are viewed as the realization of a p-dimensional Gaussian variable with moments: E(Y p ) = µ p E [ (Y p µ p )(Y p µ p ) ] = Σ p Rossi ARMA Estimation Financial Econometrics / 1

18 MLE AR(p) γ 0 γ 1... γ p 1 Σ p = σ 2 γ 1 γ 0... γ p 2 V p = γ p 1 γ p 2... γ 0 [ f y (y 1,..., y p ; θ) = (2π) p 2 σ 2 Vp exp (Y ] p µ p ) Vp 1 (Y p µ p ) 2σ 2 Rossi ARMA Estimation Financial Econometrics / 1

19 MLE AR(p) The log-likelihood is: L(θ) = log f Y (y 1, y 2,..., y T ; θ) = log f ε (y t y t 1,..., y t p ; θ) + log f y (y 1,..., y p ; θ) t=p+1 = T 2 log (2π) T 2 log (σ2 ) + 1 log V 1 p 2 1 2σ 2 (Y p µ p ) Vp 1 (Y p µ p ) (y t c φ 1 y t 1... φ p y t p ) 2 2σ 2 t=p+1 The exact MLE follows from: θ = arg max L(θ) θ Rossi ARMA Estimation Financial Econometrics / 1

20 Conditional MLE AR(p) Conditional MLE = OLS Take y p = (y 1,..., y p ) as fixed pre-sample values Conditioning on Y p : θ = arg max f Y p+1,...,y T Y 1,...,Y p (y p+1,..., y T y p ; θ) θ T = arg max f ε (y t y t 1,..., y t p ; θ) θ t=p+1 L(θ) = log f Yp+1,...,Y T Y 1,...,Y p (y p+1,..., y T y p ; θ) = log f ε (ε t Y t 1 ; θ) t=p+1 = T p 2 log (2π) T p 2 log (σ 2 ) 1 2σ 2 t+p ε 2 t Rossi ARMA Estimation Financial Econometrics / 1

21 Conditional MLE AR(p) where ε t = Y t (c + φ 1 Y t φ p Y t p ) Thus the MLE of (c, φ 1,..., φ p ) results by minimizing the sum of squared residuals: arg max L(c, φ 1,..., φ p ) = arg (c,φ 1,...,φ p) min (c,φ 1,...,φ p) t=p+1 The conditional ML estimate of σ 2 turns out to be: σ 2 = 1 T p t=p+1 ε 2 t ε 2 t (c, φ 1,..., φ p ) Rossi ARMA Estimation Financial Econometrics / 1

22 Conditional MLE AR(p) The ML estimates γ = ( c, φ 1,..., φ p ) are equivalent to OLS estimates. ( c, φ 1,..., φ p ) are consistent estimators if {Y t } is stationary and T ( γ γ) is asymptotically normally distributed. The exact ML estimates and the conditional ML estimates have the same large-sample distribution. Rossi ARMA Estimation Financial Econometrics / 1

23 Conditional MLE AR(p) Asymptotically equivalent MLE of the mean-adjusted model Y t µ = φ 1 (Y t 1 µ) φ p (Y t p µ) + ε t where µ = (1 φ 1... φ p ) 1 c. OLS of (φ 1,..., φ p ) in the mean adjusted model, where µ = 1 T t=1 Y t Rossi ARMA Estimation Financial Econometrics / 1

24 Yule-Walker estimation Yule-Walker estimation of (φ 1,..., φ p ) φ 1. φ p = γ 0... γ p γ p 1... γ 0 1 γ 1. γ p where and γ h = (T h) 1 T t=h+1 µ = y = 1 T (y t y)(y t h y) t=1 y t Rossi ARMA Estimation Financial Econometrics / 1

25 MLE ARMA(p,q) Gaussian MA(q): Y t = µ + ε t + θ 1 ε t θ q ε t q ε t NID(0, σ 2 ) Conditional MLE = NLLS Conditioning on ε 0 = (ε 0, ε 1,..., ε 1 q ) = 0, we can iterate on: ε t = Y t (θ 1 ε t θ q ε t q ) for t = 1,..., T. The conditional likelihood is where θ = (µ, θ 1,..., θ q, σ 2 ). L(θ) = log f YT ε 0 =0(y T ε 0 = 0; θ) = T 2 log (2π) T 2 log (σ2 ) t=1 ε 2 t 2σ 2 Rossi ARMA Estimation Financial Econometrics / 1

26 MLE ARMA(p,q) The MLE of (µ, θ 1,..., θ q ) results by minimizing the sum of squared residuals. Analytical expressions for MLE are usually not available due to highly non-linear FOCs. MLE requires to apply numerical optimization techniques. Rossi ARMA Estimation Financial Econometrics / 1

27 MLE ARMA(p,q) Conditioning requires invertibility, i.e. the roots of 1 + θ 1 z + θ 2 z θ q z q = 0 lie outside the unit circle. For MA(1) process: t ε t = Y t µ θ 1 ε t 1 = ( θ 1 ) t ε 0 + ( θ 1 ) j [Y t j µ] j=1 Rossi ARMA Estimation Financial Econometrics / 1

28 MLE ARMA(p,q) Y t = c + φ 1 Y t φ p Y t p + ε t + θ 1 ε t θ q ε t q ε t NID(0, σ 2 ) Conditional MLE = NLLS Conditioning on Y 0 = (Y 0, Y 1,..., Y p+1 ) and ε 0 = (ε 0, ε 1,..., ε q+1 ) = 0, the sequence {ε 1, ε 2,..., ε T } can be calculated from {Y 1, Y 2,..., Y T } by iterating on: ε t = Y t (c + φ 1 Y t φ p Y t p ) (θ 1 ε t θ q ε t q ) for t = 1,..., T. Rossi ARMA Estimation Financial Econometrics / 1

29 The conditional log-likelihood is: L(θ) = log f YT Y 0,ε 0 (y T y 0, ε 0 ) = T 2 log (2π) T 2 log (σ2 ) t=1 ε 2 t 2σ 2 One option is to set the initial values equal to their expected values: Y s = (1 φ 1... φ p ) 1 c s = 0, 1,..., p + 1 ε s = 0 s = 0, 1,..., q + 1 Rossi ARMA Estimation Financial Econometrics / 1

30 Box and Jenkins (1976) recommended setting ε s to zero but y s equal to their actual values. The iteration is started at date t = p + 1, with Y 1, Y 2,..., Y p set to the observed values and ε p = ε p 1 =... = ε p q+1 = 0 Rossi ARMA Estimation Financial Econometrics / 1

Σχετικά έγγραφα

HW 3 Solutions 1. a) I use the auto.arima R function to search over models using AIC and decide on an ARMA(3,1)

HW 3 Solutions 1. a) I use the auto.arima R function to search over models using AIC and decide on an ARMA(3,1) HW 3 Solutions a) I use the autoarima R function to search over models using AIC and decide on an ARMA3,) b) I compare the ARMA3,) to ARMA,0) ARMA3,) does better in all three criteria c) The plot of the