Estimation for ARMA Processes with Stable Noise. Matt Calder & Richard A. Davis Colorado State University

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Estimation for ARMA Processes with Stable Noise. Matt Calder & Richard A. Davis Colorado State University"

Λάρισα Δημαράς
6 χρόνια πριν
Προβολές:

1 Estimation for ARMA Processes with Stable Noise Matt Calder & Richard A. Davis Colorado State University 1

2 ARMA processes with stable noise Review of M-estimation Examples of M-estimation LS LAD MLE Bootstrapping Simulations 2

3 ARMA(p,q) model with heavy tailed noise X t φ 1 X t-1 φ p X t-p = Z t + θ 1 Z t θ q Z t-q, a. {Z t } ~ IID(α) with Pareto tails (0 < α < 2) b. φ(z) = 1 φ 1 z φ p z p and θ(z) = 1 + θ 1 z + + θ q z q have no common zeroes and no zeroes on or inside the unit circle. Shorthand notation. φ(b)x t = θ(b)z t, {Z t } ~ IID(α) (Β=backward shift) β= (φ 1,..., φ p, θ 1,..., θ q ) T (parameter vector) 3

4 Examples of AR(2) models

6 Review of M-Estimation for ARMA Models Data. X 1,..., X n Model. (ARMA(p,q)) φ(b)x t = θ(b)z t, {Z t } ~ IID(α). Loss function. ρ( ) Criterion. Minimize where n Σ t=1 T n (β) = ρ(z t (β)) with respect to β, Z t (β ) = 0 and X t = 0, for t < 1,... Z t (β ) = φ(b)x t θ 1 Z t-1 (β ) θ q Z t-q (β ), t >0. 6

7 Result (Davis, Knight & Liu `92, Davis `95). If {Z t } ~SαS (symmetric α stable) and ψ(x)= ρ'(x) satisfies 1. Lipschitz of order τ > max(α 1,0). 2. Ε ψ(z 1 ) < if α < 1, 8 3. Ε ψ(z 1 ) = 0 and Var(ψ(Z 1 ) ) <, if α > 1, then n 1/α (β M β) d η where β M is the M-estimate of β. The limit random vector η is the minimizer of a stochastic process. 8 7

8 Εxample (MA(1)). Data. X 1,..., X n Model. X t = Z t + θ Z t-1, LAD estimation: Minimize θ 0 < 1, {Z t } ~ IID(α) n Σ t=1 n Σ t=1 T n (θ) = ( ρ(z t (θ)) - ρ(z t (θ 0 )) ) = (ρ(x t θx t-1 +θ 2 X t-2 - (-θ) t-1 X 1 ) - ρ (Z t (θ 0 ))) Set u=n 1/α (θ θ 0 ),... 8

9 S n (u) = T n (θ 0 + un -1/α ) n Σ t=1 = (ρ(z t (θ 0 + un -1/α )) - ρ(z t (θ 0 )) ) (Not a convex function of u even if ρ is convex!) Linearize Z t (θ 0 + un -1/α ) to get n Σ t=1 S n (u) (ρ(z t (θ 0 ) + un -1/α Z t ' (θ 0 )) ρ(z t (θ 0 )) ) where -Z t ' (θ 0 ) is the AR(1) process Y t = θ 0 Y t-1 +Z t. Result : u n := argmin(s n (u)) = n 1/ α (θ M θ 0 ) d η:= argmin(s (u)) 9

10 M-Estimation Examples 1. LS (least squares). ρ(x) = x 2 does not satisfy assumptions 1 3 of previous slide. However, ( n / ln n ) 1/α (β LS β) η LS d Remark: Estimation procedures which are inherently second order based will have scaling factor (n / ln n) 1/α. Examples are moment estimation (Davis & Resnick `85, `86) Yule-Walker estimation for AR's (Davis & Resnick `85, `86) Whittle estimate (and max Gaussian likelihood?) (Mikosch, Gadrich, Kluppelberg and Adler `95) 10

11 Moreover, β M β β LS β p 0 2. LAD (least absolute deviations). ρ(x) = x does not satisfy assumptions 1 3 of previous slide either. However, n 1/α (β LAD β) d η LAD 11

12 3. MLE (maximum likelihood). Suppose Z t has pdf f and ρ(x) = ln f(x). Then β MLE, which minimizes T n (β) = ln f(z t (β)), is an approximate MLE estimator. If one chooses f to be the symmetric λ stable density f λ, then ρ(x) = ln f λ (x) satisfies the assumptions of the result mentioned previously so that n Σ t=1 n 1/α d (β MLE,λ β) η λ Call β MLE,λ the maximum (λ-stable) likelihood estimate. 12

13 Remark. Can minimize n Σ t=1 T n (β) = ln f λ (Z t (β)) with respect to both λ and β to obtain pseudo-mle's of both parameters. 13

14 Bootstrapping the M-Estimate (Davis and Wu `95). Data. X 1,..., X n Model. X t = φ 1 X t φ p X t-p +Z t, {Z t } ~ IID(α) M-estimate. φ Estimated residuals. Z t =X t φ 1 X t φ p X t-p Bootstrap sample. X * t = φ 1 X * t φ p X * * t-p +Z t for t = 1,..., m n, where {Z * t }~IID(F n ), F n = empirical df of Z p+1,..., Z n. BS M-estimate. φ... 14

15 Result. If m n / n 0, then P(m 1/α n (φ φ) p X n ) P( η ). Removing the dependence on normalizing constants. Let M n = max{ X 1,..., X n } M * m = max{ X * 1,..., X * m }. Then M n (φ φ) d w and P(M * m (φ φ) p X n ) P( w ). 15

16 Simulation Comparison. Principal objectives: compare performance of β MLE,α, β LAD, and β LS. compare performance of β MLE,α, β MLE, 1, β MLE,α, and β LAD. investigate performance of the MLE estimator of α. 3 Models ({Z t } ~ SαS) AR(1) : X t =.4 X t-1 + Z t MA(1): X t = Z t +.8 Z t-1 ARMA(1,1): X t =.4 X t-1 + Z t +.8 Z t-1 Sample size = 200, replications = 10,000 16

17 α = 1.75 Model True Values β β β LAD LS MLE, α M.1 φ = (.0465).397(.0474).398(.0394) M.2 θ = (.0317).803(.0330).803(.0270) M.3 φ = (.0456).397(.0525).398(.0440) θ = (.0353).804(.0363).803(.0296) α = 1.0 Model True Values β β β LAD LS MLE, α M.1 φ = (.0073).3971(.0263).3999(.0061) M.2 θ = (.0043).8005(.0207).7997(.0039) M.3 φ = (.0083).3979(.0320).3999(.0071) θ = (.0048).8012(.0232).7996(.0048) 17

18 α =.50 Model True Values β β β LAD LS MLE, α M.1 φ = (.00058).3988(.01022).4000(.00007) M.2 θ = (.00226).8004(.01052).7576(.04242) M.3 φ = (.00142).3986(.01396).3998(.00071) θ = (.00012).8014(.01179).7529(.04715) 18

19 M.1, φ α=1.75 M.2, θ α= LAD LS MLE LAD LS MLE M.1, φ α=1.0 M.2, θ α= LAD LS MLE LAD LS MLE 19

20 M.1, φ, α=0.5 M.2, θ α= LAD LS MLE LAD LS MLE 20

21 M.1, φ α=1.75 M.2, θ α = LAD MLE Cchy Est. LAD MLE Cchy Est. M.1, φ α=0.8 M.2, θ α= LAD MLE Cchy Est LAD MLE Cchy Est. 21

22 Estimation of α M.1, φ α = M1, φ α = M.2, θ α = 0.80 M.2, θ α =

23 Robustness of the estimates. Z 1 has a Pareto tail with exponent α = 4.0. M.1, φ α =4.0 M.2, θ α = LAD LS Cauchy LAD LS Cauchy 23

24 Conclusions Parameters of an ARMA model with heavy tailed noise can be estimated quite well. LAD estimates almost as good as MLE when noise is stable. Drawbacks of MLE: computationally more difficult requires an estimate of α MLE estimation of α performs well but is not very robust. 24

Σχετικά έγγραφα

4.6 Autoregressive Moving Average Model ARMA(1,1)

84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this