On Second-Order Error Terms In Fully Exponential Laplace Approximations.

Transcript

1 O Secod-Order Error Terms I Fully Expoetial Laplace Approximatios. Yoichi MIYATA Revised Jue 23, 2009 Revised Sep. 1, 2009 Abstract Posterior meas ca be expressed as the ratio of itegrals, which is called fully expoetial form. To approximate the posterior meas aalytically, Laplace s method might be useful. I this article, we preset explicit error terms of order 1, ad of order 2 i the Laplace approximatios with asymptotic modes. Moreover, we give secod-order error terms i fully expoetial Laplace approximatios to posterior meas with asymptotic modes, which are proposed by Miyata Itroductio Laplace s method for the asymptotic evaluatio of itegrals Laplace, 147 is a simple ad useful techique. This method has bee applied frequetly i statistical theory by may authors Mosteller ad Wallace 1964; Lidley 190; Tierey ad Kadae 196. Θ = Θ 1 Θ d R d is a ope parameter space of θ θ 1,..., θ d T, where T deotes the traspose of a matrix. Suppose that Ω, A is a measurable space, P = {P θ : θ Θ} is a family of probability distributios o Ω, A, ad {X i : i = 1, 2,...} is a stochastic process o Ω, A with X i s takig values i X, B where X is a subset of R, ad B is the class of Borel subsets of X. We will assume that for all, the distributios of X = X 1,..., X are domiated by a σ-fiite measure, ad we will deote a desity of X uder P θ by p x θ. Note that x = x 1, x 2,..., x,... is the observed sequece ad p x θ depeds o the first observatio x = x 1,..., x. Let θ 0 be a true parameter, ad let a.e. P θ0 be abbreviated to a.e. or omitted. Let g + θ be a smooth ad strictly positive fuctio. The purpose of this article is to give first-order ad secod-order error terms i the fully expoetial Laplace approximatio to the posterior mea E[g + Θ Θ] = g+ θp x θπθdθ p, 1.1 Θ x θπθdθ where Θ is a ope subset of R d, p x θ is the likelihood, ad π is a prior. Although Θ also deotes a radom vector with a posterior distributio, we ca see from the cotext which Θ idicates. For coveiece, the itegrals i 1.1 are reexpressed as E[g + Θ] = Θ exp{ h θ}dθ exp{ h Θ θ}dθ, 1.2 where h θ = 1 log[g + θp x θπθ] ad h θ = 1 log[p x θπθ]. Faculty of Ecoomics, Takasaki City Uiversity of Ecoomics, 1300 Kamiamie, Takasaki, Gumma, , Japa. ymiyatagbt@tcue.ac.jp 1

2 Sectio 2 sketches out the cocept of asymptotic modes, ad describes the Laplace approximatios with asymptotic modes. I particular, the first-order ad secod-order errors are give i a explicit form. I Sectio 3, we give the explicit error terms of order 2 i a fully expoetial Laplace approximatio to a posterior mea of g Asymptotic modes ad Laplace s approximatios This sectio itroduces the Laplace method for a itegral of the form Θ e hθ dθ with a asymptotic mode of h θ. For coveiece of expositio, we write s θ i1 θ is h θ h i1 is θ, d d, i 1 i s i 1 =1 i s =1 ad the first ad secod derivatives are represeted by D 1 h θ θ h θ = h θ,, h θ T θ 1 θ d 2 D 2 h θ 2 θ θ θ h 1 θ 1 h θ 2 θ 1 θ d h θ θ =. T θ d θ 1 h θ 2 θ d θ d h θ We say that ˆθ is a asymptotic mode of h if ˆθ coverges to the exact mode of h θ as the sample size teds to ifiity. The exact mode of h θ is deoted by ˆθ EX. Note that sice ˆθ EX satisfies D 1 h ˆθ EX = 0, it is also take as a asymptotic mode for h. Additioally, the followig asymptotic modes are defied. Defiitio 1 ˆθ is a asymptotic mode of order 1 for h if ˆθ ˆθ EX 0 a.e., ad D 1 h ˆθ = O 1 a.e. Remark 1. Let h θ = 1 log[p x θπθ]. The maximum likelihood estimator ˆθ ML for p x θ is a asymptotic mode of order 1 for h because D 1 h ˆθ ML = 1 πˆθ ML = O 1, ad ˆθ ML coverges to the exact mode of h θ, as teds to ifiity uder suitable coditios. For the covergece of ˆθ ML, see Heyde ad Johstoe Defiitio 2 ˆθ is a asymptotic mode of order 2 for h if ˆθ ˆθ EX 0 a.e., ad D 1 h ˆθ = O 2 a.e. Remark 2. Let ˆθ ML be the maximum likelihood estimator for px θ, ad let h θ = 1 log[px θπθ]. The, it follows from the same argumet as i the proof of Theorem 3 of Miyata 2004 that ˆθ ML ˆθ ML [D 2 h ˆθ ML ] 1 D 1 h ˆθ ML satisfies D 1 h ˆθ ML = O 2. Subsequetly, we itroduce the regularity coditios A1, A2, A3, A4 ad A5 for which the asymptotic expasios for Θ e h θ dθ will be valid. Let a a T a 1/2 for ay vector 2

3 a, deote the determiat of a matrix. We use B δ ˆθ to deote the ope ball of radius δ cetered at ˆθ, amely B δ ˆθ = {θ Θ : θ ˆθ < δ}. Let {ˆθ} {ˆθ : = 1, 2,...} be the sequece of asymptotic modes. We list the followig assumptios for {h θ}, {ˆθ}: A1 {h θ : = 1, 2,...} is a sequece of eight times cotiuously differetiable real fuctios o Θ. There exists positive umbers ɛ, M, ζ ad a iteger 0 such that for the asymptotic mode ˆθ, 0 implies A2 the itegral Θ e h θ dθ is fiite; A3 for all θ B ɛ ˆθ ad all 1 j 1,..., j m d with m = 1,...,, h θ < M ad m h θ/ θ j1 θ jm < M; A4 D 2 h ˆθ is positive defiite ad D 2 h ˆθ > ζ; A5 for all δ for which 0 < δ < ɛ, B δ ˆθ Θ ad D 2 h ˆθ 1/2 C ˆθ 1 exp{ [h θ h ˆθ]}dθ = O 3, Θ B δ ˆθ where C ˆθ = exp 2 D1 hˆθ T [D 2 hˆθ] 1 D 1 hˆθ. The pair {h }, {ˆθ} will be said to satisfy the aalytical assumptios for the asymptotic-mode Laplace method if A1, A2, A3, A4 ad A5 are satisfied for the asymptotic mode ˆθ. Our coditios A1 A5 are aalogous to those of Kass, Tierey ad Kadae 1990 except ˆθ is a asymptotic mode. If ˆθ is a asymptotic mode of order 1, Θ B δ ˆθ exp{ [h θ h ˆθ]}dθ = O 3 d/2 holds uder A5. Hece A5 meas that the probability outside a eighborhood of the ˆθ coverges to zero as the sample size teds to ifiity. Let h j1 jm deote the mth partial derivative m h θ/ θ j1 θ jm with respect to θ evaluated at ˆθ, for example, h 112 meas 3 h ˆθ/ ˆθ 2 1 ˆθ 2. Let h ij be the compoets of [D 2 hˆθ] 1 ad b = b i [D 2 h ˆθ] 1 D 1 h ˆθ. We defie the sixth, eighth, teth, ad twelfth cetral momets of a multivariate ormal distributio havig covariace matrix [D 2 hˆθ] 1 as µ, µ tu, µ tuvw, ad µ tuvwxy, respectively. Theorem 3. Suppose that ˆα is a asymptotic mode of order 1 for h ad the pair {h α}, {ˆα} satisfies aalytical assumptios for the asymptotic-mode Laplace method. The it follows that for large, e hθ dθ = 2π d/2 Σ 1/2 e hˆα C ˆα 1 + λ 1 Θ + λ O 3, 2.1 where C ˆα = exp 2 D1 h ˆα T [D 2 h ˆα] 1 D 1 h ˆα, 3

4 λ 1 = 1 h ˆαb i h jk 1 h q ˆαh ij h kq q h ˆαh qrs ˆαµ 3, λ 2 = 3 h b i b j b k 2 hq h ij b k b q + 4 h h qrs b i b j µ kqrs h h qrs b i b q µ jkrs 3 hqr b i µ jkqr 24 3 h µ + 4 h h qrst b i µ jkqrst h h qrst b q µ rst + 4 hq h rstu µ tu h h qrstu µ tu 5 h h qrs h tuv b i µ jkqrstuv h h qrs h tuvw µ tuvw h h qrs h tuv h wxy µ tuvwxy, Σ [D 2 hˆα] 1 = 1 h ij, ad µ are the sixth cetral momets of a multivariate Normal distributio havig covariace matrix Σ. Note that λ 1 ad λ 2 are of order O1. The proof is give i Appedix C. Theorem 4. Suppose that ˆθ is a asymptotic mode of order 2 for h ad the pair {h θ}, {ˆθ} satisfies aalytical assumptios for the asymptotic-mode Laplace method. The it follows that for large, e hθ dθ = 2π d/2 Σ 1/2 e hˆθ C ˆθ 1 + a 1 Θ + a O 3, 2.3 where C ˆθ = exp 2 D1 h ˆθ T [D 2 h ˆθ] 1 D 1 h ˆθ, a 1 = 1 a 2 = q ad Σ [D 2 hˆθ] 1 = 1 h ij. h q h ij h kq + 3 h h qrs µ, h 2 b i h jk 3 h µ 0 hq h rstu µ tu + 4 h h qrstu µ tu 0 h h qrs h tuvw µ tuvw h h qrs h tuv h xyz µ tuvwxy, 4

5 a 1, ad a 2 are of order O1, ad the proof is give i Appedix C. Remark 5. O 3 icludes h i,..., h t, h ij, b i, exp 1! B δ ˆθ 2 θ yt Σ 1 θ y h tu γ 1 z i z j z k z q z r z s z t z u dθ, exp{ [h θ h ˆθ]}dθ Θ B δ ˆθ ad θ ˆθ + b, [D 2 h ˆθ] 1 R 5 k θ, ˆθdθ, B δ ˆθ where γ 1 is a poit betwee θ ad ˆθ, k θ, ˆθ 1 = h z i z j z k ! γ 2 is a poit betwee θ ad ˆθ R 5 k θ, ˆθ = k θ, ˆθ 5 4! 1 3. Mai results 0 t 1 λ 4 exp{λk θ, ˆθ}dλ. h t z i z t + 1! tu h tu γ 2 z i z u This sectio maily gives the explicit error terms of order 2 i the fully expoetial Laplace approximatios to posterior meas. Let b = b i [D 2 h ˆθ] 1 D 1 h ˆθ Theorem 6. Let ˆθ be a asymptotic mode of order 2 for h θ, ad ˆθ N is a sigle Newto step from ˆθ toward the maximum of h θ, i.e., ˆθ N ˆθ [D 2 h ˆθ] 1 D 1 h ˆθ. If pairs {h θ}, {ˆθ} ad {h θ}, {ˆθ N } satisfy aalytical assumptios for the asymptotic-mode Laplace method, The D 2 1/2 h ˆθ C E[gΘ] = ˆθ N exp[ h ˆθ N ] 1 + c D 2 h ˆθ N C ˆθ exp[ hˆθ] + 2 O 3, 3.1 where C ˆθ N = exp{/2d 1 h ˆθ N T [D 2 h ˆθ N ] 1 D 1 h ˆθ N } ad c = 2 h b i h jk + 2 h b i h jk + 1 hq h lαβ h ij h kα h βq h lm G m hq h ij h kα h βq G αβ 1 hql h ij h kq h lm G m + 1 h ij h kq G q hα h qrs h αβ [ 3 µ ]G β 1 hqrs [ 3 µ ]G h h qrs h lαβ h jk h rs h iα h βq h lm G m + 1 h h qrs h jk h rs h iα h βq G αβ αβlm αβlm αβlm h h qrs h lαβ h jr h ks h iα h βq h lm G m h h qrs h lαβ h kq h rs h iα h βj h lm G m αβ αβ αβ h h qrs h jr h ks h iα h βq G αβ h h qrs h kq h rs h iα h βj G αβ. 5

6 Note that c is of order O1. The first term i the right side of c is reexpressed as 2 2 h b i h jk = 1 4 lmrαβ h h lαβ h il h jk h αr h βm G r G m + O If ˆθ is replaced with the exact mode ˆθ EX of h θ i 3.1, the it follows that D 2 1/2 h ˆθ EX E[gΘ] = Cˆθ N exp[ h ˆθ N ] 1 + ċ D 2 h ˆθ N exp[ hˆθ EX ] + 2 O 3, 3.3 where ċ is c with the term 2 /2 h b i h jk beig 0. Furthermore, the approximatio with ˆθ N with the exact mode of h is equivalet to the Tierey Kadae approximatio. As a geeralized form, the followig result holds. Theorem 7. Suppose that ˆθ ad θ are asymptotic modes of order 2 for h θ ad h θ respectively. The D 2 1/2 h ˆθ C E[gΘ] = θ exp[ h θ] 1 + O D 2 h θ C ˆθ exp[ hˆθ] Appedix A: Lemmas cocerig matrices This sectio prepares some lemmas cocerig matrices to prove the mai results. Lemma. Suppose that a is a 1 d matrix, M is a d d symmetric matrix. The, where y = ˆθ 1/2M 1 a. Proof. a T θ ˆθ + θ ˆθ T Mθ ˆθ = θ y T Mθ y 1 4 at M 1 a, θ ˆθ M 1 a T Mθ ˆθ M 1 a 1 4 at M 1 a = θ ˆθ T Mθ ˆθ at M 1 Mθ ˆθ + θ ˆθ T M 1 2 M 1 a at M 1 M 1 2 M 1 a 1 4 at M 1 a = a T θ ˆθ + θ ˆθ T Mθ ˆθ. Lemma 9. Let I d be a d d uit matrix. For ay d d matrix A ad ay scalar x, A + xi d = d r=0 x r {i 1,...,i r } A {i 1,...,i r }, A.1 where deotes the determiat of a matrix, {i 1,..., i r } is a r-dimesioal subset of the first d positive itegers 1,..., d ad the secod summatio is over all d r such subsets, ad A {i 1,...,i r } is the d r d r pricipal submatrix of A obtaied by strikig out the i 1,..., i r th rows ad colums. For r = d, the sum {i 1,...,i r} A{i 1,...,i r } is to be iterpreted as 1. Note that the expressio A.1 is a polyomial i x, the coefficiet of x 0 i.e., the costat term of the polyomial equals A, ad the coefficiet of x d 1 equals tra. 6

7 Proof. See Harville 1997; corollary , p.197. Lemma 10. For d d osigular matrices A ad B, d A t/b = 1 dt A t=0 trba 1. Proof. Usig Lemma ad the relatio betwee a iverse matrix ad the cofactors, we have d A t/b dt A t=0 = d BA 1 AB 1 t d dt I t=0 = d BA 1 AB 1 t AB 1 {i1} + t2 dt ξ + + td t=0 2 d = 1 1 AB 1 {i1} AB 1 {i 1 } = 1 trab 1 1 = 1 trba 1, {i 1 } where ξ = {i 1,i 2 } AB 1 {i 1,i 2 } ad {i 1 } ad {i 1, i 2 } are the same otatios as i Lemma. Hece this lemma is proved. Lemma 11. For d d osigular matrices A ad B ad ay scalar x, it follows that A + xb 1 A 1 = Ox. A.2 Proof. Multiplyig A + xb 1 A + xb = I d by A 1 from the right side yields Hece the lemma is proved. A + xb 1 A 1 = xa + xb 1 BA 1. A.3 Lemma 12. For d d osigular matrices A ad B, Proof: Usig A.3, d dt A t B 1 t=0 = 1 A 1 BA 1. A t/b 1 A 1 Hece, lettig t 0 yields the lemma. t = 1 A t B 1 BA 1. A.4 7

8 Appedix B: Some lemmas This sectio presets some lemmas required i the evaluatio of the asymptotic errors i Sectios 2 ad 3. Lemma 13. Let deote a sample size, ad let z i Θ i ˆθ i. Suppose that Θ = Θ 1,..., Θ d T is accordig to N d ˆθ + b, Σ, where ˆθ = ˆθ 1,..., ˆθ d T, b = b 1,..., b d T = O 1, ad Σ = µ ij is the covariace matrix with µ ij = O 1. The we have the followig results. a E[Θ i ˆθ i Θ j ˆθ j Θ k ˆθ k ] = b i µ jk + b j µ ik + b k µ }{{ ij + b } i b j b }{{} k O 2 O 3 b E[Θ i ˆθ i Θ j ˆθ j Θ k ˆθ k Θ q ˆθ q ] = µ ij µ kq + µ ik µ jq + µ iq µ }{{ kj + b } m1 b m2 µ m3 m }{{} 4 O 2 [m 1,m 2 ] 4 O 3 +b i b j b k b q, where [m 1, m 2 ] 4 idicates ways of a pair chose amog four alphabets i, j, k, ad q. Hece the summatio [m1,m2] 4 is over 4 2. c E[z i z j z k z q z r ] = b m1 µ m2 m 3 m 4 m 5 + b }{{} l1 b l2 b l3 µ l4 l 5 +b }{{} i b j b k b q b r, [m 1 ] 5 [l 1,l 2,l 3 ] 5 O 3 O 4 where [m 1 ] 5 idicates ways of oe chose amog five alphabets i, j, k, q, ad r, ad [l 1, l 2, l 3 ] 5 idicates ways of a triple chose amog i, j, k, q, ad r. The otatios [l 1, l 2, l 3, l 4 ] 6, [m 1, m 2 ] 6, etc below are also defied similarly. d E[z i z j z k z q z r z s ] = µ } + b {{} m1 b m2 µ m3 m 4 m 5 m }{{} 6 O 3 [m 1,m 2 ] 6 O 4 + b l1 b l2 b l3 b l4 µ l5 l 6 +b }{{} i b j b k b q b r b s [l 1,l 2,l 3,l 4 ] 6 O 5 e E[z i z j z k z q z r z s z t ] = b m1 µ m2 m 3 m 4 m 5 m 6 m }{{} 7 [m 1 ] 7 O 4 + b l1 b l2 b l3 µ l4 l 5 l 6 l 7 + [l 1,l 2,l 3 ] 7 f E[z i z j z k z q z r z s z t z }{{ u ] = µ } tu +O 5 }{{} O 4 g E[z i z j z k z q z r z s z t z u z }{{} v ] = b m1 µ m2 m 3 m 4 m 5 m 6 m 7 m m 9 +O 6 }{{} 9 [m 1 ] 9 O 5 h E[z i z j z k z q z r z s z t z u z v z }{{ w ] = µ } tuvw +O 6 }{{} 10 O 5 i E[z i z j z k z q z r z s z t z u z v z w z x z }{{} y ] = µ tuvwxy +O 7 }{{} 12 O 6 [ 1, 2, 3, 4, 5 ] 7 b 1 b 2 b 3 b 4 b 5 µ b i b j b k b q b r b s b t

9 Proof of a. Puttig u i Θ i ˆθ i + b i, E[u i ] = 0, ad E[u i u j ] = µ ij. By usig the biomial theorem ad the result i which odd momets of u i are 0, we have Proof of b. E[Θ i ˆθ i Θ j ˆθ j Θ k ˆθ k ] [ Θi ] = E ˆθ i + b i + b i Θj ˆθ j + b j + b j Θk ˆθ k + b k + b k = E[u i + b i u j + b j u k + b k ] = E[u i u j u k ] + b i E[u j u k ] + b j E[u i u k ] + b k E[u i u j ] + b i b j E[u k ] + b j b k E[u i ] + b i b k E[u j ] + b i b j b k = b i µ jk + b j µ ik + b k µ ij + b i b j b k. E[Θ i ˆθ i Θ j ˆθ j Θ k ˆθ k Θ q ˆθ q ] [ Θi ] = E ˆθ i + b i + b i Θj ˆθ j + b j + b j Θk ˆθ k + b k + b k Θq ˆθ q + b q + b q = E[u i + b i u j + b j u k + b k u q + b q ] = E[u i u j u k u q ] + b m1 E[u m2 u m3 u m4 ] + b }{{} m1 b m2 E[u m3 u m4 ] [m 1 ] 4 0 [m 1,m 2 ] 4 + b m1 b m2 b m3 E[u m4 ] +b }{{} i b j b k b q [m 1,m 2,m 3 ] 4 0 = µ q + b m1 b m2 E[u m3 u m4 ] + b i b j b k b q. [m 1,m 2 ] 4 Proof of c. E[z i z j z k z q z r ] = E[Θ i ˆθ i + b i + b i Θ j ˆθ j + b j + b j Θ r ˆθ r + b r + b r ] = E[u i + b i u j + b j u k + b k u q + b q u r + b r ] = µ } qr + b {{} m1 E[u m2 u m3 u m4 u m5 ] + b m1 b m2 E[u m3 u m4 u m5 ] }{{} 0 [m 1 ] 5 [m 1,m 2 ] b m1 b m2 b m3 E[u m4 u m5 ] + b m1 b m2 b m3 b m4 E[u m5 ] + b i b j b k b q b r [m 1,m 2,m 3 ] 5 [m 1,m 2,m 3,m 4 ] 5 = b m1 µ m2 m 3 m 4 m 5 + b m1 b m2 b m3 µ m4 m 5 + b i b j b k b q b r. [m 1 ] 5 [m 1,m 2,m 3 ] 5 9

10 Proof of d. E[z i z j z k z q z r z s ] = E[u i + b i u j + b j u k + b k u q + b q u r + b r u s + b s ] = µ + b m1 E[u m2 u m3 u m4 u m5 u m6 ] }{{} [m 1 ] b m1 b m2 E[u m3 u m4 u m5 u m6 ] + b m1 b m2 b m3 E[u m4 u m5 u m6 ] [m 1,m 2 ] 6 [m 1,m 2,m 3 ] 6 + b m1 b m2 b m3 b m4 E[u m5 u m6 ] + b m1 b m2 b m3 b m4 b m5 E[u m6 ] [m 1,m 2,m 3,m 4 ] 6 [m 1,m 2,m 3,m 4,m 5 ] 6 Similarly, e h ca be proved. + b i b j b k b q b r b s. Let Gθ log g + θ ad let G j1...j m deote the mth partial derivatives m Gθ/ θ j1...θ jm evaluated at θ = ˆθ. For example, G 213 = 3 Gˆθ/ ˆθ 2 ˆθ 1 ˆθ 3. For simplicity, we let h = h ad h = h. Lemma 14. Let ˆθ be a asymptotic mode of order 2 for hθ, ad let h θ hθ 1/Gθ. Suppose that h θ ad hθ are five times cotiuously differetiable sequeces o Θ, Gθ is four times cotiuously differetiable, ad [D 2 hˆθ] 1 exists. The the followig results hold: a h ˆθ N hˆθ = 1 Gˆθ + 1 D1 hˆθ T [D 2 hˆθ] 1 D 1 Gˆθ +O 2 }{{} O 3 b D 1 h ˆθ N D 1 hˆθ = O 2 c h ijˆθ N h ij ˆθ = 1 D1 h ij ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 G ijˆθ + O 2 = 1 h αij h αβ G β 1 G ij + O 2 α,β d h ˆθ N h ˆθ = 1 D1 h ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 G ˆθ + O 2 = 1 h α h αβ G β 1 G + O 2 αβ e h qˆθ N h q ˆθ = 1 D1 h q ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 G qˆθ + O 2 = 1 h αq h αβ G β 1 G q + O 2 αβ f h ij ˆθ N h ij ˆθ = 1 D1 h ij ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ + O 2 = 1 h iα h lαβ h βj h lm G m + O 2 αβlm 10

11 g h ij ˆθ N h ij ˆθ = 1 h iα h lαβ h βj h lm G m + 1 h iα G αβ h βj + O 2 α,β,l,m Proof of a. From Dh ˆθ = Dhˆθ 1/DGˆθ = 1/DGˆθ + O 2 α,β ˆθ N ˆθ = [D 2 h ˆθ] 1 D 1 h ˆθ B.1 = 1 [D2 hˆθ] 1 D 1 Gˆθ + O 2 B.2 It follows from the result B.2 that h ˆθ N hˆθ = hˆθ N hˆθ 1 Gˆθ N = D 1 hˆθ T ˆθ N ˆθ 1 Gˆθ N + O 2 { } 1 = D 1 hˆθ T [D2 hˆθ] 1 D 1 Gˆθ 1 Gˆθ + O 2 = 1 D1 hˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 Gˆθ + O 2. Proof of b. Because ˆθ N is a asymptotic mode of order 2 for h, this is immediate from Miyata 2004, p Proof of c. Because D 2 h ˆθ N = D 2 hˆθ N 1/D 2 Gˆθ N, arguig as i the proof of a of Lemma 13, we have h ijˆθ N h ij ˆθ = h ij ˆθ N h ij ˆθ 1 G ijˆθ N = D 1 h ij ˆθ T ˆθ N ˆθ 1 G ijˆθ N + O 2 { } 1 = D 1 h ij ˆθ T [D2 hˆθ] 1 D 1 Gˆθ 1 G ijˆθ + O 2 = 1 D1 h ij ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 G ijˆθ + O 2 = 1 h αij h αβ G β ˆθ 1 G ijˆθ + O 2. α,β Proof of d. Arguig as i the proof of a of Lemma 13, h ˆθ N h ˆθ = h ˆθ N h ˆθ 1 G ˆθ N = D 1 h ˆθ T ˆθ N ˆθ 1 G ˆθ N + O 2 { } 1 = D 1 h ˆθ T [D2 hˆθ] 1 D 1 Gˆθ 1 G ˆθ + O 2 = 1 D1 h ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ 1 G ˆθ + O 2 = 1 h α h αβ G β ˆθ 1 G ˆθ + O 2. Proof of e. This is the same as that of d essetially. 11 α,β

12 Proof of f. It follows from B.2 that h ij ˆθ N h ij ˆθ = D 1 h ij ˆθ T ˆθ N ˆθ + O 2 = 1 D1 h ij ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ + O 2 B.3 By usig the matrix algebra D.A. Harville p.30, ˆθ [D 2 hˆθ] 1 = [D 2 hˆθ] 1 k ˆθ D 2 hˆθ[d 2 hˆθ] 1. k Hece, Therefore, ˆθ k h ij ˆθ = α,β h iα h kαβ h βj. D 1 h ij ˆθ = α,β h iα h βj D 1 h αβ ˆθ. Thus h ij ˆθ N h ij ˆθ = 1 αβ h iα h βj D 1 h αβ ˆθ T [D 2 hˆθ] 1 D 1 Gˆθ + O 2 = 1 h iα h kαβ h βj h kl G l + O 2. αβkl Proof of g. It follows from A.3 that [D 2 h ˆθ N ] 1 [D 2 hˆθ] 1 = [D 2 hˆθ N 1 D2 Gˆθ N ] 1 [D 2 hˆθ] 1 = [D 2 hˆθ N ] 1 [D 2 hˆθ] [D2 h ˆθ N ] 1 D 2 Gˆθ N [D 2 hˆθ N ] 1 = [D 2 hˆθ N ] 1 [D 2 hˆθ] [D2 hˆθ] 1 D 2 Gˆθ[D 2 hˆθ] 1 + O 2. Usig f of this lemma, we have h ij ˆθ N h ij ˆθ = h ij ˆθ N h ij ˆθ + 1 h h 1j i1 h id D 2 Gˆθ. + O 2 = 1 h dj h iα h lαβ h βj h lm G m + 1 h iα G αβ h βj + O 2. αβlm The followig expasios are used to derive the fully expoetial Laplace approximatios i Sectio 3. αβ 12

13 Lemma 15. Suppose that ˆθ is a asymptotic mode of order 2 for hˆθ. Let µ be sixth cetral momets of a multivariate Normal distributio with mea 0 ad covariace Σ = [D 2 h ˆθ N ] 1. Let k 5k 6 = αβlm h k 5α h lαβ h βk 6 h lm G m αβ h k 5α h βk 6 G αβ. The the followig results hold. a 3 µ µ = 1 <k 1,k 2 k 6 > h k 1k 2 h k 3k 4 k 5k 6 + O 2, where A, A, ad B are three rooms i which two rooms are the same, ad the other is differet from them, ad < k 1, k 2,..., k 6 > deotes ways to arrage elemets i, j, k, q, r ad s ito the three rooms A, A, ad B by two alphabets. Hece the summatio <k 1,k 2 k 6 > is over /2 = 45 terms. b = h h qrs 3 µ µ αβlm αβlm αβlm + O 2. h h qrs h lαβ h jk h rs h iα h βq h lm G m + 9 h h qrs h lαβ h jr h ks h iα h βq h lm G m + 1 h h qrs h lαβ h kq h rs h iα h βj h lm G m + 1 αβ αβ αβ h h qrs h jk h rs h iα h βq G αβ h h qrs h jr h ks h iα h βq G αβ h h qrs h kq h rs h iα h βj G αβ c 3 {h ˆθ N h ˆθ}h qrs µ d h l ˆθ N = Proof of a. = 1 αβ h α h qrs h αβ 3 µ G β h lαβ h αr h βm G r G m + O 3. rmαβ h qrs 3 µ G + O 2. It is well kow that sixth cetral momets of the Normal distributio are decomposed with elemets of the covariace matrix i.e., µ = 1 h ij h kq h rs + h ij h kr h qs + + h is h jr h kq 3 }{{}, B.4 15 terms where h ij are i, j-elemets of [D 2 h ˆθ N ] 1. For example, see Kass et al. 1990, p.477. From g of Lemma 13, h ij ˆθ N = h ij ˆθ ij + O 2. 13

14 The we have h ij ˆθ N h kq ˆθ N h rs ˆθ N = h ij ˆθh kq ˆθh rs ˆθ 1 hij ˆθh kq ˆθ h rα h lαβ h βs h lm G m αβlm αβ 1 hij ˆθh rs ˆθ h kα h lαβ h βq h lm G m αβlm αβ 1 hkq ˆθh rs ˆθ h iα h lαβ h βj h lm G m αβlm αβ h rα h βs G αβ h kα h βq G αβ h iα h βj G αβ + O 2 = h ij ˆθh kq ˆθh rs ˆθ 1 hij ˆθh kq ˆθ rs 1 hij ˆθh rs ˆθ kq where 1 hkq ˆθh rs ˆθ ij + O 2, ij = α,β,l,m h iα h lαβ h βj h lm G m α,β h iα h βj G αβ. B.5 B.6 Substitutio of B.5 ito the decompositio B.4 leads to the equatio a. Proof of b. From Lemma 14 a, we have = 1 = 1 h h qrs 3 µ µ h h qrs h } ij h kq rs + h ij kq h rs + {{ ij h kq h rs + + h is h jr kq } h h qrs <k 1 k 2 k 6 > 45terms h k 1k 2 h k 3k 4 k 5k 6 + O 2. + O 2 B.7 Equatio B.7 are divided ito three kids of terms, h h qrs iq h jk h rs, h h qrs iq h jr h ks, ad h h qrs ij h kq h rs as described below. Note that h h qrs h ij h kq rs ad h h qrs ij h kq h rs are the same if the suffixes i ad j are exchaged for r ad s. The secod summatio i B.7 is 14

15 <k 1 k 2 k 6 > h k 1k 2 h k 3k 4 k 5k 6 = h ij h kq rs + h ij h kr qs + h ij h ks qr + h ik h jq rs + h ik h jr qs + h ik h js qr + h iq h jk rs + h iq h jr ks + h iq h js kr + h ir h jk qs + h ir h jq ks + h ir h js kq + h is h jk qr + h is h jq kr + h is h jr kq + h ij kq h rs + h ij kr h qs + h ij ks h qr }{{} + h ik jq h rs + h ik jr h qs + h ik js h qr }{{} +hiq jk h rs + h iq jr h ks + h iq js h kr + h ir jk h qs + h ir jq h ks + h ir js h kq + h is jk h qr + h is jq h kr + h is jr h kq + ij h kq h rs + ij h kr h qs + ij h ks h qr + ik h jq h rs + ik h jr h qs + ik h js h qr + iq h jk h rs }{{} + iq h jr h ks + iq h js h kr + ir h jk h qs }{{} + ir h jq h ks + ir h js h kq + is h jk h qr }{{} + is h jq h kr + is h jr h kq. The we classify the foregoig equatio ito three type listed below. 1 The uderlie I the suffixes k 5 ad k 6 of k 5k 6, oe is from i, j ad k, ad aother is from q, r ad s. Additioally, i k 3 ad k 4 of h k 3k 4, oe is from i, j ad k, ad aother is from q, r ad s. 1 2 The uderbrace I the suffixes k 5 ad k 6 of k 5k 6, oe is from i, j ad k, ad aother is from q, r ad s. Additioally, the suffixes k 3 ad k 4 have two alphabets i either i, j, k or q, r, s. 9 3 The suffixes k 5 ad k 6 of k 5k 6 have two alphabets i either i, j, k or q, r, s. 1 Cosequetly, h h qrs 3 µ µ = 9 h h qrs iq h jk h rs 1 h h qrs iq h jr h ks 1 h h qrs ij h kq h rs + O 2 = 9 h h qrs h jk h rs h iα h lαβ h βq h lm G m α,β,l,m α,β 1 h h qrs h jr h ks h iα h lαβ h βq h lm G m α,β,l,m α,β 1 h h qrs h kq h rs h iα h lαβ h βj h lm G m α,β,l,m α,β h iα G αβ h βq h iα G αβ h βq h iα G αβ h βj + O 2. 15

16 Thus the foregoig equatio becomes 9 h h qrs h lαβ h jk h rs h iα h βq h lm G m αβlm αβlm αβlm + O 2. h h qrs h lαβ h jr h ks h iα h βq h lm G m + 1 h h qrs h lαβ h kq h rs h iα h βj h lm G m + 1 Proof of c. It follows from Lemma 13 d that 3 h ˆθ N h ˆθh qrs µ αβ αβ αβ h h qrs h jk h rs h iα h βq G αβ h h qrs h jr h ks h iα h βq G αβ h h qrs h kq h rs h iα h βj G αβ = 1 3 h α h αβ G β 1 G + O 2 h qrs µ + O 4 α,β = 1 h α h qrs h αβ 3 µ G β h qrs 3 µ G + O 2 αβ Hece, c is proved. Proof of d Expadig the first derivatives h l ˆθ N aroud ˆθ, ad usig B.1 ad B.2 yields h l ˆθ N = h l ˆθ + D 1 h l ˆθ T ˆθ N ˆθ ˆθ N ˆθ T D 2 h l ˆθˆθ N ˆθ + O 3 = h l ˆθ + D 1 h l ˆθ [D T 2 h ˆθ] 1 D 1 h ˆθ T 1 1 [D2 hˆθ] 1 D 1 Gˆθ D 2 h l ˆθ [D2 hˆθ] 1 D 1 Gˆθ + O 3. It follows from defiitio of a iverse matrix that Thus we have D 1 h l ˆθ T [D 2 h ˆθ] 1 = 0,..., 0, 1 l, 0,..., 0. h l ˆθ N = h l ˆθ h l ˆθ D1 Gˆθ T [D 2 hˆθ] 1 D 2 h l ˆθ[D 2 hˆθ] 1 D 1 Gˆθ + O 3 = D1 Gˆθ T [D 2 hˆθ] 1 D 2 h l ˆθ[D 2 hˆθ] 1 D 1 Gˆθ + O 3 = r = lαβqr Hece, the lemma is proved. h 1r G r h dr G r D 2 h l ˆθ r h lαβ h αr h βq G r G q + O q h1q G q. q hdq G q + O 3

17 Appedix C: Proofs of the mai results This sectio proves the mai results. The otatios are the same as i Sectios 2 ad 3. Proof of Theorem 1. Θ e hθ dθ = B δ θ First, we evaluate the first term i the r.h.s. By expadig hθ about ˆθ, hθ = hˆθ + i q e hθ dθ + Θ B δ θ e hθ dθ. h i ˆθz i + 1 h ij ˆθz i z j + 1 h ˆθz i z j z k 2 6 ij h q ˆθz i z j z k z q h ˆθz i z j z k z q z r z s + R 1, hqr ˆθz i z j z k z q z r where R 1 1/7! h t z i z j z k z q z r z s z t +r 1, ad r 1 is bouded over B ɛ ˆθ by a polyomial i z i z j z k z q z r z s z t z u. By usig Lemma 7, we have h i ˆθz i + 1 h ij ˆθz i z j 2 where y = ˆθ [D 2 hˆθ] 1 D 1 hˆθ. i ij = D 1 hˆθ T θ ˆθ θ ˆθ T D 2 hˆθθ ˆθ = 1 2 θ yt D 2 hˆθθ y 1 4 D1 hˆθ T [ 1 2 D2 hˆθ] 1 D 1 hˆθ, Usig the expasio e x = 1 + x + 1/2x 2 + 1/6x 3 + 1/24x 4 + 1/120e τ 1 x 5, where τ 1 is a poit betwee 0 ad x, it follows that { [ 1 exp h ˆθz i z j z k { 1 = 1 h z i z j z k [ h ˆθz i z j z k z q z r z s + R 1 ]} hq z i z j z k z q + 1 hqr z i z j z k z q z r h z i z j z k z q z r z s + R 1 0 h z i z j z k ] [ h z i z j z k ][ h qrst z q z r z s z t ] qrst 2 [ h qz i z j z k z q ] [ h z i z j z k ][ h qrstu z q z r z s z t z u ] + R 2 qrstu 3 1 [ h z i z j z k ] [ h z i z j z k ] 2 [ h qrst z q z r z s z t ] + R 3 qrst [ } h z i z j z k ] 4 + R 4 + R = J θ, ˆθ + R, C.1 17

18 where R 2, R 3, ad R 4 are the rests of terms squared, cubed, ad to the fourth power i the bracket, R 5 is a term with polyomials of degree greater tha or equal to 15, ad R is the sum of all terms ivolvig R 1, R 2, R 3, R 4, ad R 5. I C.1, J θ, ˆθ is J θ, ˆθ = 1 6 h z i z j z k 24 hq z i z j z k z q + 2 h h qrs z i z j z k z q z r z s hqr z i z j z k z q z r h h qrst z i z j z k z q z r z s z t From C.1, we have e hθ dθ B δ ˆθ = e hˆθ exp exp B δ ˆθ h h qrstu z i z j z k z q z r z s z t z u h h qrs h tuv z i z j z k z q z r z s z t z u z v h z i z j z k z q z r z s 0 h h qrs h tuvw z i z j z k z q z r z s z t z u z v z w 2 hq h rstu z i z j z k z q z r z s z t z u 1152 h h qrs h tuv h wxy z i z j z k z q z r z s z t z u z v z w z x z y. { 4 D1 hˆθ T [ 1 } 2 D2 hˆθ] 1 D 1 hˆθ 1 2 θ yt 1 D2 hˆθ 1 1 θ y { 1 exp h ˆθz i z j z k = e hˆθ C ˆθ + e hˆθ C ˆθ B δ ˆθ B δ ˆθ exp 1 2 θ yt Σ 1 θ y h z i z j z k z q z r z s + R 1 } dθ. J θ, ˆθdθ, exp 1 2 θ yt Σ 1 θ y R dθ C.2 C.3 where Σ = [D 2 hˆθ] 1. Secod, we evaluate C.3. The terms composig R may be represeted explicitly usig the mea value form of the remaiders i terms of higher derivatives of h evaluated at poits betwee ˆθ ad θ, for example, oe such term is /! h tu γ 1 z i z j z k z q z r z s z t z u, where γ 1 is a poit betwee θ ad ˆθ. It is oe piece of the error term appearig as R. Because it follows from coditio A3 that h tu γ 1 < M 1

19 o B δ ˆθ, we have! M! B δ ˆθ exp 1 2 θ yt Σ 1 θ y h tu γ 1 z i z j z k z q z r z s z t z u dθ exp 1 2 θ yt Σ 1 θ y z i z j z k z q z r z s z t z u dθ B δ ˆθ = Σ 1/2 O 3. C.4 The other terms are similar. Thus C.3 becomes 2π d/2 e hˆθ Σ 1/2 C ˆθ O 3. Third, we evaluate C.2. Sice there exists a symmetric matrix A 1/2 such that A 1/2 A 1/2 = D 2 hˆθ, puttig 1/2 A 1/2 θ y = u, we have Θ B δ ˆθ = {θ : θ ˆθ T θ ˆθ > δ 2 } = {θ : u 1/2 A 1/2 b T [D 2 hˆθ] 1 u 1/2 A 1/2 b > δ 2 } 1 {θ : u 1/2 A 1/2 b T u 1/2 A 1/2 b > δ 2 } λ 1 {θ : 2u T u + 2b T A 1/2 A 1/2 b > λ 1 δ 2 } = {θ : u T u > c 2 } = {θ : θ y T Σ 1 θ y > c 2 }, where λ 1 is the smallest eigevalue of D 2 hˆθ, b = b i [D 2 hˆθ] 1 D 1 hˆθ ad c 2 = 1/2{λ 1 δ 2 2D 1 hˆθ T [D 2 hˆθ] 1 D 1 hˆθ}. Note that c 2 > 0 for large because λ 1 is strictly positive from the assumptio A4. Hece, oce agai puttig 1/2 A 1/2 θ y = u = u i, we have Θ B δ ˆθ exp 1 2 θ yt Σ 1 θ y J θ, ˆθdθ θ y T Σ 1 θ y>c 2 exp 1 2 θ yt Σ 1 θ y J θ 2, ˆθdθ = Σ u 1/2 exp 1 1/2 T u>c 2 2 ut u P oudu, where P ou deotes a term with multivariate polyomials i u 1,..., u d of fiite degree. Cosequetly, Θ B δ ˆθ exp 1 2 θ yt Σ 1 θ y J θ, ˆθdθ. becomes the term of expoetially decreasig error by the same argumet as i Kass et al. 1990, pp This allows the replacemet of the domai i C.2 by the whole Euclidea space. Therefore, C.2 is replaced with e hˆθ C ˆθ exp 1 R 2 θ yt Σ 1 θ y J θ, ˆθdθ. d Sice Σ = h ij /, it follows from Lemma 12 a that 1/2 E N [Θ i ˆθ i Θ j ˆθ j Θ k ˆθ k ] = b i h jk + b j h ik + b k h ij + b i b j b k. 19

20 The, usig D 1 hˆθ = O 1, we have [ E N ] h ˆθz i z j z k = 1 h ˆθb i h jk h b i b j b k C.5 Moreover, [ ] E N h q z i z j z k z q = hq E[z i z j z k z q ] q = h ij h q 24 = 1 = 1 q q hkq + hik hjq + hiq hkj + h q h ij h kq h q h ij b k b q 24 q h q h ij h kq 1 h q h ij b k b q + O 3 4 q q b m1 b hm3m4 m2 [m 1,m 2 ] 4 q h q b i b j b k b q + b i b j b k b q C.6 Similarly, applyig iii xi of Appedix E to the other expaded terms i J θ, ˆθ, ad the combiig C.5 ad C.6 yields e hθ dθ = 2π d/2 Σ 1/2 e hˆθ C ˆθ 1 + λ 1 + λ O 3, C.7 B δ ˆθ where C ˆθ = exp{/2d 1 hˆθ T [D 2 hˆθ] 1 D 1 hˆθ}, λ 1 = 1 h ˆθb i h jk 1 2 q h q ˆθh ij h kq + 1 h ˆθh qrs ˆθµ 3, ad µ are the sixth cetral momets of a multivariate Normal distributio havig covariace matrix [D 2 hˆθ] 1. O the other had, it follows from assumptio A.5 of Sectio 2 that e hθ dθ = 2π d/2 Σ 1/2 e hˆθ C ˆθ O 3. Θ B δ ˆθ Therefore, combiig C.7 ad C. yields the result. C. Lemma 16. Suppose that ˆθ is a asymptotic mode of order 2 for h. The it follows that D 1 h ˆθ N = O 2. Proof. Expadig h ˆθ N / θ i aroud ˆθ yields, for i = 1,..., d, h ˆθ N = h ˆθ + D 1 [ h ˆθ T ˆθ N θ i θ i θ ˆθ + ˆθ N ˆθ T i D 2 ] h ξ i ˆθ N θ ˆθ, i C.9 where ξ i is a iterior poit o the lie from ˆθ to ˆθ N. Writig C.9 i a matrix form gives D 1 h ˆθ N = D 1 h ˆθ + D 2 h ˆθ T ˆθ N ˆθ + R ξ, C.10 20

21 where R ξ = ˆθ N ˆθ T [D 2 h ξ 1 / θ 1 ]ˆθ N ˆθ,..., ˆθ N ˆθ[D 2 h ξ d / θ d ]ˆθ N ˆθ T. Substitutig ˆθ N ˆθ = [D 2 h ˆθ] 1 D 1 h ˆθ ito C.10, we have Proof of Theorem 5. e h θ dθ e hθ dθ D 1 h ˆθ N = D 1 h ˆθ D 2 h ˆθ[D 2 h ˆθ] 1 D 1 h ˆθ + O 2 = O 2. D 2 1/2 hˆθ C = ˆθ N exp[ h ˆθ N ] D 2 h ˆθ N C ˆθ exp[ hˆθ] D 2 1/2 hˆθ C = ˆθ N exp[ h ˆθ N ] D 2 h ˆθ N C ˆθ exp[ hˆθ] 1 + a 1 + a O a 1 + a O a 1 a 1 + a 2 a 2 a 1 a 1 a 1 + O 3 2 First, we shall evaluate a 1 a 1. It follows from Lemma 13 d that 3 h ˆθ N h qrsˆθ N µ 3 h ˆθh qrs ˆθµ C.11 = 3 { h ˆθ N h ˆθh qrsˆθ N µ } + h ˆθh qrsˆθ N h qrs ˆθµ + h ˆθh qrs ˆθµ µ = hα h αβ G β 1 G h qrs [ 3 µ ] + 1 h h qrs 3 µ µ + O 2 The left term of the foregoig equatio becomes 1 hα h qrs h αβ [ 3 µ ]G β 1 hqrs [ 3 µ ]G By usig Lemma 14 b, the right term equatio is give by { 1 9 h h qrs h lαβ h jk h rs h iα h βq h lm G m + 9 h h qrs h jk h rs h iα h βq G αβ h h qrs h lαβ h jr h ks h iα h βq h lm G m + 1 h h qrs h jr h ks h iα h βq G αβ 1 1 h h qrs h lαβ h kq h rs h iα h βj h lm G m + 1 } h h qrs h kq h rs h iα h βj G αβ = 1 h h qrs h lαβ h jk h rs h iα h βq h lm G m + 1 h h qrs h jk h rs h iα h βq G αβ 1 h h qrs h lαβ h jr h ks h iα h βq h lm G m + 1 h h qrs h jr h ks h iα h βq G αβ 4 4 h h qrs h kq h rs h iα h βj G αβ. 1 4 h h qrs h lαβ h kq h rs h iα h βj h lm G m C.12 C.13 C.14 C.15 21

22 I cotrast, it follows from Lemma 13 g, ad Lemma 13 e that 1 h qˆθ N h ij ˆθ N h kq ˆθ N h q ˆθh ij ˆθh kq ˆθ = 1 { h qˆθ N h ij ˆθ N [h kq ˆθ N h kq ˆθ] q } + h qˆθ N [h ij ˆθ N h ij ˆθ]h kq ˆθ + h ij ˆθh kq ˆθ[h qˆθ N h q ˆθ] = 1 2 hq h ij h kα h lαβ h βq h lm G m + 2 hq h ij h kα h βq G αβ + 1 hql h ij h kq h lm G m 1 h ij h kq G q + O 2 = 1 hq h ij h kα h lαβ h βq h lm G m 1 hq h ij h kα h βq G αβ hql h ij h kq h lm G m + 1 h ij h kq G q + O 2. C.16 Combiig C.12, C.13, C.14, C.15, ad C.16, we have a 1 a 1 = κ /, where κ is of order 1. Next, we evaluate a 2 a 2. Followig the proof of Tierey ad Kadae 196, we have Therefore, a 2 a 2 = 1 h 2 ˆθ N 2 b i h jk + 1 h ˆθ 2 b i h jk + O 1. 2 D E[g + 2 1/2 h ˆθ C Θ] = ˆθ N exp[ h ˆθ N ] 1 + c D 2 h ˆθ N C ˆθ exp[ hˆθ] + 2 O 3, where C ˆθ N = exp{/2d 1 h ˆθ N T [D 2 h ˆθ N ] 1 D 1 h ˆθ N } ad c = 2 h b i h jk + 2 h b i h jk + 1 hq h lαβ h ij h kα h βq h lm G m hq h ij h kα h βq G αβ 1 hql h ij h kq h lm G m + 1 h ij h kq G q hα h qrs h αβ [ 3 µ ]G β 1 hqrs [ 3 µ ]G h h qrs h lαβ h jk h rs h iα h βq h lm G m + 1 h h qrs h jk h rs h iα h βq G αβ αβlm αβlm αβlm h h qrs h lαβ h jr h ks h iα h βq h lm G m h h qrs h lαβ h kq h rs h iα h βj h lm G m αβ αβ αβ h h qrs h jr h ks h iα h βq G αβ h h qrs h kq h rs h iα h βj G αβ C.17 22

23 As a alterative expressio of 2 /2 h b i h jk i c, from Lemma 14 d, we have b i ˆθ N = l h il h l ˆθ N + O 3 Thus 2 2 = h b i h jk = 1 4 αβlmr lmrαβ h lαβ h il h αr h βm G r G m + O 3. h h lαβ h il h jk h αr h βm G r G m + O 1 C.1 Appedix D: Momets of multivariate ormal distributios A d 1 radom vector u is accordig to N0, Σ, where Σ µ ij, ad 0 is a d 1 ull vector. Let µ q ad µ deote the fourth ad sixth cetral momets of multivariate ormal distributio N0, Σ. The we have the formula cocerig the momets: µ q = µ ij µ ks + µ ik µ js + µ is µ jk, µ = µ ij µ kq µ rs + µ ij µ kr µ qs + µ ij µ ks µ qr + µ ik µ jq µ rs + µ ik µ jr µ qs + µ ik µ js µ qr + µ iq µ jk µ rs + µ iq µ jr µ ks + µ iq µ js µ kr + µ ir µ jk µ qs + µ ir µ jq µ ks + µ ir µ js µ kq + µ is µ jk µ qr + µ is µ jq µ kr + µ is µ jr µ kq Appedix E:Auxiliary calculatio o the Laplace approximatios iii From Lemma 12 d, [ 2 ] E h h qrs z i z j z k z q z r z s = 2 h h qrs E[z i z j z k z q z r z s ] = 2 h h qrs µ + b m1 b m2 µ m3 m 4 m 5 m 6 + b l1 b l2 b l3 b l4 µ l5 l 6 + b i b j b k b q b r b s [m 1,m 2 ] 6 [l 1,l 2,l 3,l 4 ] 6 = 2 h h qrs µ + 2 h h qrs b m1 b m2 µ m3 m 4 m 5 m 6 + O 3 E.1 [m 1,m 2 ] 6 23

24 Because [m 1,m 2 ] 6 b m1 b m2 µ m3 m 4 m 5 m 6 = b i b j µ kqrs + b i b k µ jqrs + b i b q µ jkrs + b i b r µ jkqs + b i b s µ jkqr + b j b k µ iqrs + b j b q µ ikrs + b j b r µ ikqs + b j b s µ ikqr + b k b q µ ijrs + b k b r µ ijqs + b k b s µ ijqr + b q b r µ s + b q b s µ r + b r b s µ q, 2 h h qrs b m1 b m2 µ m3 m 4 m 5 m 6 is devided ito two kids of terms. Hece, the secod [m 1,m 2 ] term i E.1 is 2 h h qrs b m1 b m2 µ m3 m 4 m 5 m 6 [m 1,m 2 ] = 2 6 h h qrs b i b j µ kqrs h h qrs b i b q µ jkrs. Coseqetly, we have [ 2 ] E h h qrs z i z j z k z q z r z s = 2 h h qrs µ + 2 h h qrs b i b j µ kqrs + 2 h h qrs b i b q µ jkrs. 12 iv From Lemma 12 c, [ E ] hqr z i z j z k z q z r = hqr E[z i z j z k z q z r ] = h qr b m1 µ m2 m m 4 m 5 + O 3 qr [m 1 ] 5 = h qr b i µ jkqr + O 3 qr = h qr b i µ jkqr + O v From Lemma 12 d [ E 0 qr h z i z j z k z q z r z s ] = 0 vi From Lemma 12 e, [ 2 ] E h h qrst z i z j z k z q z r z s z t 144 = 2 h h qrst E[z i z j z k z q z r z s z t ] 144 = = 0 h E[z i z j z k z q z r z s ] h µ + O 3. h h qrst b m1 µ m2 m 3 m 4 m 5 m 6 m 7 + O 3 [m 1 ] 7 = h h qrst b i µ jkqrst h h qrst b q µ rst + O 3 = 2 h h qrst b i µ jkqrst + 2 h h qrst b q µ rst + O

25 vii From Lemma 12 f, [ 2 ] E hq h rstu z i z j z k z q z r z s z t z u = 2 hq h rstu E[z i z j z k z q z r z s z t z u ] = 2 hq h rstu µ tu + O viii From Lemma 12 f, [ 2 ] E h h qrstu z i z j z k z q z r z s z t z u = 2 h h qrstu µ tu + O ix From Lemma 12 g, E [ 3 ] h h qrs h tuv z i z j z k z q z r z s z t z u z v 1296 = 3 h h qrs h tuv b m1 µ m2 m m 4 m 5 m 6 m 7 m m 9 + O 3 [m 1 ] 9 = h h qrs h tuv b i µ jkqrstuv + O 3 = 3 h h qrs h tuv b i µ jkqrstuv + O x From Lemma 12 h, E [ 3 ] h h qrs h tuvw z i z j z k z q z r z s z t z u z v z w = 3 h h qrs h tuvw µ tuvw + O xi From Lemma 12 i, [ 4 ] E h h qrs h tuv h wxy z i z j z k z q z r z s z t z u z v z w z x z y = 4 h h qrs h tuv h wxy µ tuvwxy + O The iequality i Theorem a 1 + a 2 + a a 1 + a 2 + a = 1 + a 1 a 1 + a 2 a 2 a 1 a 1 a O 3, where O 3 is give by [ a 3 a 3 3 a1 3 + a 2 + a 3 a 2 a 2 a 1 a 1 a a 1 + a 2 + a a2 2 + a 3 3 ] a 1 a 1 25

26 Proof. 1 + a 1 + a 2 + a a 1 + a 2 + a [ = 1 + a 1 + a 2 + a a 1 + a 2 + a Therefore this iequality is proved. 1 + a 1 a 1 + a 2 a 2 a 1 a 1 a a 1 a 1 + a 2 a 2 a 1 a 1 a a a 2 + a ] Refereces Harville, D. A. 1997, Matrix Algebra From A Statisticias Perspective, Spriger-Verlag, New York. Kass, R. E., Tierey, L., ad Kadae, J. B. 1990, The Validity of Posterior Expasios Based o Laplace s Method, i Essays i Hoor of George Barard, eds. S. Geisser, J. S. Hodges, S. J. Press, ad A. Zeller, Amsterdam: NorthHollad. Laplace, P. S. 196, Memoir o the Probability of Causes of Evets traslatio by S. Stiger of the 1774 origial, Statistical Sciece, 1, Lidley, D. V. 190, Approximate Bayesia Methods, i Bayesia Statistics, eds. J. M. Berardo, M. H. Degroot, D. V. Lidley, ad A. F. M. Smith, Valecia, Spai: Uiversity Press, pp Miyata, Y., 2004, Fully Expoetial Laplace Approximatios Usig Asymptotic Modes, Joural of the America Statistical Associatio, 99, Mosteller, F., ad Wallace, D. L. 1964, Iferece ad Disputed Authorship: The Federalist Papers, Readig, MA: Addiso-Wesley. Tierey, L., ad Kadae, J. B. 196, Accurate Approximatios for Posterior Momets ad Margial Desities, Joural of the America Statistical Associatio, 1,