Laplace Expansion. Peter McCullagh. WHOA-PSI, St Louis August, Department of Statistics University of Chicago

Transcript

1 Laplace Expansion Peter McCullagh Department of Statistics University of Chicago WHOA-PSI, St Louis August, 2017

2 Outline Laplace approximation in 1D Laplace expansion in 1D Laplace expansion in R p Formal power series FPS Faà di Bruno Composition of FPS Bi-partition expansions

3 Laplace approximation in 1D g(x) is a differentiable function with max at 0 g(0) = 0 for notational simplicity Problem: approximate the integral Taylor series expansion I n (g) = e ng(x) dx g(x) = g x 2 /2 + g x 3 /3! + g (iv) x 4 /4! + + O(?) exp(ng(x)) = e ng x 2 /2 exp(ng x 3 /3! + ng (iv) x 4 /4! + + O(?)) = e ng x 2 /2 (1 + h 3 x 3 /3! + h 4 x 4 /4! + ) e ng(x) dx 2π/( ng ) ( 1 + O(n 1 ) ) Role of n

4 Laplace expansion in 1D Taylor series expansion g(x) = g x 2 /2 + g x 3 /3! + g (iv) x 4 /4! + + O(?) exp(ng(x)) = e ng x 2 /2 exp(ng x 3 /3! + ng (iv) x 4 /4! + + O(?)) = e ng x 2 /2 (1 + h 3 x 3 /3! + h 4 x 4 /4! + h3 2 x 6 /72 + ) e ng(x) 2π dx ( ng ) E ( σ 1 + h3 X 3 /3! + h 4 X 4 /4! + h3 2 X 6 /72 + σ 2π ( 1 + 3σ 4 h 4 /4! + 15σ 6 h 2 3 /72 + O(n 2 ) ) σ 2 = 1/( ng ): σ = O(n 1/2 ) Coefficients {h r } versus {ng r }: h(x) = exp(ng(x))

5 Laplace expansion in R p g : R p R: differentiable, max at 0 Taylor series using summation convention g(x) = g ij x i x j /2! + g ijk x i x j x k /3! + g ijkl x i x j x k x l /4! + Σ = [ ng ij ] 1 positive definite Order-zero Laplace approximation: I n (g) = exp( ng(x)) dx = det(2πσ)(1 + O(n 1 )) Higher-order Laplace expansion:??

6 Formal power series (FPS) Setting: U.S.: an Unspecified Space (containing 0) x a [formal] variable; a point in the U.S. f a formal power series (of exponential type) h another FPS acting on the U.S. f = (f 1, f 2,...) are the components of f h = (h 1, h 2,...) are the Taylor coefficients of h f (x) = f 1 x + f 2 x 2 /2! + f 3 x 3 /3! + f 4 x 4 /4! + No coefficient f 0 : f (0) = 0 Where does f live? co-domain of FPS? What can we do with FPS? Add f + g, Multiply fg = gf, Compose f g g f?

7 Formal power series (FPS) Setting: U.S.: an Unspecified Space (containing 0) x a [formal] variable; a point in the U.S. f a formal power series (of exponential type) h another FPS acting on the U.S. f = (f 1, f 2,...) are the components of f h = (h 1, h 2,...) are the Taylor coefficients of h f (x) = f 1 x + f 2 x 2 /2! + f 3 x 3 /3! + f 4 x 4 /4! + No coefficient f 0 : f (0) = 0 Where does f live? co-domain of FPS? What can we do with FPS? Add f + g, Multiply fg = gf, Compose f g g f?

8 Formal power series (FPS) Setting: U.S.: an Unspecified Space (containing 0) x a [formal] variable; a point in the U.S. f a formal power series (of exponential type) h another FPS acting on the U.S. f = (f 1, f 2,...) are the components of f h = (h 1, h 2,...) are the Taylor coefficients of h f (x) = f 1 x + f 2 x 2 /2! + f 3 x 3 /3! + f 4 x 4 /4! + No coefficient f 0 : f (0) = 0 Where does f live? co-domain of FPS? What can we do with FPS? Add f + g, Multiply fg = gf, Compose f g g f?

9 Faà di Bruno composition formula Composition h = f g U.S. A R f (x) f U.S. B f f R x g U.S. C g g R p f (x) = f 1 x + f 2 x 2 /2! + f 3 x 3 /3! + f 4 x 4 /4! + g(ξ) = g i ξ i + g ij ξ i ξ j /2! + g ijk ξ i ξ j ξ k /3! + g ijkl ξ i ξ j ξ k ξ l /4! + h = (h i, h ij, h ijk,...) h ijkl =?? (as FPS) ξ

10 Faà di Bruno composition formula Composition h = f g R f R g R p f = (f 1, f 2, f 3,...) components g = (g i, g ij, g ijk, g ijkl,...) Taylor coefficients of g h = (h i, h ij, h ijk, h ijkl,...) Taylor coefficients of h h i = f 1 g i h ij = f 1 g ij + f 2 g i g j h ijk = f 1 g ijk + f 2 (g ij g k + g ik g j + g jk g i ) + f 3 g i g j g k h ijkl = f 1 g ijkl + f 2 g ijk g l [4] + f 2 g ij g kl [3] + f 3 g ij g k g l [6] + f 4 g i g j g k g l h [n] = f #σ σ Pn b σ g b

11 Moments and cumulants X P on some space S = R p : ξ S = dual(s) R exp R log R M S M(ξ) = E[e ξx ( ] = E P 1 + ξi X i + [ξ i X i ] 2 /2! + [ξ i X i ] 3 /3! + ) = 1 + ξ i E(X i ) + ξ i ξ j E(X i X j )]/2! + ξ i ξ j ξ k E(X i X j X k )/3 = 1 + ξ i µ i + ξ i ξ j µ ij /2! + ξ i ξ j ξ k µ ijk /3! + K (ξ) = log M(ξ) = ξ i κ i + ξ i ξ j κ i,j /2! + ξ i ξ j ξ k κ i,j,k /3! + What does Faà di Bruno imply?

12 Moments and cumulants (contd) M(ξ) = exp(k (ξ)): Taylor coefficients are exp n = 1 µ([n]) = exp #σ κ(b) σ Pn b σ K (ξ) = log M(ξ): Taylor coefficients log n = ( 1) n 1 (n 1)! κ([n]) = log #σ µ(b) σ Pn b σ κ i,j = log 1 µ ij + log 2 µ i µ j κ i,j,k = log 1 µ ijk + log 2 µ ij µ k [3] + log 3 µ i µ j µ k κ i,j,k,l = log 1 µ ijkl + log 2 (µ ijk µ l [4] + µ ij µ kl [3]) + log 3 µ ij µ k µ l [6] + log 4 µ i µ j µ k µ l

13 Bi-partition expansions Related pair of FPS: ξ and ξ on S = R p 1 + ξ(x) = exp(ξ(x)); ξ[n] = X P arbitrary, but finite moments... σ Pn b σ M(ξ) = E exp(ξ(x)) = 1 + ξ i µ i + ξ ij µ ij /2! + ξ ijk µ ijk /3! + = 1 + ξ i κ i + ξ ij κ i,j /2! + ξ ijk κ i,j,k /3! + = ξ b, κ d n! n 1 σ,τ Pn b σ d τ log M(ξ) = 1 ξ b, κ d n! n 1 b σ d τ σ,τ Pn σ τ=1 n Generalized cumulant; vacuum cluster expansion,... ξ b

14 Application to Laplace expansion Need the integral over R p of exp(g ij x i x j /2!) exp ( g ijk x i x j x k /3! + g ijkl x i x j x k x l /4! + ) ( = exp(g ij x i x j /2) 1 + g [n], x n / ) n! n 3 ( I n = det 1/2 (2πΣ) 1 + g [n], µ [n] / ) n! n 3 = det 1/2 (2πΣ) (1 + 1 g b, κ d ) n! b σ d τ log I n = log det(2πσ) 2 + n=2k n=2k 1 n! σ,τ Pn τ 2 k σ τ=1 n σ,τ Pn τ 2 k b σ g b, c τ Σ c

15 The first major correction term for log(i n ) Typically: g... = O(n); [Σ i,j ] = [g ij ] 1 = O(n 1 ) n = 4 : σ = {[4]}, τ = gives g ijkl Σ i,j Σ k,l [3]/4! n = 6 : σ = , τ = gives g b, Σ c = g ijk g lmn Σ i,j Σ k,l Σ m,n [10][9]/6! b σ c τ n = 6 : σ = ; τ = gives g b, Σ c = g ijk g lmn Σ i,k Σ j,l Σ k,n [10][6]/6! b σ c τ g ijkl Σ i,j Σ k,l 8 + g ijkg lmn Σ i,j Σ k,l Σ m,n 8 + g ijkg lmn Σ i,l Σ j,m Σ k,n 12

16 Some further terms n = 6 : σ = = {[6]}, τ = gives g b, Σ c = g ijklmn Σ i,j Σ k,l Σ m,n [1][15]/6! b σ c τ n = 8 : σ = , τ = gives g b, Σ c = g i1 i 2 i 3 g i4...i 8 Σ i 1,i 2 Σ i 3,i 4 Σ i 5,i 6 Σ i 7,i 8 [56][45]/8! b σ c τ n = 8 : σ = , τ = gives g b, Σ c = g i1 i 2 i 3 i 4 g i5...i 8 Σ i 1,i 5 Σ i 2,i 6 Σ i 3,i 4 Σ i 7,i 8 [35][72]/8! b σ c τ and so on...