Supporting information for: Functional Mixed Effects Model for Small Area Estimation

Transcript

1 Supportng nformaton for: Functonal Mxed Effects Model for Small Area Estmaton Tapabrata Mat 1, Samran Snha 2 and Png-Shou Zhong 1 1 Department of Statstcs & Probablty, Mchgan State Unversty, East Lansng, MI mat@stt.msu.edu, pszhong@stt.msu.edu and 2 Department of Statstcs, Texas A&M Unversty, College Staton, TX snha@stat.tamu.edu Ths document contans some lemmas and ther proof that are ey n provng Theorems 1 and 2 stated n the man paper. Frst, we re-state some notatons that we have already ntroduced n the man paper. Model (3.3) n the paper s Y = Z F b F + W ν + Υ, where W = (Z R, M 0,, M p ) and ν = (b T R, U T 0, U T 1,, U T p ) T. Denote G = var(ν) = dag(cov(b R ), I n Σ u0,, I n Σ up ) and cov(y ) = Σ. Also, Σ = Σ (δ) = Z F cov(b R )Z T F + cov(u 0 ) + p =1 Dag(X ) cov(u )Dag(X ) + Ω for = 1,..., n. The estmator of b F s bf = (Z T F Σ 1 Z F ) 1 Z T F Σ 1 Y and the predctor of ν s ν = GW T (Y Z F bf ). The covarance matrx Σ(δ) s nvolved wth parameters δ = (σ 2 b 0,, σ 2 b p, ψ 0,, ψ p, ρ 0,, ρ p ) T. Let s = (p + 1)(q + 2) be the number of parameters n δ. These parameters are estmated through restrcted maxmum lelhood method by maxmzng l(δ) = 1 2 log ZT F Z F 1 2 log Σ 1 2 (Y Z F b F ) T (Y Z F bf ). =1 Then there exst some T such that T T Z F = 0 and ran(t ) = mn L 1 (p + 1), and defne P = T (T T ΣT ) 1 T T = Z F (Z T F Z F ) 1 Z T F such that the lelhood can be wrtten as (S1) l(δ) = 1 2 log T T ΣT 1 2 Y T P Y. (S2) and defne the REML estmator δ as the soluton to the score equaton l(δ)/ δ = 0. Now we shall state the lemmas. Lemma 1 s needed for Lemma 2 whch s needed for Theorem 2. Lemmas 3 and 4 are needed for provng Theorems 1 and 2. Lemma 1 Let Σ = Σ (δ ) and Σ = Σ (δ). If max, X <, then tr{(σ Σ ) 2 } C δ δ δ 2, where C δ s a constant and δ δ 2 = (δ δ) T (δ δ). 1

2 Proof: We wrte Σ Σ = J 1 + J 2 + J 3 + J 4 + J 5, where Therefore, J 1 = Z F Dag{(σ b 0 2 σ 2 b 0 )Dag(λ 1 L 1 ),, (σ b p 2 σ 2 b p )Dag(λ 1 L 1 )}Z T F, J 2 = (ψ 0 ψ 0 )A m (ρ 0 ), J 3 = ψ0a m (ρ 0) ψ0a m (ρ 0 ), p J 4 = Dag(X )[ψ{a m (ρ ) A m (ρ )}]Dag(X ), J 5 = =1 p Dag(X ){(ψ ψ )A m (ρ )}Dag(X ). =1 tr{(σ Σ ) T (Σ Σ )} C[tr(J T 1 J 1 ) + tr(j T 2 J 2 ) + tr(j T 3 J 3 ) + tr(j T 4 J 4 ) + tr(j T 5 J 5 )]. We see that tr(j T 1 J 1 ) C p m (σb 2 σb 2 ) 2 =1 {Z ()T F j,l=1 (t j )Dag(λ 2 L 1 )Z () F (t l )} 2. Also, notce that tr(j T 2 J 2 ) = tr{(ψ 0 ψ 0 )A m (ρ 0 )} 2 C(ψ 0 ψ 0 ) 2 tr{a m (ρ 0 )} 2 and tr[{ψ 0A m (ρ 0) ψ 0A m (ρ 0 )} 2 ] = tr { [(ψ 0 ψ 0 ){A m (ρ 0) A m (ρ 0 )} + ψ 0 {A m (ρ 0) A m (ρ 0 )}] 2} Hence, tr(j T 3 J 3 ) C δ δ 2. Next, tr(j T 4 J 4 ) C C 2{(ψ 0 ψ 0 ) 2 + ψ 2 0}tr{A m (ρ 0) A m (ρ 0 )} 2 C{(ψ 0 ψ 0 ) 2 + ψ 2 0} ρ 0 ρ 0 2. p tr { Dag(X )[ψ{a m (ρ ) A m (ρ )}]Dag(X ) } 2 =1 p =1 [ tr { Dag(X )[(ψ ψ ){A m (ρ ) A m (ρ )}]Dag(X ) } 2 +tr { Dag(X )[ψ {A m (ρ ) A m (ρ )}]Dag(X ) } ] 2 p C [(ψ ψ ) 2 tr{a m (ρ ) A m (ρ )} 2 tr{dag(x )} 4 =1 +ψ 2 tr{a m (ρ ) A m (ρ )} 2 tr{dag(x )} 4 ]. If A m (ρ ) has bounded second dervatves wth respect to ρ, then tr{a m (ρ ) A m(ρ )} 2 C ρ ρ. And tr(j T 5 J 5 ) C p =1 (ψ ψ ) 2 tr { Dag(X )A m (ρ )Dag(X ) } 2. In summary, tr{(σ Σ ) 2 } C δ δ δ 2. 2

3 Lemma 2 Let C δ be the constant defned n Lemma 1 and assume that the smallest egenvalue of Σ s bounded below by c 0 > 0. Suppose that δ δ 2 such that 2C δ /c 2 0 < 1 when n s large enough. Then tr{(σ 1 ) 2 } [ ] (1 2c 4 0 Cδ 2 2 ) 1 2tr{( ) 2 } + 2C δ /c 4 0. Proof: By the matrx nverse formula, Σ 1 Σ ), we then have tr{(σ 1 ) 2 } 2tr{( +2tr[{ = Because (Σ Σ )Σ 2 (Σ Σ ) s non-negatve defnte, tr[{ (Σ Σ )Σ 1 (Σ Σ ) (Σ Σ ) + ) 2 } + 2tr[{ (Σ Σ ) } 2 ] (Σ Σ )Σ 1 (Σ (Σ Σ )Σ 1 (Σ Σ ) } 2 ]. (S3) } 2 ] tr 2 { (Σ Σ )Σ 1 (Σ Σ ) } tr{(σ 1 ) 2 }tr[{(σ Σ )Σ 2 (Σ Σ )} 2 ] tr 2 {(Σ Σ )Σ 2 (Σ Σ )} tr{(σ 1 ) 2 }λ 4 mn (Σ )tr 2 {(Σ Σ ) 2 } tr{(σ 1 ) 2 }c 4 0 Cδ 2 2, where the last nequalty follows from Lemma 1 and the assumpton n ths Lemma. In addton, tr{( (Σ Σ ) ) 2 } λ 4 mn (Σ )tr{(σ Σ ) 2 }. Hence, from (S3), tr{(σ 1 ) 2 } (1 2c 4 0 Cδ 2 2 ) 1 (2tr{( ) 2 } + 2C δ /c 4 0). Ths completes the proof of Lemma 2. Lemma 3 Let d 2 = max j, { ( tr(p V P V ), tr P V P V ) (, tr P 2 V P 2 V )} δ δ and d = mn d. Then there exsts δ such that for any 0 < q 0 < 1 and large n, δ δ = A 1 a + o p (d 2q 0 ), where a = l(δ)/ δ and A = E{ 2 l(δ)/ δ 2 }, on the set B wth P (B) convergng to 1. Proof: We wll apply Theorem 2.1 of Das et al. (2004). Let us frst verfy the followng condtons. 3

4 The gth moment of the followng quanttes are bounded for some d and d = mn d, 1 l(δ) 1 2 l(δ), E( δ δ0 d d j δ j δ0 2 l(δ) ) δ j δ0, d M j, d d j d d where M j = sup δ Sδ (δ 0 ) 3 l(δ)/( δ ) wth S γ (δ 0 ) = {δ : δ δ 0 γd /d 1 s}. Usng the lelhood gven n (S2), we obtan the frst dervatve of l(δ) wth respect to δ l(δ) = 1 2 {ɛt P V P ɛ tr(p V )}, (S4) where ɛ = Y Z F b F and V = dag(v 1,, V n ). Let ɛ = Σ 1/2 u and u N(0, I mn ). Then for any g 2, E l(δ) g = 2 g E u T Σ 1/2 P V P Σ 1/2 u E(u T Σ 1/2 P V P Σ 1/2 u) c Σ 1/2 P V P Σ 1/2 g 2 = ctr(v P V P ) g/2. Thus, f we tae d = tr(v P V P ) 1/2, the gth moment of (1/d ) l(δ 0 )/ δ s bounded. Because P / = T (T T ΣT ) 1 T T ( Σ/ )T (T T ΣT ) 1 T T = P ( Σ/ )P = P V j P, we have 2 l(δ) = 1 { ɛ T Q j ɛ + tr(p V j P V ) tr(p V } ), (S5) 2 where Q j = P {V j P V + V P V j ( V / )}P := P K j P. Then we have where E 2 l(δ) E( 2 l(δ) ) g = 2 g E u T Σ 1/2 Q j Σ 1/2 u E(u T Σ 1/2 Q j Σ 1/2 u) c Σ 1/2 Q j Σ 1/2 g 2 = ctr(k j P K j P ) g/2, tr(k j P K j P ) = tr((v j P V + V P V j V )P (V j P V + V P V j V )P ) = 2tr(V j P V P V j P V P ) + 2tr(V P V P V j P V j P ) 2tr(V j P V P V P ) 2tr(V P V j P V P ) + tr( V P V P ) and applyng Lemma 5.2 of Das et al. (2004), we have tr(v j P V P V P ) tr(v P V P V j P V j P ) 1/2 tr( V P V P ) 1/2 ; tr(v P V j P V P ) tr(v P V P V j P V j P ) 1/2 tr( V P V P ) 1/2 ; tr(v j P V P V j P V P ) tr(v P V P V j P V j P ). g g 4

5 Therefore, tr(k j P K j P ) { 2tr(V P V P V j P V j P ) 1/2 + tr( V P V P ) 1/2} 2. Notce that tr(a 2 ) tr 2 (A) for any non-negatve matrx A. Snce P 1/2 V P V P 1/2 s a nonnegatve defnte matrx, we have tr(v P V P V j P V j P ) tr(v P V P V P V P ) 1/2 tr(v j P V j P V j P V j P ) 1/2 tr(p 1/2 V P V P 1/2 )tr(p 1/2 V j P V j P 1/2 ) = tr(v P V P )tr(v j P V j P ). Hence f we tae d = max j [tr(v P V P ) 1/2, tr{ V / )P ( V / )P } 1/2 ] the gth moment of 1 2 l(δ) E( d d j δ j δ0 2 l(δ) ) δ j δ0 s bounded for any g 2. Next, we compute the thrd dervatves, 3 l(δ) δ { = 2 1 ɛ T P V P (V j P V + V P V j V )P + P ( V j δ P V V j P V P V V j P V δ + V δ P V j V P V P V j + V P V j P (V j P V + V P V j + V )P V P } ɛ E(ɛ T R j ɛ) δ = ɛ T (P V P V j P V P + P V P V P V j P + P V P V P V j P )ɛ ɛ T P 2 V δ P ɛ 2 V δ )P ɛ T (P V P V P + P V P V j P + P V j P V P )ɛ + E{ɛ T P (R j + Rj 2 1 )P ɛ}, δ δ δ where R j = V P V j P V + V P V P V j + V P V P V j and R j = V P ( V / ) + V P ( V j / δ ) + V j P ( V / δ ). Consder the frst term n the thrd dervatves. Denote Σ for Σ evaluated at δ and smlarly for Ṽj. Then t can be shown that (T T ΣT ) 1 = (T T ΣT ) 1 + (T T ΣT ) 1 T T (Σ Σ)T (T T ΣT ) 1 and T T Ṽ j T = T T V j T + T T (Ṽj V j )T. For convenence, denote H = (T T ΣT ) 1 and G = T T V T. Further 1 = HT T (Σ Σ)T H, 2j = T T (Ṽj V j )T. It can be seen that 2 V H = H + (δ δ )HG H + ψ0 HT T ( A m (ρ0) I n )T H + 2(p+1) =1 5 p =1 ψ HT T D{ A m (ρ )}T H,

6 where D( A m ) = dag{dag(x 1 ) A m dag(x 1 ),, dag(x n ) A m dag(x n )}, A m (ρ ) = A m (ρ ) A m ( ρ ).For 1 j (p + 1), Ṽ j = V j ; f j = p + 2, V j Ṽj = A m (ρ 0 ) I n ; f p + 3 j 2(p + 2), V j Ṽj = D{ A m (ρ )}; f 2(p + 1) + 1 j 2(p + 1) + q, V j Ṽj = ( A m (ρ 0 )/ ρ 0,j T ) I n wth j = j 2(p + 1) and f 2(p + 1) + ( 1)q + 1 j 2(p + 1) + q, V j Ṽj = D{ ( A m (ρ )/ ρ,j )} wth 2 (p + 1) and j = j 2(p + 1) + ( 1)q. Snce H, A m (ρ ) and A T m(ρ ) are postve defnte, f γ n S γ s small enough such that, (1/2)H H 2H, (1/2)A m (ρ ) A m ( ρ ) 2A m (ρ ) and (1/2)A T m(ρ ) A T m( ρ ) 2A T m(ρ ). Then f p + 1 H 1/2 G HT T ɛ = H 1/2 G HT T ɛ 2 H 1/2 G HT T ɛ 2 H 1/2 G HT T ɛ + 2(p+1) δ j δ j H 1/2 G HG j HT T ɛ + ψ 0 H 1/2 G HT T ( A m (ρ 0 ) I n )T HT T ɛ p + ψ j H 1/2 G HT T D( A m (ρ j ))T HT T ɛ. It can be shown that there exsts some constant C(γ) such that =1 H 1/2 G HT T ( A m (ρ 0 ) I n )T HT T ɛ C(γ) H 1/2 G H G 2p+3 HT T ɛ C(γ) H 1/2 G H 1/2 H 1/2 G2p+3 HT T ɛ, and H 1/2 G HT T D( A m (ρ j ))T HT T ɛ C(γ) H 1/2 G H 1/2 H 1/2 G HT T ɛ for = j+2(p+1)+1. Therefore, for p + 1, H 1/2 G HT T ɛ 2 H 1/2 G HT T ɛ + 2(p+1) 2 δ j δ j H 1/2 G H 1/2 H 1/2 G j HT T ɛ + 2 3(p+1) =2(p+1)+1 C(γ) ψ 2p 3 H 1/2 G H 1/2 H 1/2 G HT T ɛ 2(p+1) 2 H 1/2 G HT T ɛ + 2γd H 1/2 G H 1/2 d 1 j H 1/2 Gj HT T ɛ +2C(γ) H 1/2 G H 1/2 3(p+1) =2(p+1)+1 ψ 2p 3 H 1/2 G HT T ɛ, 6

7 where H 1/2 G H 1/2 = tr(p V P V ) 1/2. For (q + 2)(p + 1) > p + 1, H 1/2 G HT T ɛ 2{1 + C (γ)} H 1/2 G HT T ɛ In summary, defne then It follows that j = 2(p+1) +2γ{1 + C (γ)}d H 1/2 G H 1/2 d 1 j H 1/2 Gj HT T ɛ +2{1 + C (γ)}c(γ) H 1/2 G H 1/2 3(p+1) =2(p+1)+1 { 2γ{1 + C (γ)}d 1 j d H 1/2 G H 1/2 for j p + 1 2C{1 + C (γ)}c(γ) H 1/2 G H 1/2 for p + 1 j s, H 1/2 G HT T ɛ 2(1 + C (γ)) H 1/2 G HT T ɛ + sup H 1/2 G HT T ɛ 2{1 + C (γ)} H 1/2 G HT T ɛ + δ S γ If we tae γ smaller enough such that s j < 1, then ψ 2p 3 H 1/2 G HT T ɛ. s j H 1/2 Gj HT T ɛ. s sup δ S γ H 1/2 G HT T ɛ 2{1 + C (γ)} H 1/2 G HT T ɛ For any g > 4 and some constant C, + 2{1 + C (γ)}(1 s j ) 1 j sup δ S γ H 1/2 Gj HT T ɛ. s j H 1/2 G j HT T ɛ. E H 1/2 G j HT T ɛ g = E ɛ T T HG j HG j HT T ɛ g/2 = E ɛ T P V j P V j P ɛ g/2 Ctr g/2 (P V j P V j ). Hence the frst term n 3 l(δ)/( δ ) can be bounded by (S6) (S7) (S8) ɛ T P Ṽ P Ṽ j P Ṽ P ɛ = ɛ T T H G H Gj H G HT T ɛ λ max ( H 1/2 Gj H1/2 ) H 1/2 G HT T ɛ H 1/2 G HT T ɛ C 1 (γ)λ max (H 1/2 G j H 1/2 ) H 1/2 G HT T ɛ H 1/2 G HT T ɛ. Combnng (S8) and the above two nequalty, t can be seen that ( d ) g E sup ɛ T P Ṽ P Ṽ j P Ṽ P ɛ d d j d δ S γ C g 1(γ)λ g max(h 1/2 G j H 1/2 )E( 1 d d sup δ S γ H 1/2 G HT T ɛ sup δ S γ H 1/2 G HT T ɛ ) g C g 1(γ)λ g max(h 1/2 G j H 1/2 ). 7

8 We choose γ small enough such that C 1 (γ)λ max (H 1/2 G j H 1/2 ) <. The other terms n 3 l(δ)/( δ ) can be bounded smlarly. For example, ɛ T P Ṽ P Ṽ P ɛ H1/2 G HT T ɛ H 1/2 G HT T ɛ, where the bound for the rght hand sde can be obtaned smlarly as of (S8). condton (v) n Theorem 2.1 of Das et al. (2004) holds. Notce that from (S5), (A) j = E( 2 l(δ) { ) = 2 1 tr(q j Σ) tr(p V j P V ) tr(p V } ) = 2 1 tr(p V P V j ). Therefore, Then the (, j)th component of D1 1 AD1 1, where D 1 = Dag(d 1,, d n ), s tr(p V P V j )/(d d j ). Condton () n Das et al. (2004) s equvalent to requre that the smallest egenvalue of (D 1 AD 1 ) must be bounded away from 0 and. (D 1 AD 1 ) s λ mn. Snce λ x T ( D 1 )( A)( D 1 )x mn = nf x 0 x T x we requre that x T D 2 x λ max ( A) nf x 0 x T x Suppose the smallest egenvalue of λ max ( A)(mn (d )) 2 <, λ max ( A) = O(mn d 2 ). (S9) Under condton (S9), condton () of Das et al. (2004) holds. Therefore, condtons ()-(v) n Theorem 2.1 of Das et al. (2004) hold and g can be any nteger greater than 4. Ths fnshes the proof of Lemma 3. Lemma 4 Defne t(δ) = l T bf + m T ν as the BLUP estmator of Y 0 (t m ; δ) for some specfc and δ be the REML estmator of δ. If condtons (a)-(d) hold, then { t(δ) } 2 E{t( δ) t(δ)} 2 = E δ ( δ δ) + o(n 1 ). Proof: For convenence, let us defne ũ := (ũ T 1,, ũ T n) T = Y Z F bf, u = Y Z F b F and ζ T (δ) := m T GW T = (ζ1 T (δ),, ζn T (δ)) where ζ T (δ) = Z T R 0 (t m )cov(b R )Z T R f Z T R 0 (t m )cov(b R )Z T R 0 +Σ (m) u0 + p q=1 X (t m )[Σ uq Dag(X q0 )] (m) f =. where s the area we are nterested n predctng (n the man text, we used nstead of. In ths supplemental, we used ), Σ (m) u0 s the mth row of Σ u0 and [Σ uq Dag(X q0 )] (m) s the 8

9 mth row of Σ uq Dag(X q0 ). Let C 1 and C 2 be constants whch may tae dfferent values n each appearance. By the Taylor expanson of t( δ) around δ, we have where δ δ δ δ. Then t( δ) t(δ) = t(δ) δ ( δ δ) ( δ δ) T 2 t(δ ) δ 2 ( δ δ) E{t( δ) t(δ)} 2 = E{ t(δ) { t(δ) δ ( δ δ)} 2 + E δ ( δ δ)( δ δ) T 2 t(δ ) } ( δ δ) δ { 2 +(1/4)E ( δ δ) T 2 t(δ ) } 2 { t(δ) } 2 ( δ δ) := E δ 2 δ ( δ δ) + R1 + R 2. Frst, we would le to show R 2 = o(n 1 ). Notce that { E ( δ δ) T 2 t(δ ) } 2 { ( δ δ) = E[ tr 2 2 t(δ ) }] ( δ δ)( δ δ) T δ 2 δ ( { 2 E tr ( 2 t(δ ) } ]) ) 2 tr [{( δ δ)( δ δ) T } 2 δ ( { 2 = E tr ( 2 t(δ ) }{ } 2 ) ) 2 ( δ δ) T ( δ δ) δ 2 = s =1 s [ E Because s s a fxed number, we only need to show that [ E The frst dervatve of t(δ) s ( 2 t(δ ) δ δ j t(δ) = l T b F ( 2 t(δ ) δ δ j } 2 ] ) {( δ 2 δ) T ( δ δ). } 2 ] ) {( δ 2 δ) T ( δ δ) = o(n 1 ). (S10) + bt (δ) ũ b T b F (δ)z F, where b F / = (Z T F Σ 1 Z F ) 1 Z T F Σ 1 ( Σ/ ) ũ and the second dervatves of t(δ) s 2 t(δ) T 2 bf = l + 2 ζ T (δ) ũ ζt (δ) b F Z F := J 1 (δ) + J 2 (δ) + J 3 (δ) + J 4 (δ) + J 5 (δ), ζt (δ) b F Z F ζ T 2 bf (δ)z F 9

10 where 2 bf = (ZF T Z F ) 1 Z T 1 Σ F Σ Z F (ZF T Z F ) 1 Z T 1 Σ F Σ Z F ũ (ZF T Z F ) 1 ZF T 1 Σ Σ Z F (ZF T Z F ) 1 Z T 1 Σ F Σ Z F ũ +(ZF T Z F ) 1 ZF T 1 Σ Σ Σ ũ +(ZF T Z F ) 1 ZF T 1 Σ Σ Σ ũ 2 Σ (ZF T Z F ) 1 ZF T ũ := I 1 (δ) + I 2 (δ) + I 3 (δ) + I 4 (δ) + I 5 (δ). Let us loo at J 1 (δ). We can wrte J 1 (δ) = l T {I 1 (δ) + I 2 (δ) + I 3 (δ) + I 4 (δ) + I 5 (δ)}. Snce lt I 1 (δ ) and l T I 2 (δ ) are smlar, we only show that l T I 1 (δ ) s bounded by l T I 1 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ), where C 1 and C 2 are some constants. By the Cauchy- Schwarz nequalty and /2 Z T F (ZT F Σ 1 Z F ) 1 Z F /2 beng an dempotent matrx, we have l T I 1 (δ) = l T (ZF T Z F ) 1 ZF T 1 Σ Σ Z F (ZF T Z F ) 1 Z T 1 Σ F Σ Z F ũ l T (ZF T Z F ) 1 ZF T 1 Σ Σ Σ Z F (ZF T Z F ) 1 l 1/2 ũ T 1 Σ Σ Σ ũ 1/2. Denote d T = l T (Z T F Σ 1 Z F ) 1 Z T F /2 and P 1 (δ) = l T (ZF T Z F ) 1 ZF T 1 Σ Σ Σ Z F (ZF T Z F ) 1 l. Then we can wrte P 1 (δ) = = = d T Σ Σ δ d j =1 [tr(d d T d d T )] 1/2 [tr(/2 Σ =1 (d T d )[tr(/2 Σ =1 (d T d )tr 2 (/2 Σ /2 ) =1 =1 (d T d )tr 2 ( Σ ). Σ /2 )] 1/2 Σ /2 )] 1/2 10

11 Smlar to the proof of Lemma 2, we have tr 2 {Σ 1 ( Σ / δ j )} C 1 tr 2 { ( Σ / )}(1 + C 2 δ δ ) and assumng that max 1n tr 2 { ( Σ / )} <, we have P 1 (δ ) C 1 max 1n tr2 ( Σ )(1 + C 2 δ δ ) = C 1 max 1n tr2 ( Σ )(1 + C 2 δ δ ) d T d = C 1 max 1n tr2 ( Σ )(1 + C 2 δ δ ) l T (ZF T Σ 1 Z F ) 1 l. =1 lt (ZF T Σ 1 Z F ) 1 ZF T Σ 1 Z F (ZF T Σ 1 Z F ) 1 l It can be shown that l T (Z T F Σ 1 Z F ) 1 l = C1 lt (Z T F Σ 1 Z F ) 1 l(1 + C2 δ δ ). Snce ZF T Σ 1 Z F = O(n 1 ) and max 1n tr 2 { ( Σ / } <, P 1 (δ ) C 1 n 1 (1 + C 2 δ δ ). Next, ũ T 1 Σ Σ Σ ũ = Hence ũ T Σ 1 Σ δ =1 =1 [tr(/2 ũ ũ T ũũ T /2 )] 1/2 [tr( Σ Σ ) 2 ] 1/2 ũ T 1 Σ ũtr(σ Σ ) ũ T ũtr 2 ( Σ ). =1 =1 Σ 1 Σ Σ 1 ũ C δ 1 max 1n tr2 ( Σ )(1 + C 2 δ δ ) ũ T Σ 1 ũ C 1 u T u (1 + C 2 δ δ ). (S11) Note that we used the fact that ũ T ũ u T u. It follows that l T I 1 C 1 n 1/2 u T u 1/2 (1+ C 2 δ δ ). Smlarly, l T I 2 C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). The thrd term n J 1 s l T I 3 = l T (ZF T Z F ) 1 ZF T 1 Σ Σ Σ ũ l T (ZF T Z F ) 1 l 1/2 ũ T 1 Σ Σ Σ Σ Σ ũ 1/2. Applyng the nequalty tr(a 2 ) tr 2 (A) for any nonnegatve matrx A, we have ũ T 1 Σ Σ Σ Σ Σ ũ = ũ T Σ Σ Σ Σ =1 [tr(/2 ũ ũ T ũũ T /2 )] 1/2 [tr( Σ Σ Σ Σ δ )2 ] 1/2 =1 ũ T ũtr 2 ( )2 tr( Σ ) 2 tr( Σ ) 2. =1 11 ũ

12 Hence, ũ T Σ 1 Σ δ C 1 max C 1 max tr 2 ( tr 2 ( Σ 1 Σ δ j Σ 1 Σ δ j )2 max tr( Σ )2 max tr( Σ ) 2 max ) 2 max Σ 1 Σ Σ 1 ũ δ =1 ũ T Σ 1 tr( Σ ) 2 (1 + C 2 δ δ ) tr( Σ ) 2 (1 + C 2 δ δ ) ũ tr 2 (Σ 1 ) 2 tr( Σ =1 =1 ũ T Σ 1 ũ u T u, δ j ) 2 tr( Σ ) 2 δ and t s easy to see that l T (Z T F Σ 1 Z F ) 1 l C1 l T (Z T F Σ 1 Z F ) 1 l (1 + C2 δ δ ). Therefore, l T I 3 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). Smlarly, we can bound l T I 3 (δ ) and l T I 5 (δ ). So, n summary, J 1 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). Next, for J 3, we have J 3 = ζt (δ) b F Z F Notng that for = ζt (δ) ζt (δ) Z F (ZF T Z F ) 1 ZF T ζ T (δ) = ZR T 0 (t m )cov(b R )ZR T Σ j Z F (ZF T Z F ) 1 ZF T 1 Σ Σ ũ ζ(δ) 1/2 ũ T 1 Σ Σ Σ ũ 1/2. + ZR T 0 (t m ) cov(b R) Z T R = O(n 1/2 ), where cov(b R )/ = 0 f δ j σb 2 and cov(b R )/ = Dag(0,, Dag(λ 1 L 1 ),..., 0) f δ j = σb 2, and ζ T 0 (δ)/ = O(1) for all δ j. Hence, { ζ T (δ)/ }Z F s of order O(1) for each component. It follows that { ζ T (δ)/ }Z F (Z T F Σ 1 Z F ) 1 Z T F { ζ(δ)/} = O(n 1 ). We have already shown n (S11) that ũ T Σ 1 Σ δ Σ 1 Σ Σ 1 ũ C δ 1 u T u (1 + C 2 δ δ ). Therefore, J 3 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). Smlarly, we can show the same bound for J 4 (δ ). Now let us chec J 5, the proof s almost the same as J 1, where we replace l by ζ T (δ)z F. Notce that each component of ζ T (δ)z F s O(1). Then ζ T (δ)z F (Z T F Σ 1 Z F ) 1 Z T F ζ(δ) = O(n 1 ). Hence, as we have shown for J 1, t can also be shown that J 5 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). It remans to show J 2 = O p (1). Notce that for, ζ T (δ) = ZR T 0 (t m ) cov(b R) j ZR T 1 Σ 1 + ZR T 0 (t m )cov(b R )ZR T Σ + Z T R 0 (t m )cov(b R )Z T R + ZR T 0 (t m ) cov(b R) Σ 2 Σ Σ = O(n 1/2 ), 12 ZR T 1 Σ 1 + Z T R 0 (t m )cov(b R )Z T R 1 Σ Σ

13 and { ζ T 0 (δ)/ } = O(1). It follows that J 2 = ζt (δ) ζ T ũ = (δ) ũ ζt (δ) ũ =1 =1 =1 ( ζt (δ) Σ ζ T (δ) ) 1/2 (ũ T ũ) 1/2, where ũ = (ũ 1,, ũ m ) T. It s easy to see that ũ T Σ 1 ũ = O p (1). Hence J 2 (δ ) C 1 n 1/2 u T u 1/2 (1 + C 2 δ δ ). In summary, from J 1 (δ ) J 5 (δ ), 2 t(δ ) δ C 1 n 1/2 (u T u) 1/2 (1 + C 2 δ δ ), δ j where C s some constant. Applyng the Cauchy-Schwarz nequalty, R 2 = E(( 2 t(δ ) 2) ) [( δ 2 δ) T ( δ δ)] 2Cn 1 {E(u T u δ δ 4 ) + E(u T u δ δ 6 )} δ δ j [ 2Cn 1 {E(u T u) 2 } 1/2 {E( δ δ 8 )} 1/2 +{E(u T u) 2 } 1/2 {E( δ δ 12 )} 1/2 ]. Because E(u T u) 2 = O(n 2 ) and E( δ δ 8 ) = O(n 4 ), we have R 2 = o(n 1 ). To show the order of R 1, we would le to now the order of E{ t(δ) s δ ( δ δ)} 2 C {E( t(δ) ) 4 } 1/2 {E( δ δ ) 4 )} 1/2. Now we can rewrte t(δ)/ n the followng form ( t(δ) = f j (δ) ζ(δ)z F (ZF T Z F ) 1 Z T Σ F =1 D + b(δ) ) D ɛ := h j (δ) T ɛ, (S12) where f j (δ) = l T (ZF T Σ 1 Z F ) 1 ZF T ( Σ/)D, D = I Z F (ZF T Σ 1 Z F ) 1 ZF T Σ 1. Defne h (2) j (δ) T = ζ T (δ)z F (ZF T Σ 1 Z F ) 1 ZF T ( Σ/)D and h (3) j (δ) = { ζ T (δ)/ }D. Snce ɛ N(0, Σ), E( t(δ) ) 4 = 3(h j (δ) T Σh j (δ)) 2 6{(f T j (δ)σf j (δ)) 2 + (h (2) j (δ) T Σh (2) j (δ)) 2 + (h (3) j (δ) T Σh (3) j (δ)) 2 }. Defne B = Z F (Z T F Σ 1 Z F ) 1 Z T F. Notce that Σ 1/2 B/2 s an dempotent matrx. Then the frst term on the rght hand sde of (S12) s fj T (δ)σf j (δ) = l T (ZF T Z F ) 1 ZF T Σ DΣD Σ Z F (ZF T Z F ) 1 l = l T (ZF T Z F ) 1 ZF T Σ (Σ B) Σ Z F (ZF T Z F ) 1 l = l T (Z T F Z F ) 1 Z T F =1 Σ Σ Σ Z F (Z T F Z F ) 1 l lt (Z T F Z F ) 1 Z T F Σ Σ Σ Z F (Z T F Z F ) 1 l λ max ( Σ Σ Σ ) lt (ZF T Z F ) 1 ZF T Z F (ZF T Z F ) 1 l, =1 13

14 whch s of order O(n 1 ). Smlarly, the second term of the rght hand sde of (S12) s h (2) j (δ) T Σh (2) j (δ) = ζ(δ)b Σ (Σ B) Σ Bζ T (δ) ζ T (δ)b Σ Σ Σ Bζ(δ) ζ T (δ)σ Σ Σ Σ Σζ(δ) T ζ T Σ Σ (δ)σ Σ Σ ζ (δ). =1 { } If λ 1 L 1 = O(n 1/2 (2) ) and λ max ( Σ / )Σ ( Σ / )Σ <. Then h j (δ) T Σh (2) j (δ) = O(1). Then the thrd term on the rght hand sde of (S12) s h (3) j (δ) T Σh (3) j (δ) = ζt (δ) = ζ(δ) = =1 DΣD T ζ(δ) = ζ(δ) {Σ Z F (ZF T Z F ) 1 ZF T } ζ(δ)t Σ 1/2 {I /2 Z F (ZF T Z F ) 1 ZF T /2 1/2 ζ(δ)t }Σ ζ (δ) ζ (δ) T Σ λ max (Σ ) =1 ζ (δ) ζ (δ) T. ζ(δ) Σ ζ(δ)t If λ max (Σ ) < and λ 1 L 1 = O(n 1/2 ), then ζ (δ)/ = O(n 1/2 ) and hence h (3) j (δ) T Σh (3) j (δ) = O(1). Hence, E {( t(δ)/ ) 4 } = O(1). It follows that E{( t(δ)/ δ)( δ δ)} 2 = O(n 1 ). Agan by the Cauchy-Schwarz nequalty, t s easy to see that R 1 = o(n 1 ). Therefore, we have Ths completes the proof of Lemma 4. E{t( δ) t(δ)} 2 = E{ t(δ) δ ( δ δ)} 2 + o(n 1 ). 14

15 Some addtonal detals n the proof of Theorem 2: For K 1, because m T G m = Z T R 0 (t m )cov(b R )Z R0 + e T m,0(i n Σ u0 )e m,0 + =1 et,x (I n Σ u )e,x, m T ( 2 G/ ) m s a summaton of fxed number functons of varance components δ. Therefore, t can be shown that K1 K 1 C δ δ. For K 2, notce that { 2 γ T (δ) 0 both δ and δ j are σ b 2 s; γ(δ) = δ 2 γ T (δ) 0 j γ (δ) f one of δ and δ j s not σb 2 s, where γ 0 (δ) s the th m-dmensonal subvector of γ T (δ) = (γ T 1 (δ),, γ T m(δ)). Therefore, K2 K 2 tr{γ 0 (δ ) 2 γ T 0 (δ ) Σ 1 } tr{γ 0 (δ) 2 γ T 0 (δ) } tr{(γ 0 (δ ) 2 γ T (δ ) γ 0 (δ) 2 γ T 0 (δ) } δ ) j + tr{γ 0 (δ) 2 γ T 0 (δ) (Σ 1 δ )} j + tr{(γ 0 (δ ) 2 γ T (δ ) γ 0 (δ) 2 γ T 0 (δ) )(Σ 1 )} := K(1) 2 + K (2) 2 + K (3) From Lemma 2 we now tr(σ 1 ) C δ δ, hence to show that K2 K 2 C δ δ, t s enough to show that ( 2 γ (l) (δ )/ )γ () (δ ) ( 2 γ (l) (δ)/ )γ () (δ) C δ δ where subscrpt (l) denotes the lth component. Notce that 2 γ (l) (δ ) γ () (δ ) 2 γ (l) (δ) (δ) C 2 γ (l) (δ ) 2 γ (l) (δ) + C 2 γ (l) (δ ) 2 γ (l) (δ) γ () + C γ () γ () 2. (δ ) γ () (δ) (δ ) γ () (δ). Clearly, γ () (δ ) γ () (δ) C δ δ from the expresson of γ () (δ) and t also easy to show that 2 γ (l) (δ )/ 2 γ (l) (δ)/ C δ δ. It follows that K2 K 2 C δ δ. The dervaton of K 3 to K 7 are smlar, here we only gve the detals for K 4. We frst wrte K 4 K 4 C C =1 =1 γt (δ ) Σ 1 Σ Σ 1 (K () 41 + K () 42 + K () 41 K () 42 ), γ (δ ) γt (δ) Σ δ γ (δ) j where K () 41 = tr[{γ (δ )( γ T (δ )/ ) γ (δ)( γ T (δ)/)} ( Σ / ) ] and K() tr[{σ 1 ( Σ / δ j )Σ 1 ( Σ / ) }γ (δ)( γ T (δ)/)]. It can be seen that K () 41 = tr{(γ (δ ) γ (δ)) γt (δ) + tr{(γ (δ ) γ (δ))( γt (δ ) Σ δ } + tr{γ (δ)( γt (δ ) j γt (δ) 15 ) Σ δ }. j γt (δ) 42 = ) Σ δ } j

16 For, γ (l) (δ ) γ (l) (δ) C λ L 1 1 p =0 σ2 b σb 2 and each element of γ T (δ)/ s of the same order of λ L1 1. Hence, tr{(γ (δ ) γ (δ))( γ T (δ)/) ( Σ / ) } C λ L1 2 δ δ. Smlarly, we can show the other terms are also bounded by C λ L1 2 δ δ. It follows that K () 41 C λ L1 2 δ δ f. By notng that tr(σ 1 )2 C δ δ, tr{( Σ / δ j ) ( Σ / )} 2 C δ δ, γ (δ) and γ T (δ)/ are both O( λ L1 1 ) for, t can be shown that K () 42 C λ L1 2 δ δ for. For =, K () 41 C δ δ and K () 42 C δ δ. In summary, usng the assumpton λ L1 = O(n 1/2 ), we have K 4 K 4 C( λ L ) δ δ C δ δ. Here we show that g 4 (δ)/ = o(n 1/2 ). Observe that g jl 4 (δ) δ = ηt j δ ΣP V j P V l P Ση l + η T j By the Cauchy-Schwarz nequalty, Σ δ P V j P V l P Ση l + η T j ΣP V P P V j P V l P Ση l +η T j ΣP V j δ P V l P Ση l + η T j ΣP V j P V P V l P Ση l + η T j ΣP V j P V l δ P Ση l +η T j ΣP V j P V l P V P Ση l + η T j ΣP V j P V l P Σ δ η l + η j ΣP V j P V l P Σ ηt l δ. η T j ΣP V j δ P V l P Ση l (η T j ΣP V j δ P V j δ P Ση j ) 1/2 (η T l ΣP V l P V l P Ση l ) 1/2 By the defnton of η and h(δ) T Σh(δ) = o(n 3/2 ), we can see that η T j ΣP ( V j / δ )P V l P Ση l = o(n 1/2 ). The order of the other terms of g 4 (δ)/ can be derved smlarly. 16