Supplement to A theoretical framework for Bayesian nonparametric regression: random series and rates of contraction

Transcript

1 Supplemet to A theoretical framework for Bayesia oparametric regressio: radom series ad rates of cotractio A Proof of Theorem 31 Proof of Theorem 31 First defie the followig quatity: ɛ = 3 t α, δ = α α + 1 α ɛ, ζ = α α ɛ It follows from simple algebra that t > δ + 1 > ζ > αδ ad ζ > 1 ζ/α Set m = 1/α+1 log δ, ɛ = α/α+1 log t, ɛ = α/α+1 log ζ ad deote f m x = m β kψ k x give that N = m, ie, β k = 0 for al k > m We first verify coditio 5 with ω = 1 ad k = ɛ 1/α Clearly, k ɛ = O1 For sufficietly large, write f k x = = k f k x = f k x = β k ψ k x : f 0 f k ɛ, k k sice f 0 H α Q ad for sufficietly large, We proceed to boud f k x = k β k ψ k x : k=k +1 k β 0k β k ψ k x : β k ψ k x : 1 ɛ 1/α α β k β 0k ɛ k k k=k +1 k k=k +1 β k β 0k + β k β 0k ɛ β k β 0k ω k=k +1, β 0k ɛ β 0kk α 1 ɛ 1/α α 1 ɛ, f k x = k β k ψ k x : m β k β 0k β k : β k β 0k A 1/ k ɛ k A 1 k ɛ for some sequece A k such that A 1 k 1/ We pick A k = c k γ for some costat c > 0 Set c m = mi 1 k m mi k γ β k β 0k 1 gk γ β k It follows that mi k γ β k β 0k 1 gkγ β k mi exp τ 0 k γ β k τ β:k γ β β 0k 1 exp τ k γ β 0k τ ] exp τ Q τ ], mi exp τ 0 k γ β k β 0k + k γ β 0k τ ] β:k γ β β 0k 1 1

2 sice k γ β 0k k α β 0k β 0k k α Q for all k N +, as we assume that f 0 C α Q Therefore c m c 0 for some costat c 0 > 0 for all m N + Hece for sufficietly large, Π f k f 0 ɛ N = k Therefore Π f f 0 ɛ, k=k +1 = k k Π β k : k γ β k β 0k A 1/ k k γ ɛ N = k Π β k : k γ β k β 0k cɛ N = k = k k γ β 0k +cɛ k γ β 0k cɛ gk γ β k dk γ β k k ] cɛ mi β k :k γ β k β 0k 1 gkγ β k exp k logcc 0 k log 1 ] ɛ exp k log 1 ] ɛ β k β 0k ω Π f k f 0 ɛ N = k π N k exp ɛ 1/α log 1 ɛ b 0 ɛ 1/α log ɛ 1/α exp D 1/α+1 log 1 ζ/α] exp D 1/α+1 log ζ] = exp Dɛ ] for some costat D > 0 Now set δ = Q ad costruct the sieve F m δ as follows: F m Q = fx = β k ψ k x : k α β k β 0k Q Clearly, F m Q satisfies, m ɛ 0 Next we verify coditio 3 Note N j N ξjɛ, F m δ f f 0 j + 1ɛ, Write m F m δ f f 0 j + 1ɛ fx = β k ψ k x : β k β 0k j + 1 ɛ fx = β k ψ k x : β k β 0k k α Q It follows that N ξjɛ, F m δ f f 0 j + 1ɛ, ξjɛ N, m β 1,, β m : β k β 0k j + 1 ɛ,

3 N ξjɛ, β m+1,, : β k β 0k k α Q, We ow boud the two coverig umber separately For the first coverig umber, computatio of coverig umber i Euclidea space due to lemma 41 i Pollard 1990 yields ξjɛ N, m β 1,, β m : β k β 0k j + 1 ɛ, For the secod coverig umber, we see that m 6j + 1ɛ exp m log 4 ξjɛ / ξ ξjɛ N, β m+1,, : β k β 0k k α Q, ξjɛ 1/α 6Q N, β 1, β,, : β k β 0k k α Q, exp log4e α ], ξjɛ where the last iequality is due to the coverig umber of Sobolev ball see lemma 64 i Belitser ad Ghosal 003 We coclude that N j exp D 1 1/α+1 log δ] for some costat D 1 > 0, sice ɛ 1/α 1/α+1 log t/α 1/α+1 log δ m Hece j=m N j exp Dj ɛ exp D 1 1/α+1 log δ] j=m j j 1 exp Dɛ x dx exp D 1 1/α+1 log δ] exp Dɛ x dx M 1 exp D 1 1/α+1 log δ] exp 1 ] DM 1 1/α+1 log t 0 for sufficietly large, ad hece, coditio 3 holds We are ow left to show that F m Q satisfies 4 with the same costat D Write F c m Q = fx = fx = β k ψ k x : β k ψ k x : β k β 0k k α > Q βkk α > Q 4 sice by defiitio f 0 H α Q ad β 0k kα < Q /4 for sufficietly large Next write F c m Q fx = m=1 β k ψ k x : fx = β k ψ k x : βkk α > Q 4 β kk α > Q 4, N = m 3

4 = m=1 m=m +1 fx = m β k ψ k x : fx = m β k ψ k x : βkk α > Q 4, N = m m βkk α > Q 4, N = m m=m +1 N = m It follows that ΠF c m Q m=m +1 exp π N m exp b 1 m log m exp D 1/α+1 log δ+1] D + 1σ ] 1/α+1 log ζ = exp D + 1σ ] ɛ for some costat D > 0 whe is sufficietly large Hece coditio 4 holds with the same costat D B Proof of Theorem 3 Defie K α = 8Q 1/α 1/α+1 Let L to be the smallest iteger such that e L k = e L > K α, ad defie Lemma B1 For k defied above, ɛ = 1/α+1, ad f 0 H α Q, Π holds for sufficietly large k=k +1 Proof First write by the uio boud β k β 0k ɛ, β k β 0k Q 1 k=k +1 Π β k β 0k ɛ, β k β 0k Q k=k +1 k=k +1 Π β k β 0k ɛ + Π β k β 0k Q 1 k=k +1 k=k +1 = Π β k β 0k ɛ Π β k β 0k > Q k=k +1 k=k +1 By lemma 54 i Gao ad Zhou 016a, we kow that the fist term o the right-had side of the proceedig display is 1 o1 Hece it suffices to show that the secod term o the right-had side is o1 Write Π k=k +1 β k β 0k > Q 1 Q k=k +1 E Π β k β 0k 1 Q k=k +1 EΠ β k + β 0k ] 1/ 4

5 Q Q Q k=k +1 k=k +1 k=k +1 We are ow left with showig k>k EΠ β k = o1: k=k +1 E Π β k where the last iequality is due to 33 l=l 1 E Π β k + Q E Π β k + Q E Π β k + o1 k l+1 1 k=k l EΠ A l k=k +1 k=k +1 l=l 1 β 0k β 0kk α 1/ k=k +1 e l+1 cl / = o1, 1/ 1 k α Lemma B For the block prior Π defied i sectio 3 with f 0 H α Q for some α > 1/ ad Q > 0, there exists some costat D > 0 such that Π B k, ɛ, Q exp Dɛ, where ɛ = α/α+1, k = e L, L is the smallest iteger such that e L > K α, ad K α = 8Q 1/α 1/α+1 Proof By lemma B1 we have ΠB k, ɛ, Q = Π β k β 0k ɛ, k=k +1 Π β k β 0k ɛ, k>k 1 k Π β k β 0k ɛ k=k +1 β k β 0k Q k β k β 0k Q Π β k β 0k ɛ Exploitig the proof of 6 i theorem 1 i Gao ad Zhou 016a, we see that k Π β k β 0k ɛ exp D ɛ for some costat D > 0, ad thus the proof is completed Proof of theorem 3 Set ɛ = ɛ = α/α+1, δ = Q, ad m = κ 1 1/α+1, where κ is a costat determied later Let k =1 be defied as i lemma B ad ω = Q Clearly, m ɛ 0, k ɛ = O1, ad δ = O1, sice α > 1/ By assumptio f 0 H α Q, ad hece yields the followig series expasio f 0 x = β 0k ψ k x, where k α β0k Q 5

6 For coditio 5, by lemma B we see that it holds for some costat D > 0 with k ɛ = O1 ad ω = Q = O1 For coditio 3 ad 4, We use the followig sieve F m Q have F m Q = fx = β k ψ k x : k α β k β 0k Q We ext verify coditio 3 with the same costat D > 0 Followig the proof of theorem 31, we ξjɛ N j N, m β 1,, β m : β k β 0k j + 1 ɛ, ξjɛ N, β m+1,, : β k β 0k k α Q, For the first coverig umber, lemma 41 i Pollard 1990 yields N ξjɛ, m β 1,, β m : β k β 0k j + 1 ɛ, exp m log 4 ξ For the secod coverig umber, we obtai by lemma 64 i Belitser ad Ghosal 003 that N ξjɛ, β m+1,, : 1/α 6Q β k β 0k k α Q, exp log4e α ], ξjɛ We coclude that N j expd 1 ɛ for some costat D 1 > 0 by applyig the fact that ɛ 1/α ɛ = 1/α+1, ad hece, m N j exp Dj ɛ expd 1 ɛ j=m j=m expd 1 ɛ exp j exp Dɛ x dx expd 1 ɛ exp Dɛ x dx j 1 M 1 1 ] DM 1 ɛ 0 as log as M is sufficietly large Hece coditio 3 holds We are ow left to verify coditio 4 with the same costat D Set κ = mi 8e c 3 D + 1 ] α+1 3e σ, D Q + 1σ ] α+1, deote Π A = Π A for ay sequece A l l=1, ad defie the set A = A l e l for all l logm / 1 It follows that ΠF c m Q = ΠF c m Q A A ΠA + ΠF c m Q A A c ΠA c sup A A Π A F c m Q + ΠA c 6

7 Usig a similar argumet as that i page 340 i Gao ad Zhou 016a, for sufficietly large we have ΠA c l log m / 1 exp Π A l > e l 1 ] c 3 explog m log l log m / 1 exp exp c 3 e l exp 1 ] c 3 exp log m / 1 c 3 8e κ 1/α+1 1/α+1] = exp D + 1σ ] ɛ, where we use the fact that log m log m 1 ad m κ 1 1/α+1 / for sufficietly large For ay A A ad sufficietly large, the followig holds: Π A F c m Q Π β kk α + sice for sufficietly large, β 0k kα < Q /4 Write βkk α > Q 4 l:k l+1 m k l+1 1 k=k l l= logm / 1 l= logm / 1 β kk α > Q 4 k α l+1 β l > β 0kk α > Q Π l= logm / 1 l= logm / 1 l k α l+1 β l > Q = Q l l= logm / 1 βkk α > Q, 4 k l+1 1 kl+1 α k=k l β k > Q 4 l k α l+1a l χ l l > Q, sice for sufficietly large, l= logm / 1 l < 1/4, ad χ l l are idepedet χ l -radom variables We proceed to compute Π A F c m Q = l= logm / 1 l= logm / 1 l= logm / 1 Π α e αl+1 l e l χ l l > Q Π expα log + αl log l l χ l l > Q Π exp 1 l χ l l > Q for sufficietly large By the Cheroff boud for χ -radom variables, we obtai for sufficietly large Π A F c m Q l= logm / 1 l= logm / 1 l= logm / 1 ] exp l 4 log l Q e l / + l logq e l / ] exp el 4 log el Q e l / + e l logq e l / ] exp Q 4 el / exp Q 16 exp logm / 1 7

8 exp D + 1σ ] ɛ, where i the secod iequality we use the fact that e l l e l for sufficietly large whe l logm / 1 Therefore we coclude that ΠFm c Q sup Π A Fm c Q + ΠA c exp D + 1σ ] A A ɛ, ad coditio 4 holds with the same costat D C A geeral theorem for rates of cotractio with wavelet basis fuctios For a give iteger J ad positive real δ > 0, aalogous to, we require that the class of fuctios F J δ satisfies the followig property: F J δ fx = β jk ψ jk x : j/ max β jk β 0jk δ k I j j=0 k I j j=j holds for all J N + ad δ 0, The sieves F =1 are the costructed by takig F = F J δ for carefully chose sequeces J =1 N ad δ =1 0, We provide the correspodig local testig result below for the wavelet series, which is similar to lemma 1 Lemma C1 Let F J δ satisfies 1 The for ay f 1 F J δ with f 1 f 0 > 1, there exists a test fuctio φ : X Y 0, 1] such that 1 E 0 φ exp C f 1 f 0, sup E f 1 φ exp C f 1 f 0 f F J δ: f f 1 ξ f 0 f 1 + exp C f 1 f 0 J f 1 f 0 + δ for some costat C > 0 ad ξ 0, 1 Proof Take ξ = 1/4 Followig the proof of lemma 1, we obtai the followig bouds for the type I ad type II errors: E 0 φ exp C 1 3 f 1 f 0 E f 1 φ exp C 1 3 f 1 f 0 + exp C f 1 f 0 f 1 f 0 + exp 1 4 f 1 f 0 4 /104 g + f 1 f 0 g /3 for ay f F J δ such that f f 1 ξ f 0 f 1, where g = f f 1 f 1 f 0 /16, ad ξ ca be 8

9 take as ξ = 1/4 The rest of the proof proceeds as follows Notice that for f F J δ, oe has f f 0 j=0 J 1 J 1 J 1 j/ max β jk β 0jk j/ max β jk β 0jk + δ k I j k I j j=0 j 1/ J 1 j=0 β jk β 0jk j=0 k I j 1/ j=0 j/ + δ J/ f f 0 + δ Usig this fact ad the fact that f f 1 f 0 f 1, we obtai k Ijβ jk β 0jk 1/ + δ g f 0 f 1 J f 1 f 0 + δ, g J f 0 f 1 + δ The proof is thus completed Similar to lemma 3, a aalogous result for the wavelet series that guaratees a expoetially small lower boud for the margial likelihood expλ f D Πdf i terms of B J, ɛ, ω is also provided below Lemma C Suppose sequeces ɛ =1, ω =1 0, ad j =1 satisfy ɛ, j ɛ = O1, ad ω = O1 The for ay costat C > 0, P 0 expλ Πdf Π B j, ɛ, ω exp C + 1σ ] ɛ 0 Proof of lemma C The proof is quite similar to that of lemma 3 Deote the re-ormalized restrictio of Π o B = B j, ɛ, ω to be Π B, ad the radom variables V i, W i to be V i = f 0 x i fx i Πdf B, W i = 1 fx i f 0 x i Πdf B The by Jese s iequality H c : = 1 σ exp Λ f D Πdf ΠB exp Give the desig poits x i, we have P 0 e i V i Cɛ W i ɛ C + 1σ e i V i Cσ ɛ x 1,, x exp C σ 4 ɛ 4 P Vi, ɛ ] 9

10 Sice over the fuctio class B, we have f f 0 ɛ, j ɛ = O1, ω = O1, ad f f 0 j 1 j=0 j 1 j=0 j 1 j/ max β jk β 0jk + ω k I j j it follows from Fubii s theorem that 1/ j 1 j=0 j/ β jk β 0jk j=0 k I j 1/ k Ijβ jk β 0jk 1/ + ω + ω j/ ɛ + ω = O1, EVi f 0 f Πdf B ɛ, E ] Vi 4 E f 0 x fx 4 Πdf B f f 0 ɛ Hece by the Chebyshev s iequality, P P Vi E Vi > ɛ ɛ 1 ɛ 4 ɛ varv i 1 ɛ 4 ɛ EV i 4 1 ɛ 0 for ay ɛ > 0, ie, P V i = EV i + o P ɛ ɛ 1 + o P 1, ad hece, exp C σ 4 ɛ 4 P Vi = exp C σ 4 ɛ o P 1 i probability Therefore by the domiated covergece theorem the ucoditioal probability goes to 0: P 0 e i V i Cσ ɛ For the secod evet we use the Berstei s iequality Sice E exp C σ 4 ɛ 4 ] P Vi 0 EW i = 1 f f 0 Πdf B 1 ɛ, EWi 1 ] 4 E fx f 0 x 4 Πdf B f f 0 ɛ, the P W i > ɛ exp 1 ɛ 4 /4 4 EWi + ɛ W i / exp Ĉ1ɛ, where the last iequality is due to the fact that W i = 1 sup fx f 0 x Πdf B f f 0 = O1 x 0,1] p Hece, P i W i > ɛ 0, ad we coclude that PH c 0 10

11 The followig geeric rate of cotractio theorem ca be proved followig the exact same lies of that for theorem 1 For ay J N + ad ɛ, ω 0,, defie BJ, ɛ, ω = f = j=0 k I j β jk ψ jk : f f 0 < ɛ, j/ max β jk β 0jk ω k I j Theorem C1 Geeric Cotractio, Wavelet Series Let ɛ =1 ad ɛ =1 be sequeces such that miɛ, ɛ as with 0 ɛ ɛ 0 Assume that the sieve F J δ =1 satisfies 1 ad J ɛ 0 ad for some costat δ > 0 I additio, assume that there exist aother two sequeces j =1 N +, ω =1 0, such that j ɛ = O1 Suppose the coditios 3, 4, ad j=j ΠBj, ɛ, ω exp Dɛ hold for some costat ω, D > 0 ad sufficietly large ad M, with F J δ i replace of F m δ, ad Bj, ɛ, ω as is defied i Lemma C The E 0 Π f f 0 > Mɛ D ] 0 D Proof of Theorem 33 Defie j = 1 4 α α log 4 α 1 8Q + 1 α + 1 log Clearly oe has j ɛ 0 as sice α > 1/ Lemma D1 For j defied above, ɛ = 1/α+1, ad f 0 B,Q α with α > 1/, Π β j β 0j ɛ, j/ β j β 0j Q 1 j=j j=j holds for sufficietly large Proof The proof is similar to Lemma B1 ad is icluded here for the sake of completeess First write by the uio boud Π β j β 0j ɛ, j/ β j β 0j Q j=j j=j Π β j β 0j ɛ + Π j/ β j β 0j Q 1 j=j j=j = Π β j β 0j ɛ Π j/ β j β 0j > Q j=j j=j By lemma G4 i Gao ad Zhou 016b, we kow that the fist term o the right-had side of the proceedig 11

12 display is 1 o1 Hece it suffices to show that the secod term o the right-had side is o1 Write Π j/ β j β 0j > Q 1 j/ E Π β j β 0j 1 j/ E Π β j + β 0j 1/ Q Q j=j j=j j=j j/ E Π β j Q + j/ β 0j Q j=j j=j 1/ j/ E Π β j Q + 1 αj αj β 0j Q j=j j=j j=j = E Π β j Q + o1, j=j j/ where the last iequality is due to the fact that f 0 B α,q with α > 1/ We are ow left with showig j j j/ E Π β j = o1: j=j j/ E Π β j where the last iequality is due to 33 j=j j E Π A j j=j j/ cj / = o1, The followig Lemma is the wavelet couterpart of Lemma B The proof is icluded for completeess Lemma D For the block prior Π for the wavelet series defied i sectio 3 with f 0 H α Q for some α > 1/ ad Q > 0, there exists some costat D > 0 such that Π B j, ɛ, Q exp Dɛ, where ɛ = α/α+1 ad Proof By lemma D1 we have j=0 j = 1 4 α α log 4 α 1 8Q + 1 α + 1 log ΠB k, ɛ, Q = Π β j β 0j ɛ, j/ β j β 0j Q j=0 j=j Π β j β 0j ɛ, j 1 j/ β j β 0j Q Π β j β 0j ɛ j=j j=j j=0 1 j 1 Π β j β 0j ɛ Exploitig the proof of 6 i theorem 1 i Gao ad Zhou 016a ad together with lemma G ad lemma G3 i Gao ad Zhou 016b, we see that j 1 Π β j β 0j ɛ exp D ɛ j=0 1/ 1

13 for some costat D > 0, ad thus the proof is completed Proof of Theorem 33 The proof of Theorem 33 is very similar to that of Theorem 3 ad is icluded here for completeess We use basically the same setup as that i the proof of theorem 3 Set ɛ = ɛ = α/α+1, δ = ω = Q, j defied as i lemma D, ad J = log κ 1 /α + 1, where κ is a costat determied later Clearly, J log κ 1 /α log κ 1 1/α+1 ], ad hece J ɛ κ 1 1/α+1 α/α+1 0, j ɛ = O1, ad δ = ω = O1, sice α > 1/ By assumptio f 0 B α,q, ad hece yields the followig series expasio f 0 x = β 0jk ψ jk x, j=0 k I j where αj β0jk Q k I j j=0 Deote λ = j + k for each j, k-pair, ad write β j +k = β jk, β 0, j +k = β 0jk, ψ λ x = ψ jk x Sice I j = 0, 1,, j 1, j, k λ = j + k is oe-to-oe ad hece the two idex otatios are equivalet Thus we shall use the two idexes iterchageably For coditio, by lemma D we see that it holds for some costat D > 0 with j ɛ = O1 ad ω = Q = O1 For coditio 3 ad 4, We use a slightly differet sieve F J Q tha that i the proof of theorem 3 as follows: F J Q = fx = β jk ψ jk x : j=0 k I j j=j αj β k β 0k Q k I j We first argue that F J Q satisfies 1 I fact, for sufficietly large, j=j 1 αj 1, ad hece j=j j/ max k I j β jk β 0jk j=j j/ αj Namely, F J Q satisfies the property 1 j=j 1 αj αj jk β 0jk k Ijβ 1/ j=j αj 1/ β jk β 0jk k I j We ext verify coditio 3 Similar to the proof of theorem 35, we have N j N ξjɛ, β 1,, β J 1 : N ξjɛ, β J,, : J 1 j=j αj 1/ β λ β 0λ j + 1 ɛ, λ=1 β jk β 0jk Q, k I j We ow boud the two coverig umber separately For the first coverig umber, lemma 41 i Pollard Q 13

14 1990 yields N ξjɛ, β 1,, β J 1 : J 1 λ=1 For the secod coverig umber, we first observe that λ= J λ α β λ β 0λ = j=j = α β λ β 0λ j + 1 ɛ, exp J log 4 ξ k I j j + k α β jk β 0jk j=j αj j=j k I j β jk β 0jk α Q, ad the apply lemma 64 i Belitser ad Ghosal 003 to obtai N ξjɛ, β J, : N j=j αj ξjɛ, β J, : k I j αj+1 β jk β 0jk β k β 0k Q, k I j λ α β λ β 0λ α Q, exp log4e α α 1/α 6Q ] ξjɛ λ= J We coclude that N j expd 1 ɛ for some costat D 1 > 0 by applyig the fact that ɛ 1/α J ɛ = 1/α+1, ad hece, N j exp Dj ɛ expd 1 ɛ j=m j=m expd 1 ɛ exp j exp Dɛ x dx expd 1 ɛ exp Dɛ x dx j 1 M 1 1 ] DM 1 ɛ 0 as log as M is sufficietly large Hece coditio 3 holds We are ow left to verify coditio 4 with the same costat D Set κ = mi 4 D + 1 ] α+1 3 log σ, D Q + 1σ ] α+1, deote Π A = Π A for ay sequece A j j=0, ad defie the set A = A j exp j log for all j J It follows that ΠF c J Q = ΠF c J Q A A ΠA + ΠF c J Q A A c ΠA c sup A A Π A F c J Q + ΠA c 14

15 Usig a similar argumet as that i page 340 i Gao ad Zhou 016a, for sufficietly large we have ΠA c j J Π exp Aj > exp j log log ] 4 κ 1/α+1 1/α+1 j J exp exp j log exp 1 J log D + 1σ ] ɛ, where we use the fact that J log κ 1 1/α+1 ] 1 ad J κ 1 1/α+1 / for sufficietly large For ay A A ad sufficietly large, the followig holds: Π A Fm c Q Π j=j αj k I j β jk + j=j αj sice for sufficietly large, j=j αj k I j β 0jk < Q /4 Write j=j αj β0jk > Q Π k I j j=j αj βjk > Q, 4 k I j βjk > Q 4 = αj β j > Q 4 αj β j > Q j k I j j=j j=j j=j j αj β j > Q = j αj A j χ j j > Q, j=j j=j sice for sufficietly large, j=j j < 1/4, ad χ j j are idepedet χ j -radom variables We proceed to compute Π A F c J Q j=j Π j=j Π j αj A j χ j j > Q exp j χ j j > Q j=j Π exp log j + αj log j log χ j j > Q for sufficietly large By the Cheroff boud for χ -radom variables, we obtai for sufficietly large Π A F c J Q j=j exp exp j Q 16 J 4 j log Q e j / + j log Q e j / ] exp Q 1/α+1 κ 1/α+1 ] exp 3 exp Q 4 ej j=j / D + 1σ ɛ where we use the fact that J κ 1 1/α+1 / for sufficietly large whe j J Therefore we coclude that ΠFm c Q sup Π A Fm c Q + ΠA c exp D + 1σ ] A A ɛ, ad coditio 4 holds with the same costat D ], 15

16 E Proof of Theorem 34 The proof of Theorem 34 is immediate by combiig the proof of Theorem 1, Lemma E1, Lemma E, Lemma E3, ad Lemma E4 The proofs of these lemmas are similar to their couterparts i the mauscript, ad are preseted here for completeess The followig lemma is the key igrediet i bridgig the gap betwee the empirical L -distace ad the itegrated L -distace Lemma E1 Suppose the desig poits x i are fixed ad satisfy 35 Let F m δ be defied by 36 with a sequece m =1 such that m ad m / 0 Suppose f ad f 1 F m δ for all sufficietly large The for all sufficietly large, P f f 1 f f 1 Cm P f 0 f 1 f 0 f 1 Cm hold for some uiversal costat C > 0 idepedet of f 0, f 1, ad f f f 1 + Cδ f f 1, f 0 f 1 + Cδ f 0 f 1 Proof of lemma E1 Observe that for ay f = k β kψ k F m δ, the term-by-term dffieretiatio operatio is permitted, sice sup x 0,1] k=m+1 d dx β kψ k x k=m+1 k β k ψ k k=m+1 k α β 0k + k=m+1 β k β 0k k α As m, the first term o the right-had side of the preceedig display coverges to 0 by the defiitio of C α Q, ad the secod term also coverges to 0 by the defiitio of F m Q Hece the series k dβ kψ k x]/dx coverges uiformly over x 0, 1] Now suppose fx = β kψ k x permits term-by-term differetiatio We proceed to compute ad hece, d dx fx = d β k+1 dx ψ d k+1x + β k dx ψ kx = 1 0 fx d dx fx dx β k 1/ Sice the desig poits satisfy 35, lemma 71 i Yoo et al 017 yields P f f 1 = f x i 1 0 π f xdx 1 πkβ k+1 ψ k x + πkβ k ψ k x, 1/ 1/ k βk 1 + k βk f k βk 1 0 fx d dx fx dx 1 1/ f k βk 16

17 Observig that f, f 0, ad f 1 are all term-by-term differetiable, we obtai ad similarly, P f f 1 f f 1 1 f f 1 m β k β 1k + m 1 f f 1 m f f 1 + P f 0 f 1 f 0 f 1 1 f 0 f 1 1 f f 1 m f f 1 + m m β 0k β 1k + 1 f 0 f 1 m f 0 f f 0 f 1 m f 0 f 1 + By the defiitio of F m δ ad the fact that α > 1, we have ad hece, k β k β 1k = k β k β 0k + β 0k β 1k k β k β 0k + k β k β 0k + k β k β 1k ] 1/ k β k β 1k ] 1/ ] 1/ k β k β 1k, k β 0k β 1k ] 1/ k β 0k β 1k ] 1/ ] 1/ k β 0k β 1k k β 1k β 0k k β 1k β 0k 4δ, P f f 1 f f 1 Cm f f 1 + Cδ f f 1, P f 0 f 1 f 0 f 1 Cm f 0 f 1 + Cδ f 0 f 1 for some uiversal costat C > 0 The proof is completed by observig that m / = o1 Lemma E Suppose the desig poits x i are fixed ad satisfy 35 Let F m δ be defied as 36 with m, m / 0, ad δ is some costat The for ay f 1 F m δ with f 1 f 0 > 1, there 17

18 exists a test fuctio φ : Y 0, 1] such that E 0 φ exp C f 1 f 0, sup E f 1 φ exp C f 1 f 0 f F m δ: f f 1 ξ f 0 f 1 for some costat C > 0 ad ξ i 0, 1 Proof of lemma E Take ξ = 1/8 Sice f 0 f 1 > 1/, we obtai from lemma E1 that P f 0 f 1 f 0 f 1 Cm f 0 f 1 + Cδ f 1 f f 1 f 0 whe is sufficietly large, implyig that 3/4 f 1 f 0 P f 0 f 1 5/4 f 1 f 0 O the other had, P f f f f 1 + Cδ f f f 0 f 1 + Cξδ f 0 f 1 0/3 56 P f 0 f 1 + Cξδ f 0 f P f 0 f 1 + Cξδ f 0 f P f 1 f 0 Defie the test fuctio to be φ = 1T > 0, where T = y i f 1 x i f 0 x i 1 P f1 f0 We first cosider the type I error probability Uder P 0, we have y i = f 0 x i + e i, where e i s are iid Gaussia errors with Ee i = 0 ad Vare i = σ Namely, there exists a costat C 1 > 0 such that P 0 e i > t exp 4C 1 t for all t > 0 The for a sequece a i R, Cheroff boud yields P 0 a i e i t exp 4C 1t a i Now we set a i = f 1 x i f 0 x i ad t = P f 1 f 0 / The uder P 0, we have E 0 φ = P 0 T > 0 exp C 1 P f 0 f 1 exp C 1 16 f 0 f 1 We ext cosider the type II error probability Uder P f, we have y i = fx i + e i with e i s beig iid mea-zero sub-gaussia Sice P f f 1 P f 1 f 0 /16, we obtai T = e i f 1 x i f 0 x i ] + P f f 1 f 1 f P f 1 f 0 e i f 1 x i f 0 x i ] + 1 P f 1 f 0 P f f 1 P f 1 f 0 e i f 1 x i f 0 x i ] P f 1 f 0 18

19 Hece we use the sub-gaussia tail boud to obtai P f T < 0 P e i f 1 x i f 0 x i ] 1 4 P f 1 f 0 exp C 1 4 f 1 f 0 Hece we obtai the followig expoetial boud for type I ad type II error probabilities: E 0 φ exp C f 1 f 0, E f 1 φ exp C f 1 f 0 for some costat C > 0 for ay f f F m δ : f f 1 f 1 f 0 /64 The proof is completed by takig the supremum over f F m δ : f f 1 f 1 f 0 /64 Lemma E3 Suppose the desig poits x i are fixed ad satisfy 35 Let the sieve F mδ be defied by 36 Let ɛ =1 be a sequece with ɛ The there exists a sequece of test fuctios φ =1 such that E 0 φ j=m N j exp Cj ɛ, sup E f 1 φ exp CM ɛ, f F mδ: f f 0 >Mɛ where N j = N ξjɛ, S j ɛ,, S j ɛ = f F m δ : jɛ < f f 0 j + 1ɛ, M ca be sufficietly large, ad C is some positive costat Proof The proof is exactly the same as that of Lemma ad is omitted here Lemma E4 Suppose the desig poits x i are fixed Deote Λ f D = log p f y i ; x i p 0 y i ; x i to be the log-likelihood ratio, where p f y; x = φ σ y fx, ad p 0 = p f0 If ɛ the for ay costat C > 0, P 0 expλ Πdf Π f f 0 < ɛ exp C + 1σ ] ɛ 0 Proof Deote the re-ormalized restrictio of Π o B = f f 0 < ɛ to be Π B The by Jese s iequality H c : = exp Λ f D Πdf ΠB exp C + 1σ ] ɛ exp Λ f D Πdf B exp C + 1σ ] ɛ 1 ] σ e i f 0 x i fx i Πdf B C + 1 σ ɛ, 19

20 where we have used the fact that o the evet B, f f 0 ɛ, which implies, fx i f 0 x i Πdf B f f 0 Πdf B < ɛ Now we use the tail boud for sub-gaussia radom variables to obtai P 0 H c exp C + 1 ] 1 σ σ 4 ɛ 4 P f 0 fπdf B exp C + 1 ] 1 σ σ 4 ɛ 4 P f f 0 Πdf B exp C + 1 σ σ 4 ɛ 4 ] 1 f f 0 Πdf B exp C + 1 σ σ 4 ɛ ] 0 F Proof of Theorem 35 Proof of lemma 3 For a metric space X, d, deote the packig umber Dɛ, X, d to be the maximum umber of poits i X that are at least ɛ away from each other It is proved that Dɛ, X, d N ɛ, X, d Ghosal ad Va Der Vaart 001, ad therefore it suffices to work with the packig umber The proof here is similar to that of lemma 64 i Belitser ad Ghosal 003 Let ΘQ = β 1, β, l : k βk exp Q c Suppose β 1,, β m ΘQ are such that β i β j = ɛ wheever i j It suffices to cosider ɛ to be small eough, sice for large values of ɛ, Dɛ, ΘQ, = 1 Fixed a iteger N ad deote Θ N Q = β 1,, β N, 0, R N : N k βk exp Q c For ay β = β 1, β, ΘQ, deote β = β 1,, β N, 0, Θ N Q Now set N = c log8q /ɛ Clearly, It follows that β β = k=n+1 ] βk N + 1 exp k βk exp ɛ c c 8 k=n+1 ɛ = β i β j = β i β j + β i β i β j β j β i β j + β i β i + β j β j β i β j + ɛ, 0

21 implyig that β i β j ɛ/ For ay β i ad t = t 1,, t N, 0, Bβ i, ɛ/ R N, oe has N k t N k k exp βk exp + c c N k t k β k N exp Q ɛ + exp c c 8 4Q, ad thus m j=1 B β i, ɛ Θ N Q Sice Bβ i, ɛ/ s overlap o each other oly o a set of volume 0, the by deotig V N the volume of the uit ball i R N we obtai ɛ N/ N m V N Q N V N exp k, 8 c implyig that m exp N log 4 Q 1 6c N 3 + N log 1 ] ɛ Sice the maximum umber of m is the packig umber, the proof is completed by oticig that N log1/ɛ] 1/ Lemma F1 Suppose f Π = GP0, K where K is the squared-expoetial covariace fuctio, ad f 0 A 4 Q The for sufficietly small ɛ > 0 it holds that log Π f f 0 < ɛ log 1 ɛ Proof Deote H to be the reproducig kerel Hilbert space RKHS associated with K Defie the cocetratio fuctio φ f0 ɛ = 1 if f H log Π f < ɛ f H: f f 0 <ɛ By lemma 53 i va der Vaart ad va Zate 008, it holds that log Π f f 0 < ɛ φ f0 ɛ/ By theorem 41 i va der Vaart ad va Zate 008, H is the set of fuctios fx = β kψ k x such that β k /λ k < Sice λ k e k /4, there exists some costats λ, λ > 0 such that λe k /4 λ k λe k /4 Usig the fact that f 0 A 4 Q, we obtai β 0k λ k β0k k λ exp 1 4 λ Q < Therefore f 0 H, ad the first term i φ f0 ɛ is upper bouded by f 0 H / = O1 Furthermore, by lemma 46 i va der Vaart ad va Zate 009, the secod term i φ f0 ɛ is upper bouded by a costat multiple of log1/ɛ] The proof is thus completed Proof of theorem 35 Set ɛ = ɛ = 1/ log, δ = Q, ad m = log Clearly, m ɛ 0 ad 1

22 δ = O1 By assumptio f 0 A 4 Q, ad hece yields the followig series expasio f 0 x = β 0k ψ k x, where k β0k exp Q 4 Still let costats λ, λ be such that λe k /8 λ k λe k /8 Defie the sieve F m Q to be F m Q = fx = β k ψ k x : k β k β 0k exp Q 8 Clearly, F m Q satisfies the property 36 I fact, for ay f = k β kψ k F m Q, we directly compute by Cauchy-Schwartz iequality 1/ ] 1/ β k β 0k β k β 0k e /8] k 1 Q e k /8 k>m k>m k>m I light of Theorem 34, it suffices to verify the coditios 3, 4, ad Π f f 0 < ɛ e Dɛ some costat D > 0 By lemma F1, it holds for some costat D > 0 ad all sufficietly small ɛ > 0 that Π f f 0 < ɛ exp D log 1 ] ɛ for Usig the fact that ɛ log1/ɛ ] log, it follows that there exists some costat D > 0, such that Π f f 0 < ɛ exp Dɛ We ext verify coditio 3 with the same costat D > 0 Observe that N j N ξjɛ, F m δ f f 0 j + 1ɛ,, it suffices to boud the right-had side of the preceedig display Write m F m δ f f 0 j + 1ɛ fx = β k ψ k x : β k β 0k j + 1 ɛ k fx = β k ψ k x : β k β 0k exp Q 8 It follows that N ξjɛ, F m δ f f 0 j + 1ɛ, ξjɛ N, m β 1,, β m : β k β 0k j + 1 ɛ,

23 N ξjɛ, β m+1,, : β k β 0k e k /8 Q, k>m We ow boud the two coverig umbers separately For the first factor, computatio of coverig umber i Euclidea space due to lemma 41 i Pollard 1990 yields ξjɛ N, m β 1,, β m : β k β 0k j + 1 ɛ, For the secod coverig umber, lemma 3 yields N ξjɛ, β m+1,, : m 6j + 1ɛ exp m log 4 ξjɛ / ξ ] k β k β 0k 3/ exp Q 1, exp D 1 log 8 ξjɛ for some costat D 1 > 0 We coclude that N j expd 1 ɛ for some costat D 1 > 0 by applyig the fact that log1/ɛ ] 3/ + m ɛ log, ad hece, N j exp Dj ɛ expd 1 ɛ j=m j=m expd 1 ɛ exp j exp Dɛ x dx expd 1 ɛ exp Dɛ x dx j 1 M 1 1 ] DM 1 ɛ 0 as log as M is sufficietly large Hece coditio 3 holds Fially we verify coditio 4 with the same costat D By defiitio ΠF c m Q Π For sufficietly large, we have k β0k exp 8 k βk exp + 8 k β0k exp > Q 8 k β0k exp < Q /4 4 This is because by defiitio f 0 H α Q ad β 0k expk /4 Q Hece the preceedig display reduces to k ΠFm c Q Π βk exp > Q, 8 4 ad it suffices to boud the right-had side that Π k βk exp > Q Q exp By the Markov s iequality, it holds for sufficietly large k 8 Eβ k 4 Q k λ k exp 8 3

24 4 Q 4λ Q m exp λ exp k 4λ Q 8 x dx exp 8 m 16 k k 1 exp x dx 8 Sice m log 4 D + 1/σ log = D + 1/σ ɛ whe is sufficietly large, it follows that 4 is satisfied with the same costat D G Proof of Theorem 41 The proof of Theorem 41 follows exactly the same lies of that of Theorem 1 with the assist of Lemmas 1, G, ad G3 below that are variatios of Lemmas 1, G, ad 3, respectively Lemma G1 Let Gmδ A satisfies 4 The for ay f 1 Gmδ A with f 1 f 0 > 1, there exists a test fuctio φ : X Y 0, 1] such that E 0 φ exp C f 1 f 0, sup E f 1 φ exp C f 1 f 0 f Gm A δ: f f1 ξ f 0 f 1 + exp C f 1 f 0 A m f 1 f 0 + δ for some costat C > 0 ad ξ 0, 1 Proof We first observe the followig fact: for ay fx = j z j k β jkψ k x j G A mδ, the followig holds: f f 0 A m f f 0 + δ 3 I fact, by the Cauchy-Schwartz iequality, p f f 0 µ µ 0 + z j β jk β 0jk j=1 µ µ 0 + µ µ 0 + j j:z j=1 j 1,,j q j j:z j=1 j 1,,j q m z j β jk β 0jk + z j β jk β 0jk m fj f 0j + qδ, j j:z j=1 j 1,,j q k=m+1 ad hece, p p f f 0 µ µ 0 + A m z j f j f 0j + δ A mµ µ 0 + A m z j f j f 0j j=1 j=1 p p = A m µ + z j f j µ 0 + f 0j + δ = A m f f 0 + δ j=1 j=1 + δ 4

25 The rest of the proof is similar to that of Lemma 1 ad we oly sketch the proof Let us take ξ = 1/4 Defie the test fuctio to be φ = 1 T > 0, where T = y i f 1 x i f 0 x i 1 P f1 f0 8 f 1 f 0 P f 1 f 0 We first cosider the type I error probability Followig the proof of Lemma 1, it is immediate that E 0 φ exp C 1 3 f 1 f 0 We ext cosider the type II error probability For ay f with f f 1 f 0 f 1 /4 f 0 f 1 /4, followig the proof of Lemma G1, we have E f 1 φ exp C 1 + P + P 3 f 1 f 0 P f f 1 > 1 16 P f 1 f 0 G f 1 f 0 < f 1 f 0 Usig Berstei s iequality, we obtai the tail probability of the empirical process G f 1 f 0 P G f 1 f 0 < f 1 f 0 exp C f 1 f 0 A m f 1 f 0 +, δ for some costat C > 0, where we use the relatio 3 O the other had, whe P f f 1 > P f 1 f 0 /16, we agai use Berstei s iequality ad the fact that f f G A mδ : f f 1 5 f 0 f 1 to compute P P f f 1 > 116 P f 1 f 0 exp 1 4 where g = f f 1 f 1 f 0 /16 We further compute g f 1 f 0 4 /104 g + f 1 f 0 g /3 f f f 1 f 0 f f 1 f f f 1 f 0 f 1 f 0 f f 1 f f 1 + f 1 f 0 f 1 f 0 A m f 1 f 0 + δ f 0 f 1,, where we use 3, the fact that f f 1 f 0 f 1, ad that f f 1 f f 0 + f 0 f 1 A m f 1 f 0 + δ Similarly, we obtai o the other had, g = f f f 1 f 0 A m f 0 f 1 + δ 5

26 Therefore, we ed up with P P f f 1 > 1 16 P C f 1 f 0 f 1 f 0 exp A m f 1 f 0 +, δ where C > 0 is some costat Assemblig all the pieces obtaied above, we obtai the followig expoetial boud for type I ad type II error probabilities: E 0 φ exp C f 1 f 0, E1 φ exp C f 1 f 0 + exp C f 1 f 0 A m f 1 f 0 + δ for some costat C > 0 wheever f f 1 f 1 f 0 /3 Takig the supremum of the type II error over f f G m δ : f f 1 f 1 f 0 /3 completes the proof Exploitig the proof of Lemma, we obtai the followig lemma for the sparse additive models immediately by combiig Lemma G1 Lemma G Let m N + be a positive iteger, ad δ, A > 0 be positive Suppose that G A mδ satisfies 4 Let ɛ =1 be a sequece with ɛ The there exists a sequece of test fuctios φ =1 such that E 0 φ Nj A exp Cj ɛ, j=m sup E f 1 φ exp CM ɛ + exp CM ɛ f Gm A δ: f f0 >Mɛ A mm ɛ + δ, where N A j = N ξjɛ, S A j ɛ, is the coverig umber of S A jɛ = f G A mδ : jɛ < f f 0 j + 1ɛ, ad C is some positive costat The followig lemma for the sparse additive models is immediate by exploitig the proof of Lemma 3 ad observig the fact that f f 0 = O1 for ay f Bk, ɛ, ω give that k ɛ = O1 Lemma G3 Let Bm, ɛ, ω be defied as Theorem 41 Suppose sequeces ɛ =1 ad k =1 satisfy ɛ 0, ɛ, k ɛ = O1, ad ω is some costat The for ay costat C > 0, P 0 expλ Πdf Π Bk, ɛ, ω z 1 q exp C + 1σ ] ɛ 0 H Proof of Theorem 4 Lemma H1 Let m be a positive iteger, δ, A > 0, ad ξ 0, 1 is some absolute costat Assume that f 0j C α Q for some α > 1/ ad some Q > 0, j j 1,, j q Take G A mqq = z: z 1 Aq G mqq, z for 6

27 some positive iteger A, where p 1 G m qq, z = fx = µ + z j β jk ψ k x j, β j1 = β jk ψ k x j dx j, N = m 4 j=1 k= 0 The log N ξjɛ, S A jɛ, log 1 p Aqm + log, ξ Aq where S A j ɛ = f G A mqq : jɛ < f f 0 j + 1ɛ Proof of Lemma H1 Observe that Nj A N ξjɛ, GmqQ A f f 0 j + 1ɛ, N ξjɛ, G m qq, z f f 0 j + 1ɛ,, z 0,1 p : z 1 Aq ad that p f f 0 = µ µ 0 + z j β jk β 0jk, j=1 due to the fact that 1 0 f jx j dx j = 1 0 f 0jx j dx j = 0, j = 1,, p Write It follows that G m qq, z f f 0 j + 1ɛ p fx = µ + z j β jk ψ k x j : µ µ 0 + j=1 j:z j=1 m β jk β 0jk j + 1 ɛ, N ξjɛ, G m qq, z f f 0 j + 1ɛ, N ξjɛ, µ, β jk : z j = 1, k = 1,, m : µ µ 0 + Aqm+1 6j + 1ɛ exp Aqm + 1 log 1 ] ξjɛ ξ Therefore, N ξjɛ, S A jɛ, j:z j=1 p exp log 1 Aqm Aq ξ The proof is completed by takig the logarithm of the precedig display m β jk β 0jk j + 1 ɛ, Proof of Theorem 4 The proof is based o the proof of Theorem 41, alog with several modificatios We begi by defiig the followig quatity: ɛ = 3 t α, δ = α α + 1 α ɛ, ζ = α α ɛ 7

28 It follows from simple algebra that t > δ + 1 > ζ > αδ ad ζ > 1 ζ/α Without loss of geerality we may also assume that ɛ is small so that ζ < 1, sice cotractio for smaller t implies cotractio for larger t Set m = 1/4α+ log δ, A = 1/4α+ log, ɛ = α/α+1 log t, ɛ = α+1/4/α+1 log ζ ad deote f m x = µ + p j=1 give that N = m, ie, β jk = 0 for al k > m, j = 1,, p m ξ j β jk ψ k x j We first verify coditio 45 with ω = 1 ad k = 1/4α+ log ζ/α Clearly, k ɛ = O1 Observe that 1 0 ψ kx j dx j k 1, it follows from the Cauchy-Schwarz iequality that f f 0 = µ µ 0 + µ µ 0 + µ µ 0 + p z j β jk β 0jk j=1 j=1 k= k= 1 0 k= ψ k x j dx j ] + p ] ] β jk β 0jk 1 k + p j=1 k= z j β jk β 0jk p j=1 k= p j=1 k= β jk β 0jk β jk β 0jk Namely, f f 0 C 1 ψ µ µ 0 + p j=1 k= z jβ jk β 0jk ] for some costat C ψ > 0 For sufficietly large, write p p fx = µ + ξ j β jk ψ k x j : f f 0 < ɛ, z j β jk β 0jk 1 z 1 q j=1 j=1 k=k +1 f k x : µ µ 0 + k β jk β 0jk C ψ ɛ, N = k, z jr = 1, r = 1,, q, z 1 = q µ µ 0 < ɛ j:z j=1 k= k j:z j=1 k= β jk : β jk β 0jk A 1/ k ɛ j j 1,,j q z j = 1 j / j 1,,j q z j = 0 for some sequece A k such that A 1 k C ψ /q We pick A k = c k γ for some costat c > 0 Set c m = mi 1 k m mi kγ β k β 0k 1 gk γ β k Followig the calculatio i the proof of Theorem 31, we have mi k γ β k β 0k 1 gkγ β k exp τ Q τ ] Therefore c m c 0 for some costat c 0 > 0 for all m N + Hece for sufficietly large, Π Bk, ɛ, δ z 1 q Π µ µ 0 < ɛ N = k k j:z j=1 k= Π β jk : k γ β jk β 0jk kγ ɛ A 1/ k N = k q p q p p 8

29 Therefore ] ɛ mi πµ k Π β jk : k γ β jk β 0jk cɛ N = k exp q log p loge] µ µ 0 1 j:z j=1 k= ɛ k k γ β 0k +cɛ gk γ β k dk γ β k exp q log p loge] j:z j=1 k= k j:z j=1 k= k γ β 0k cɛ cɛ mi β k :k γ β k β 0k 1 gkγ β k ɛ 3 exp q log p exp 4qk log 1 Π Bk, ɛ, δ z 1 q ɛ ] q log p Π Bk, ɛ, δ z 1 = q N = k π N k exp 4q 1/4α+ log ζ/α log 1 ] q log p ɛ ] exp b 0 1/4α+ log ζ/α log 1/4α+ log ζ/α exp D 1/4α+ log 1 ζ/α] exp Dɛ for some costat D > 0 Now set δ = qq ad costruct the sieve Gm A qq = z G 1 A q m qq, z, where G m qq, z is give as i 4 Clearly, for ay fx = p m j=1 z jβ jk ψ k x j G m qq, z, p j=1 z j β jk β 0jk = p j=1 β 0jk p j=1 β 0jk k α qq Therefore G A m qq satisfies 4 Furthermore, A m ɛ = 1 α/α+1 log t+δ+1 0 Next we verify coditio 43 Ivokig Lemma H1, we see that N A j ] exp D 1 1/α+1 log δ+1 + A log p exp D 1 1/α+1 log t] = expd 1 ɛ for some costat D 1 > 0, ad hece Nj A exp Dj ɛ expd 1 ɛ j=m j=m expd 1 ɛ exp j exp Dɛ x dx exp D 1 ɛ exp Dɛ x dx j 1 M 1 1 ] DM 1 ɛ 0 for sufficietly large by takig M 1 + 4D 1 /D, ad hece, coditio 43 holds We are ow left to show that G A m qq satisfies 44 Followig the proof of Theorem 31, write G A m qq c z : z 1 A q m=m +1 N = m 9

30 A versio of the Cheroff s iequality for biomial distributio is of the form Therefore, It follows that PX > ap a ] p 1 exp a if X Biomial p, 1 ap p ad a 1/p Aq/p ] p 1 A q Π z : z 1 A q exp = exp A q A q loga q] A q p ΠG A m qq c m=m +1 π N m + Πz : z 1 A q exp b 1 m log m + exp A qlog A 1 A q log q] exp D mi 1/4α+ log δ+1, 1/4α+ log ] exp D + 1σ ] 1/4α+ log ζ = exp D + 1σ ] ɛ for some costat D > 0 whe is sufficietly large Hece coditio 4 holds with the same costat D Refereces Belitser, E ad Ghosal, S 003 Adaptive Bayesia iferece o the mea of a ifiite-dimesioal ormal distributio The Aals of Statistics, 31: Gao, C ad Zhou, H H 016a Rate exact Bayesia adaptatio with modified block priors The Aals of Statistics, 441: Gao, C ad Zhou, H H 016b priors Supplemet to rate exact bayesia adaptatio with modified block Ghosal, S ad Va Der Vaart, A W 001 Etropies ad rates of covergece for maximum likelihood ad Bayes estimatio for mixtures of ormal desities The Aals of Statistics, 95: Pollard, D 1990 Empirical processes: theory ad applicatios I NSF-CBMS regioal coferece series i probability ad statistics, pages i 86 JSTOR va der Vaart, A W ad va Zate, J H 008 Reproducig kerel Hilbert spaces of Gaussia priors I Pushig the limits of cotemporary statistics: cotributios i hoor of Jayata K Ghosh, pages 00 Istitute of Mathematical Statistics va der Vaart, A W ad va Zate, J H 009 Adaptive Bayesia estimatio usig a Gaussia radom field with iverse Gamma badwidth The Aals of Statistics, pages

31 Yoo, W W, Rousseau, J, ad Rivoirard, V 017 Adaptive supremum orm posterior cotractio: Spike-ad-slab priors ad aisotropic besov spaces arxiv preprit arxiv: