Section 4: Conditional Likelihood: Sufficiency and Ancillarity in the Presence of Nuisance Parameters

Transcript

1 70 Section 4: Conditional Likelihood: Sufficiency and Ancillarity in the Presence of Nuisance Parameters In this section we will: (i) Explore decomposing information in terms of conditional and marginal likelihood parts (ii) Give conditions (ancillarities) under which the conditional likelihood is efficient

2 71 Section 4.1 Partial sufficiency and ancillarity Suppose that X p(x; θ 0 ) P= {p(x; θ) :θ Θ}. We assume θ =(γ,λ),θ=γ Λ, where Γ is an open interval of the real line and Λ is an open set in R k. The parameter of interest is γ and the nuisance parameter is λ. Let T (X) be a statistic. Then, we know that p(x; θ) =h(x T (x);θ)f(t (x);θ) where h is the conditional density of X given T (X) andf is the marginal density of T (X).

3 72 Partial Sufficiency Definition: If h(x T (x); θ) = h(x T (x); λ) and f(t (x); θ) =f(t (x); γ), i.e., p(x; θ) =h(x T (x);λ)f(t (x);γ) then we say that T (X) ispartially sufficient for γ. The term partial is introduced because we must establish sufficiency for each fixed λ. Inference about γ can be made using only the marginal distribution of T (X). There will be no loss of information. If the score for γ is in the class of unbiased estimating functions, then the score is the most efficient member of the class. see Basu (JASA, 1977, pgs ).

4 73 Example 4.1: Partial Sufficiency Suppose that X = X where (1 γ)λ exp(λx) x 0 p(x; θ) = γλexp( λx) x>0 Here Γ = (0, 1) and Λ = R +. Show that T (X) =I(X >0) is partially sufficient for γ. Toseethis,notethat h(x T (x); θ) = (λ exp( λx)) T (x) 1 T (x) (λ exp(λx)) f(t (x); θ) = γ T (x) 1 T (x) (1 γ)

5 74 Partial Ancillarity Suppose h(x T (x); θ) =h(x T (x); γ) andf(t (x); θ) =f(t (x); θ), i.e., p(x; θ) =h(x T (x);γ)f(t (x);θ) Make inference about γ via the conditional distribution. We will make assumptions about the marginal distribution of T (X) in order to guarantee that there will not be information loss.

6 75 Section 4.2 Important types of partial ancillarity S-Ancillarity Definition: T (X) issaidtobes-ancillary for γ if f(t (x); θ) depends only on λ, i.e., p(x; θ) =h(x T (x);γ)f(t (x);λ) This definition is equivalent to letting T (X) be partially sufficient for λ. see Basu (JASA, 1977, pgs ).

7 76 R-Ancillarity Definition: T (X) issaidtober-ancillary for γ if there exists a reparameterization between θ =(γ,λ) and(γ,φ) such that f(t (x); θ) depends on θ only through φ. Thatis, p(x; θ) =h(x T (x);γ)f(t (x);φ) see Basawa (Biometrika, 1981, pgs ).

8 77 C-Ancillarity Definition: T (X) issaidtobec-ancillary for γ if for all γ Γ, the class {f(t (x); γ,λ):λ Λ} is complete. Completeness: E θ [m(t (X); γ)] = 0 for all λ Λ implies P θ [m(t (X); γ) =0]=1forallλ Λ see Godambe (Biometrika, 1976, pgs ).

9 78 A (Weak)-Ancillarity Definition: T (X) issaidtobea-ancillary if for any given θ 0 Θ and any other γ Γ, there exists a λ = λ(γ,θ 0 ) such that If this condition holds, then f(t; θ 0 )=f(t; γ,λ) for all t {f(t (x); γ,λ); λ Λ} isthesamewhateverbethevalueofγ. Intuitively, observation of T (X) cannot give us any information about γ when λ is unknown. see Andersen (JRSS-B, 1970, pgs ).

10 79 Example 4.2: Partial Ancillarity Let X =(Y,Z), where Y and Z are independent normal random variables with variance 1 and means γ and γ + λ, respectively. Here Γ=Λ=R. Let T (X) =Z. Then, we know that Y T (X) N(γ,1) and T (X) N(γ + λ, 1). T (X) isnot S-ancillary for γ. Let φ = γ + λ, sothatt (X) N(φ, 1) Then, T (X) is R-ancillary for γ. For fixed γ, weseethat f(t; θ) = (2π) 1/2 exp( 1 2 (t2 2γt 2λt +(γ + λ) 2 )) = (2π) 1/2 exp( 1 2 (t2 2γt)) exp( 1 2 (γ + λ)2 )exp(λt)

11 80 By exponential family results, we know that for fixed γ {f(t; γ,λ):λ R} is complete. Thus, we know that T (X) is C-ancillary. For fixed θ 0 and given γ, we define λ = λ(γ,θ 0 )= γ + γ 0 + λ 0. Then for all t, we know that So, T (X) is A-ancillary. f(t; θ 0 )=f(t; γ,λ)

12 81 Section 4.3 Information decomposition Information Decomposition Suppose h(x T (x); θ) =h(x T (x); γ) andf(t (x); θ) =f(t (x); θ), i.e., p(x; θ) =h(x T (x);γ)f(t (x);θ) Recall: The Fisher information for γ was defined as I γ (θ) = I γγ (θ) I γλ (θ)i λλ (θ) 1 I λγ (θ) = E θ [(ψ γ (X; θ) Π[ψ γ (X; θ) Λ θ ]) 2 ] The generalized Fisher information for γ from a class of unbiased estimating function G was defined as I γ(θ) =E θ [(ψ γ (X; θ) Π[ψ γ (X; θ) G γ ]) 2 ]

13 82 Let ψγ C (X; γ) andψγ M (X; θ) bethescoresforγ based on the conditional and marginal parts of the factorization of the density of X. Letψ λ (X; θ) =ψλ M (X; θ) be the score for λ from the marginal part of the factorization. Note that the nuisance tangent space Λ θ is the same as the nuisance tangent space from the marginal density of T (X). The Fisher information for γ contained in the conditional distribution h is I C γ (θ) =E θ [ψ C γ (X; γ) 2 ] The generalized Fisher information for γ from a class of unbiased estimating function G and based on the conditional distribution h is I C γ (θ) =E θ [(ψ C γ (X; γ) Π[ψ C γ (X; γ) G γ ) 2 ]

14 83 If ψγ C (X; γ) G γ,theniγ C (θ) =Iγ C (θ). The Fisher information for γ contained in the marginal distribution f is I M γ (θ) =E θ [(ψ M γ (X; θ) Π[ψ M γ (X; θ) Λ θ ]) 2 ] The generalized Fisher information for γ from a class of unbiased estimating function G based on the marginal distribution f is I M γ (θ) =E θ [(ψ M γ (X; θ) Π[ψ M γ (X; θ) G γ ]) 2 ]

15 84 Theorem 4.1: If ψ C γ (X; γ) G γ, I γ(θ) =I C γ Proof: (θ)+iγ M (θ). I γ(θ) = E θ [(ψ γ (X; θ) Π[ψ γ (X; θ) G γ ]) 2 ] = E θ [(ψ C γ (X; γ)+ψ M γ (X; θ) Π[ψ M γ (X; θ) G γ ]) 2 ] = I C γ (θ)+iγ M (θ)+ 2E θ [ψ C γ (X; γ)(ψ M γ (X; θ) Π[ψ M γ (X; θ) G γ ])] = Iγ C (θ)+iγ M (θ)+2e θ [ψγ C (X; γ)ψγ M (X; θ)] = I C γ (θ)+iγ M (θ)

16 85 Theorem 4.2: I γ (θ) =I C γ (θ)+i M γ (θ). Proof: Note that ψ C γ (X; γ) Λ θ. I γ (θ) = E θ [(ψ γ (X; θ) Π[ψ γ (X; θ) Λ θ ]) 2 ] = E θ [(ψγ C (X; γ)+ψγ M (X; θ) Π[ψγ M (X; θ) Λ θ ]) 2 ] = Iγ C (θ)+iγ M (θ)+ 2E θ [ψγ C (X; γ)(ψγ M (X; θ) Π[ψγ M (X; θ) Λ θ ])] = Iγ C (θ)+iγ M (θ)+2e θ [ψγ C (X; γ)ψγ M (X; θ)] = Iγ C (θ)+iγ M (θ) The information decompositions clearly spell out how the information about γ is partitioned between the conditional and marginal parts of the factorization.

17 86 Section 4.4 Optimality of the conditional score under ancillarities What is the efficiency of ψ C γ (X; γ)? Assume that ψ C γ (X; γ) G γ. Eff θ [ψ C γ (X; γ)] E θ[ ψ C γ (X; γ)/ γ] 2 E θ [ψ C γ (X; γ) 2 ] = {E θ[ 2 log h(x T (X); γ)/ γ 2 ]} 2 E θ [( log h(x T (X); γ)/ γ) 2 ] = E θ [ψγ C (X; γ) 2 ]=Iγ C (θ) =Iγ C (θ) Now, we say an UEF g 0 is optimal for γ if Eff θ [g 0 ]=Iγ(θ). The conditional score function will be optimal if Iγ M (θ) = 0. We will now give conditions under which this latter condition holds.

18 87 Lemma 4.3: If T (X) is S-ancillary, then the conditional score function is optimal. Proof: I M γ (θ) =E θ [(ψ M γ (X; θ) Π[ψ M γ (X; θ) G γ ]) 2 ]=0 since ψ M γ (X; θ) =0

19 88 Lemma 4.4: If T (X) is R-ancillary, then the conditional score function is optimal. Proof: It is sufficient to show that Iγ M (θ) =0sinceweknowthat 0 Iγ M (θ) Iγ M (θ). Since T (X) is R-ancillary, we know that there exists a one-to-one reparameterization between θ =(γ,λ) and (γ,φ) such that f(t (x); θ) depends on θ only through φ. Thatis, f(t; θ) =f (t; φ(θ)) Under suitable regularity conditions, we know that ψ M γ (θ) = log f (T (X); φ(θ))/ φ φ(θ)/ γ ψ M λ (θ) = log f (T (X); φ(θ))/ φ φ(θ)/ λ Assuming that φ(θ)/ λ is positive definite, we know that log f (T (X); φ(θ))/ φ = ψ M λ (θ) [ φ(θ)/ λ] 1

20 89 This implies that ψ M γ (θ) = ψ M λ (θ) [ φ(θ)/ λ] 1 φ(θ)/ γ = φ(θ)/ γ [ φ(θ)/ λ] 1 ψ M λ (θ) Note that ψγ M (θ) Λ θ.thisimpliesthatπ[ψγ M (θ) Λ θ ]=ψγ M (θ). So, Iγ M (θ) =E θ [(ψγ M (θ) Π[ψγ M (θ) Λ θ ]) 2 ]=0

21 90 Lemma 4.5: If T (X) is C-ancillary, then the conditional score function is optimal. Proof: It suffices to show that ψ M γ (X; θ) G γ.sincet (X) is C-ancillary, we know that for all γ Γ, the family {f(t; γ,λ):λ Λ} is complete. That is, E θ [m(t (X); γ)] = 0 for all λ Λ P θ [m(t (X); γ) =0]=1forallλ Λ Now, for an UEF g(x; γ), E θ [ψ M γ (X; θ)g(x; γ)] = E θ [ψ M γ (X; θ)e θ [g(x; γ) T (X)]] Note that E θ [g(x; γ) T (X)] is a function of T (X) andγ with mean zero. This implies that E θ [g(x; γ) T (X)] = 0 a.e. So, E θ [ψ M γ (X; θ)g(x; γ)] = 0 which implies that ψ M γ (X; θ) G γ.

22 91 Lemma 4.6: If T (X) is A-ancillary, then it is R-ancillary. Proof: If T (X) is A-ancillary, then for any given θ 0 Θandany other γ Γ, there exists a λ = λ(γ,θ 0 ) such that f(t; θ 0 )=f(t; γ,λ(γ,θ 0 )) for all t For any γ, we know that the distribution of T (X) depends only on φ = λ(γ,θ 0 ). So, there is a transformation between θ 0 and (γ,φ) such that f(t (x); θ) depends only on φ. Thatis,T (X) is R-ancillary. Corollary 4.7: If T (X) is A-ancillary, then the conditional score function is optimal. Proof: If T (X) is A-ancillary, then it is R-ancillary. R-ancillarity implies that the conditional score is optimal.

23 92 Four Examples of Ancillarity Example 4.3: Let X =(Y 1,Y 2 ) be independent Poisson random variables with means µ 1 and µ 2.Letγ = µ 1 µ 1 +µ 2 and λ = µ 1 + µ 2. Let T (X) =Y 1 + Y 2. Show that T (X) is S-ancillary for γ. We know that T (X) P oisson(λ). The conditional distribution of X given T (X) isequalto

24 93 h(x T (x); θ) = P [Y 1 = y 1,Y 2 = y 2,T(X) =t] P [T (X) =t] = P [Y 1 = y 1,Y 2 = t y 1 ]I(y 1 t) P [Y 1 + Y 2 = t] = exp( µ 1)µ y 1 1 exp( µ 2)µ t y 1 2 I(y 1 t)/y 1!(t y 1 )! exp( µ 1 µ 2 )(µ 1 + µ 2 ) t /t! = = y1 t y1 µ1 µ2 µ 1 + µ 2 µ 1 + µ 2 t! y 1!(t y 1 )! γy 1 (1 γ) t y 1 I(y 1 t) t! y 1!(t y 1 )! I(y 1 t) So, p(x; θ) =h(x T (x);γ)f(t (x);λ) and T (X) is S-ancillary.

25 94 Example 4.4: Let X =(X 1,X 2,...,X n ) be i.i.d. Normal random variables with mean µ and variance σ 2.Letγ = σ 2 and λ = µ. Let T (X) =X. We know that X N(λ, γ/n). By exponential family results, we know that for fixed γ, T (X) is complete for λ. The conditional distribution of X given T (X) isequalto h(x T (X) =t; θ) = (2πγ) n/2 exp( P n i=1 (x i λ) 2 /(2γ))I( P n i=1 x i = nt) ( n 2πγ )1/2 exp( n(t λ) 2 /(2γ)) = (2πγ) (n 1)/2 n 1/2 exp( ( nx i=1 x 2 i nt 2 )/(2γ)) So, p(x; θ) =h(x T (x);γ)f(t (x);θ) and T (X) is C-ancillary.

26 Example 4.5: Suppose that X 1 Binomial(n 1,p 1 )and X 2 Binomial(n 2,p 2 )wheren 1 and n 2 are fixed sample sizes. We assume X 1 is independent of X 2.LetX =(X 1,X 2 ), q 1 =1 p 1, q 2 =1 p 2, γ =log( p 2/q 2 p 1 /q 1 )andλ =log(p 1 /q 1 ). There is a one-to-one mapping between (p 1,p 2 )and(γ,λ). Let T (X) =X 1 + X 2. What is the distribution of T (X)? 95

27 96 f(t; θ) = = min(t,n 2 ) X u=0 min(t,n 2 ) X u=max(0,t n 1 ) P [X 1 + X 2 = t X 2 = u]p [X 2 = u] P [X 1 = t u]p [X 2 = u] = min(t,n 2 ) X u=max(0,t n 1 ) t u 0 n 1 0 A p t u 1 q n 1 t+u n 2 u 1 A p u 2 q n 2 u 2 = min(t,n 2 ) X u=max(0,t n 1 ) t u 0 n u 1 n 0 A 1 A exp(γu + λt)q n 1 1 q n 2 2 T (X) is C-ancillary, so the conditional score function is optimal. T (X) is not S (obvious), R, A- ancillary.

28 97 To be thorough, we compute the conditional density of X given T (X). This is given by h(x T (X) =t; θ) = P [X 1 = x 1,X 2 = x 2,T(X) =t] P [T (X) =t] = = x 1 0 n x 2 1 n 0 A 0 P min(t,n2 ) 1 u=max(0,t n 1 ) t u 0 n x 1 1 A 0 n 0 P min(t,n2 ) 1 u=max(0,t n 1 ) t u 1 A exp(γx2 + λt)q n 1 1 qn 2 2 u 1 n 0 A 1 A exp(γx2 ) x 2 u 1 n 0 A 1 A exp(γu + λt)q n 1 1 qn A exp(γu)

29 98 One advantage of using the conditional score function is that we may consider situations where there is an infinite-dimensional nuisance parameter (i.e., a semiparametric model). Suppose that there are n independent tables. Let λ i be the baseline log odds for the ith table, but let γ be the common odds ratio across the n tables. Assume that λ i L. Inthiscase,the parameters are (γ,l). Let X =(X 1,...,X n ), T (X) =(T 1 (X 1 ),...,T n (X n )), T i (X i )=X 1i + X 2i, X i =(X 1i,X 2i ), and X 1i and X 2i be independent Binomial random variables with fixed sample sizes n 1i and n 2i and random success probabilities p 1i and p 2i, respectively. Let q 1i =1 p 1i and q 2i =1 p 2i. So, we know that λ i =log(p 1i /q 1i )and γ =log( p 2i/q 2i p 1i /q 1i ) for all i. The conditional distribution of X given T (X) doesn t depend on L.

30 Example 4.6: Consider the semiparametric truncation model. Let Y and T be independent non-negative random variables. Suppose that Y k(k) andt l(l). In the right truncation problem, (Y,T ) is only observed if Y T. Lagakos (Biometrika, 1988) describes a study population of subjects infected with the HIV virus from contaminated blood transfusions. The date of infection was known for all subjects and only those subjects who contracted AIDS by a fixed date were included in the analysis. Interest is in the incubation time of AIDS, i.e., time from infection to AIDS. Let Y denote the incubation time and T be the time from infection to the fixed cutoff date. Note that Y and T are only observed if Y T. 99

31 100 Let X =(Y,T) betheobserved(y,t ). Then, p(x) =P [Y = y, T = t] =P [Y = y, T = t Y T ]= k(y)l(t)i(y t) β where P [Y T ]=β = 0 K(t)l(t)dt = 0 (1 L(y))k(y)dy Let T (X) =T be the conditional statistic. Then h(x T (x) =t) =P [Y = y T = t] =P [Y = y T = t, Y T ]= k(y)i(y t) K(t) and f(t) =P [T = t] =P [T = t Y T ]= K(t)l(t) β

32 101 Suppose that we parameterize the law of Y via a parameter γ and leave the distribution of T unspecified. So, we have a semiparametric model with parameters (γ,l). So, p(x; γ,l)= k γ(y)i(y t) K γ (t) K γ (t)l(t) β γ,l where β γ,l = 0 K γ (t)l(t)dt = 0 (1 L(y))k γ (y)dy

33 102 Claim 4.8: T (X) is A-ancillary for γ. Proof: For any given (γ 0,l 0 )andanygivenγ, thereexists l = l(γ,γ 0,l 0 ) such that f(t; γ 0,l 0 )=f(t; γ,l(t; γ,γ 0,l 0 ) for all t, where f(t; γ,l)= K γ(t)l(t) β γ,l If we take where l(t; γ,γ 0,l 0 )= K γ 0 (t)l 0 (t) K γ (t)c c = 0 K γ0 (t)l 0 (t) dt K γ (t) then the above equality holds. This implies that the conditional score function is optimal.