Other Test Constructions: Likelihood Ratio & Bayes Tests Side-Note: So far we have seen a few approaches for creating tests such as Neyman-Pearson Lemma ( most powerful tests of H 0 : θ = θ 0 vs H 1 : θ = θ 1 ) Two methods for uniformly most powerful (UMP) tests Method I: Based on Neyman-Pearson Lemma (may work for R p -valued parameters θ and tests of H 0 : θ Θ 0 R p vs H 1 : θ Θ 0 ) Method II: Monotone Likelihood Ratio (may work for real-valued θ R and tests H 0 : θ θ 0 vs H 1 : θ > θ 0 or H 0 : θ θ 0 vs H 1 : θ < θ 0 ) The tests above are often rooted in comparing likelihoods to make a testing decision (e.g., Neyman- Pearson Lemma). We next consider a very general testing procedure based on comparing the ratio of two likelihoods. A.) Likelihood Ratio Tests Definition: Let f(x θ), θ Θ R p, be the joint pdf/pmf of X = (X 1,..., X n ) (the parameter θ can be vector-valued) and let Θ 0 be a nonempty proper subset of Θ. Then, the likelihood ratio statistic (LRS) for testing H 0 : θ Θ 0 R p vs H 1 : θ Θ 0 is defined as λ(x) = max f(x θ) θ Θ 0 max f(x θ). θ Θ Note that if ˆθ MLE of θ over entire Θ & θ maximum of f(x θ) over θ Θ 0 we may write λ(x) = f(x θ) f(x ˆθ) Definition: A size α likelihood ratio test (LRT) for testing H 0 : θ Θ 0 R p vs H 1 : θ Θ 0 is defined as 1 if λ(x) < k ϕ(x) = γ if λ(x) = k 0 if λ(x) > k where γ [0, 1] and 0 k 1 are constants determined by max θ Θ0 E θ ϕ(x ) = α. 1
Example: Let X 1,..., X n be iid Gamma(α = 3, θ), θ > 0. Find a size α LRT for H 0 : θ = θ 0 vs H 1 : θ θ 0. 2
Example: Let X 1,..., X n be iid Exponential(θ, ν), θ > 0, ν R with common pdf { 1 θ f(x θ, ν) = e (x ν)/θ if x ν 0 otherwise Find a size α LRT for H 0 : ν = ν 0 vs H 1 : ν ν 0 (where ν 0 R is fixed). 3
B.) Large Sample Properties of LRT Tests (for calibration) The following result describes the asymptotic distribution of the likelihood ratio statistic (under appropriate regularity conditions) & may be used to calibrate a LRT in a simple fashion when the sample size is sufficiently large. Theorem: Let X 1,... be iid random vectors with common pdf/pmf f(x θ), θ Θ R (the parameter θ can be vector-valued). Let λ n (X 1,...,X n) denote the likelihood ratio statistic based on X 1,...,X n for testing H 0 : θ Θ 0 R p vs H 1 : θ Θ 0, where Θ 0 has the form { } Θ 0 = θ = (θ 1,..., θ p ) Θ : θ 1 = θ1, 0..., θ r = θ }{{} r 0 for some θ1, 0..., θr 0 and r p hypothesized values for first r p parameters Then, under the Cramér-Rao type regularity conditions, it holds that: if H 0 is true, 2 log λ n (X 1,...,X d n) χ 2 r as n. Remark: The above limiting distribution suggests the following testing procedure based on the (1 α)-quantile of a χ 2 r distribution, denoted as χ 2 1 α(r) for which P ( χ 2 r χ 2 1 α(r) ) = 1 α, P ( χ 2 r > χ 2 1 α(r) ) = α. Namely, for n large (e.g., say n 30 observations), ϕ(x 1,...,X 1 if 2 log λ n (X 1 n) =,...,X n) > χ 2 1 α(r) 0 otherwise is an approximate size α LRT for testing H 0 : θ 1 = θ 0 1,..., θ r = θ 0 r vs H 1 : θ i θ 0 i for some 1 i r. 4
Example: Let X 1,... be iid N 2(µ, A) random vectors, where µ = (µ 1, µ 2 ) R 2 and A is a known 2 2 positive definite matrix. Find a size α LRT for testing H 0 : 2µ 1 + 3µ 2 = 0 vs H 1 : 2µ 1 + 3µ 2 0. 5
C.) Bayes Tests Let X 1,..., X n have joint pdf/pmf f(x θ), θ Θ R p, and we want to test H 0 : θ Θ 0 R p vs H 1 : θ Θ 0. Let π(θ) be a prior pdf P (θ Θ 0 x) = Θ 0 f θ x(θ)dθ posterior probability that θ Θ 0 P (θ Θ 0 x) = Θ\Θ 0 f θ x(θ)dθ posterior probability that θ Θ 0 Note that P (θ Θ 0 x) + P (θ Θ 0 x) = 1 Then, a Bayes test for testing H 0 : θ Θ 0 vs H 1 : θ Θ 0 is given by 1 if P (θ Θ 0 x) P (θ Θ 0 x) 1 if P (θ Θ 0 x) 1/2 ϕ(x) = = 0 otherwise 0 otherwise Discussion: The Bayes test follows from minimizing the Bayes Risk BR ϕ1 simple test ϕ 1 (x) (i.e., tests where ϕ 1 (x) {0, 1} for any x) of a Consider a loss function L(θ, a) = I {θ Θ0 }I {a=1} + I {θ Θ0 }I {a=0} where a may assume two values: a = 1 means reject H 0 and a = 0 means don t reject H 0. So, the loss L(θ, a) = 0 for a correct decision and L(θ, a) = 1 for an incorrect decision: L(θ, a) = { 0 if θ Θ 0 & a = 0 or if θ Θ 0 & a = 1 1 otherwise The risk function of a simple test ϕ 1 (x) is R ϕ1 (θ) = E θ L(θ, ϕ 1 )) = I {θ Θ0 } P θ (ϕ 1 ) = 1) (X } (X {{} prob. of Type I error +I {θ Θ0 } P θ (ϕ 1 (X ) = 0) }{{} prob. of Type II error The Bayes risk of ϕ 1 (x) w.r.t. π(θ) is: BR ϕ1 = E (θ) R ϕ1 (θ) = Θ R ϕ 1 (θ)π(θ)dθ & the Bayes test ϕ(x) minimizes BR ϕ1 over all simple tests ϕ 1 (x) 6
Alternatively, we can find the Bayes test by minimizing the posterior risk of a simple test ϕ 1 (x) for each fixed x, where the posterior risk is ( ) E θ xl(ϕ 1 (x), θ) = I {θ Θ0 }I {ϕ1 (x)=1} + I {θ Θ0 }I {ϕ1 (x)=0} f θ x(θ)dθ Θ = I {ϕ1 (x)=1} f θ x(θ)dθ + I {ϕ1 (x)=0} f θ x(θ)dθ Θ 0 Θ\Θ 0 = I {ϕ1 (x)=1}p (θ Θ 0 x) + I {ϕ1 (x)=0}p (θ Θ 0 x) For each fixed x, we choose the values ϕ 1 (x) = 1 or 0 of the test to minimize the posterior risk; that is, for each fixed x, we should pick ϕ 1 (x) = 1 if P (θ Θ 0 x) P (θ Θ 0 x) and pick ϕ 1 (x) = 0 if P (θ Θ 0 x) < P (θ Θ 0 x). Note this is the same decision rule as the Bayes test ϕ(x) above. Example: Let X 1,..., X n be iid N(θ, 1), θ R. Find the Bayes test for H 0 : θ θ 0 vs H 1 : θ > θ 0 under the N(µ, τ 2 ) prior for θ, where µ, τ 2, θ 0 are fixed. 7