STAT200C: Hypothesis Testing

STAT200C: Hypothesis Testing Zhaoxia Yu Spring 2017

Some Definitions A hypothesis is a statement about a population parameter. The two complementary hypotheses in a hypothesis testing are the null hypothesis (H 0 : θ Ω 0 ) and the alterantive hypothesis (H 1 : θ Ω 1 ), where Ω 0 and Ω 1 are two disjoint subsets of the full parameter space Ω. A hypothesis is called a simple hypothesis if the space is a singleton set; else it is called a composite hypothesis. 1

Some Examples simple vs simple: H 0 ; θ = θ 0 vs H 1 : θ = θ 1 ; simple vs composite: H 0 : θ = θ 0 vs H 1 : θ θ 0 ; composite vs composite: H 0 : θ θ 0 vs H 1 : θ > θ 0 ; composite vs composite H 0 : θ θ 0 vs H 1 : θ < θ 0. 2

Critical Function A test is determined by its critical function: φ(x) = the probability of rejecting H 0 when x is the observed value of X. Example: Suppose X Binomial(5, θ), H 0 : θ = 0.5, H 1 : θ > 0.5. Consider two critical functions: φ 1 (x) = { 1 x = 5 0 x < 5, φ 2(x) = 1 x = 5 0.12 x = 4 0 x < 4 φ 1 (x) is a non-randomized test; φ 2 (x) is randomized test. A randomized test can ensure a pre-fixed type I error rate; in practice, it is not used often. 3

Errors in Hypothesis Testing Type I error: H 0 is rejected when it is true. Type I error rate: α = P r(reject H 0 H 0 is true) Type II error: H 0 is failed to reject when it is not true. Type II error rate: 1 β = P r(fail to reject H 0 H 1 is true) 4

Type I Error Type I error: H 0 is rejected when it is true. Type I error rate: α = P r(reject H 0 H 0 is true) α is called the significance level of a test. 5

Type I Error: An Example Consider the binomial example. Suppose θ = 0.5. For φ 1 (x), α = 1.0 P r(x = 5 θ = 0.5) = 0.5 5 = 0.03125. For φ 2 (x), α = 1.0 P r(x = 5 θ = 0.5) + 0.12 P (X = 4 θ = 0.5) = 0.03125 + 0.12 5 0.5 4 0.5 = 0.05 6

Type II Error Type II error: H 0 is failed to reject when it is not true. Type II error rate: 1 β = P r(fail to reject H 0 H 1 is true) The power of a function β = 1 P r(fail to reject H 0 H 1 is true) 7

Type II Error: An example Example. Consider the binomial example. Suppose θ = 0.6. β 1 = 1.0 P r(x = 5 θ = 0.6) = 0.6 5 = 0.07776, β 2 = 1.0 P r(x = 5 θ = 0.6) + 0.12 P r(x = 4 θ = 0.6) = 0.108864 Suppose θ = 0.7. β 1 = 0.16807, β 2 = 0.211288 Suppose θ = 0.8. β 1 = 0.32768, β 2 = 0.376832 Suppose θ = 0.9. β 1 = 0.59049, β 2 = 0.629856 8

Power Function The power function of test φ: β φ (θ) = E θ [φ(x)]. Let R be the rejection region, then we can also define the power function using: β(θ) = P r(x R θ) It is a measure of weakness (type I error) when θ Ω 0 strength (power) when θ Ω 1 A test is said to have a size α if sup θ Θ0 β(θ) = α. A test is said to have a level α if sup θ Θ0 β(θ) α. 9

Example Let X = (X 1,, X n ) be a random sample from N(θ, σ 2 ) with σ 2 known. H 0 : θ = θ 0 vs H 1 : θ > θ 0. Intuitively, we should reject H 0 for large values of X θ 0. We can write the rejection region as The power function is φ(x) = 1 x µ 0 σ/ n > c 0 x µ 0 σ/ n < c β(θ) = P µ ( X θ 0 σ/ n > c) = P θ( X θ σ/ n > c + θ 0 µ σ/ n ) = P θ (Z > c + θ 0 θ σ/ n ) Here β(θ) is an increasing function of θ. Homework: Write an R function to choose the sample size n such that α = β(θ 0 ) = 0.05 and β(θ 0 + σ) > 0.8 for θ > θ 0 + σ. 10

Power Function: An Example Again, consider the binomial example. β 1 (0.1) = 1e 05, β 2 (0.1) = 6.4e 05 β 1 (0.3) = 0.00243, β 2 (0.3) = 0.005832 β 1 (0.5) = 0.03125, β 2 (0.5) = 0.05 β 1 (0.7) = 0.16807, β 2 (0.7) = 0.211288 β 1 (0.9) = 0.59049, β 2 (0.9) = 0.629856 11

An Example of Power Calculation Let s look at the power function for H 0 : µ = µ 0 vs H 1 : µ > µ 0 where µ is the mean of a N(µ, σ 2 ) with known variance. The likelihood ratio test rejects H 0 when x = X µ 0 σ/ n > Φ 1 (1 α). The power function is β(µ) = P ( X µ + µ µ 0 σ/ n > Φ 1 (1 α)) = P (Z > Φ 1 (1 α) n µ µ 0 σ) = 1 Φ(Φ 1 (1 α) n µ µ 0 ) σ Suppose α, µ, and µ 0 are given, for fixed n we can calculate power and for fixed power we can calculate sample size n. 12

An Example of Power Calculation (continued) Let β = 0.8, α = 0.05, µ µ 0 σ = 0.1, then Φ(Φ 1 (0.95) 0.1 n) = 0.2 0.1 n = Φ 1 (0.95) Φ 1 (0.2) n = 10 (1.6449 + 0.84)16 n 619 Question. What is the power of the test when n = 400? β = P ( X µ 0 σ/ n > Φ 1 (0.95)) = P ( X µ σ/ n > Φ 1 (0.95) n (µ µ 0) σ/ n ) = 1 Φ(1.6449 2) 0.64 13

Type I error rate vs Type II error rate / Power Consider the two tests for the binomial example X Binomial(5, 0.5) φ 1 (x) = { 1 x = 5 0 x < 5, φ 3(x) = { 1 x > 0 0 x = 0 Power functions: β 1 (θ) = P r(x = 5) = θ 5, β 3 (θ) = 1 P r(x = 0) = 1 (1 θ) 5 It can be shown that β 1 (θ) β 3 (θ). 14

Type I error rate vs Type II error rate / Power Test 3 achieves a smaller Type II error rate, but it has higher Type I error rate How should one choose a test? One idea is to find the most powerful test among those with a Type I error rate no greater than a pre-specified value, say α. 15

Most Powerful Test: Simple vs Simple A test φ is said to be most power (MP) at level α for testing H 0 : θ = θ 0 vs H 1 : θ = θ 1 if E 0 [φ(x)] α and E 1 [φ(x)] E 1 [ψ] for any ψ so that E 0 [ψ(x)] α. In other words, we say φ is MP if it has the largest power among all tests with a level of α. How to find a MP test? Neyman-Pearson Lemma can be used to construct a most powerful level α test for simple vs simple hypotheses. 16

Neyman-Pearson Lemma Neyman-Pearson Lemma (Most powerful level α test for H 0 : θ = θ 0 vs H 1 : θ = θ 1 ) Existence. There exists a test φ and a constant k 0 such that and φ(x) = E 0 [φ(x)] = α { 1 if f1 (x) > kf 0 (x) 0 if f 1 (x) < kf 0 (x) where f j (x) = f(x θ j ). [Note that nothing is being said about φ on the set {x : f 1 (x) = kf 0 (x)}.] 17

Sufficiency If a test φ satisfies the conditions in Existence for some k 0, then it is MP at level α for testing H 0 : θ = θ 0 vs H 1 : θ = θ 1. Necessity If φ is MP at level α for testing H 0 : θ = θ 0 vs H 1 : θ = θ 1, then with probability 1 under both θ 0 and θ 1, for some k. φ(x) = { 1 if f1 (x) > kf 0 (x) 0 if f 1 (x) < kf 0 (x) For α = 0 or 1, the proof is straightforward, provided that k = 0 and are allowed. Now let 0 < α < 1. 18

Proof: Existence Let F (c) = P 0 [f 1 (X) cf 0 (X)] = P 0 [f 1 (X)/f 0 (X) c]. Then F is the cdf of T (X) = f 1 (X)/f 0 (X) under θ 0. [Note, under θ 0, f 0 (X) > 0 with probability 1]. Since F is cdf, F, right-continuous: F (c) F (c 0) = P 0 (T (X) = c). Now let α(c) = 1 F (c) = P 0 (T (X) > c). Then α( ) = 1, α( ) = 0, α, and right continuous: α(c 0) α(c) = P 0 [T (X) = c]. Let c 0 such that α(c 0 ) α α(c 0 0). 19

Proof: Existence (continued) Define φ(x) = 1 if T (x) > k [eqt f 1 (x) > kf 0 (x)] α α(c 0 ) α(c 0 0) α(c 0 ) if T (x) = k [eqt f 1 (x) = kf 0 (x)] 0 if T (x) < k [eqt f 1 (x) < kf 0 (x)] [If F is continuous at c 0, defined 0/0 = 1] The existence holds with k = c 0, because E 0 [φ(x)] = P 0 [T (X) > c 0 ] + α α(c 0 ) α(c 0 0) α(c 0 ) P 0[T (X) = c 0 ] = α(c 0 ) + α α(c 0 ) = α 20

Proof: Sufficiency Let φ be a test satisfying the conditions in Existence and let φ be an arbitrary level α test i.e., E 0 [φ (X)] α. We shall show that β φ (θ 1 ) = E 1 [φ(x)] E 1 [φ (X)] = β φ (θ 1) Let S + = {x : φ(x) φ (x) > 0} and S = {x : φ(x) φ (x) < 0}. Then for x S +, φ(x) > φ (x) 0 f 1 (x) kf 0 (x) [ eqt T (x) k] and for x S, φ(x) < φ (x) 1 f 1 (x) kf 0 (x) [ eqt T (x) k] 21

Proof: Sufficiency (continued) Thus { [φ(x) φ 0 x S (x)][f 1 (x) kf 0 (x)] + S = 0 x S + S Hence E 1 [φ(x)] E 1 [φ (X)] = = + k [φ(x) φ (x)]f 1 (x)dx [φ(x) φ (x)][f 1 (x) kf 0 (x)]dx [φ(x) φ (x)]f 0 (x)dx 0 + k{e 0 [φ(x) E 0 [φ (x)]]} 0 22

Proof: Necessity Suppose that φ is MP at level α. Let φ be a test satisfies the conditions in Existence. Let S + and S be as above. The set on which φ violates the conditions is S = (S + S ) {x : f 1 (x) kf 0 (x)} Clearly S = {x : φ(x) φ (x), f 1 (x) kf 0 (x)}. We shall show that P 0 (S) = P 1 (S) = 0 Suppose the distribution is continuous. In this case, it is enough to show that S dx = 0 23

Proof: Necessity (continued) Note that [φ(x) φ (x)][f 1 (x) kf 0 (x)] > 0 for x S. dx > 0 implies that S [φ(x) φ (x)][f 1 (x) kf 0 (x)]dx = i.e., [φ(x) φ (x)]f 1 (x)dx > k Hence S [φ(x) φ (x)][f 1 (x) kf 0 (x)] > 0, [φ(x) φ (x)]f 0 (x) > k(α E 0 [φ (X)]) > 0 The last inequality is true because E 0 [φ (X)] α. Thus, E 1 [φ(x)] > E 1 [φ (X)], contradicting the MP level α property of φ. Note: the proof of discrete distributions is similar - just replace with. 24

A Corollary Corollary. Let β = E 1 [φ(x)] where φ is a MP test at level α (0, 1) for H 0 : θ = θ 0 vs H 1 : θ = θ 1. Then β > α. Proof. For let φ 0 (x) α, the test that always reject H 0 with probability α, irregardless the value of x. It is obvious that E 0 [φ 0 (X)] = E 1 [φ 0 (X)] = α i.e., both the Type I error rate and power of φ 0 (x) is α. Since φ 0 (x) is level α test but it does not satisfy the Neyman- Pearson conditions, it is not a MP test (by the necessity), whereas φ(x) is. Therefore, β = E 1 [φ(x)] > E 1 [φ 0 (X)] = α 25

Remark 1: φ(x) = On {x : f 1 (x) = c 0 f 0 (x)}, α α(c 0 ) α(c 0 0) α(c 0 ) = α 1 + F (c 0 ) 1 F (c 0 0) 1 + F (c 0 ) = α 1 + F (c 0) F (c 0 ) F (c 0 ) where F is the cdf of T (X) = f 1 (X)/f 0 (X). θ 0 is discontinuous at c 0, then Suppose F under (1) F (c 0 0) < 1 α = F (c 0 ) φ(x) = 0 on {x : f 1 (x) = c 0 f 0 (x)} (2) F (c 0 0) = 1 α < F (c 0 ) φ(x) = 1 on {x : f 1 (x) = c 0 f 0 (x)} (3) F (c 0 0) < 1 α < F (c 0 ) φ(x) = γ(x) [0, 1] on {x : f 1 (x) = c 0 f 0 (x)} for any γ(x) satisfying {x:f 1 (x)=c 0 f 0 (x)} γ(x)f 0(x)dx = α α(c 0 ) is MP at level α for H 0 : θ = θ 0 vs H 1 : θ = θ 1. Thus we have a unique MP test at level α in case (1) or (2), but not in case (3). 26

Remark 2: A MP level α test for H 0 : θ = θ 0 vs H 1 : θ = θ 1 must satisfy E 0 [φ(x)] = α, unless there exists a test ψ having size E 0 [ψ(x)] < α and power E 1 [ψ(x)] = 1. This exceptional situation occurs in the following case: Let X 1,, X n be a random sample from Unif[0, θ] for θ > 0. We want to test H 0 : θ = θ 0 vs θ = θ 1 < θ 0 at level α. Let X (n) be the largest value in the sample and suppose that (θ 1 /θ 0 ) n < α. Then the following test is MP { 1 if 0 x(n) θ 1 ψ(x) = 0 if θ 1 < x (n) θ 0 and has size < α and power=1. 27

Remark 3: (omitted) 28

Remark 4: The Neyman-Pearson lemma expresses the MP level α test φ for H 0 : θ = θ 0 vs θ = θ 1 in terms of the likelihood ratio, or equivalently log likelihood ratio (LLR). If X = (X 1,, X n ) is a random sample from f θ (x), then the LLR is log[f 1 (x)/f 0 (x)] = n i=1 and a MP level α test for H 0 vs H 1 is φ(x) = log[f 1 (x i )/f 0 (x i )], 1 if n i=1 log[f 1 (x i )/f 0 (x i )] > k γ if n i=1 log[f 1 (x i )/f 0 (x i )] = k 0 if n i=1 log[f 1 (x i )/f 0 (x i )] < k where k and γ are determined by the condition E 0 [φ(x)] = α. 29

Example: A Random Sample from Normal Let X = (X 1,, X n ) be a random sample from N(θ, σ 2 ), where σ 2 is known but θ is unknown. Consider the following three hypothesis testing problems: 1. H 0 : θ = θ 0 vs H 1 : θ = θ 1 2. H 0 : θ θ 0 vs H 1 : θ = θ 1, θ 1 > θ 0 3. H 0 : θ θ 0 vs H 1 : θ > θ 0 30

A Random Sample from Normal, Problem 1: H 0 : θ = θ 0 vs H 1 : θ = θ 1 Let f j be the joint pdf of X corresponding to θ j. The LLR is log[f 1 (x)/f 0 (x)] = 1 2σ 2[ (x i θ 0 ) 2 (x i θ 1 ) 2 ] = n 2σ 2(θ2 0 θ2 1 ) + n σ 2(θ 1 θ 0 ) x The MP level α test for H0 vs H 1 is: { φ 1 if x 1 (x) = c1 if θ 0 if x < c 1 > θ 0, φ 2 (x) = 1 { 1 if x c2 0 if x < c 2 if θ 1 < θ 0 where c 1 = θ 0 + σ/ nφ 1 (1 α) and c 2 = θ 0 σ/ nφ 1 (1 α). 31

A Random Sample from Normal, Problem 2: H 0 : θ θ 0 vs H 1 : θ = θ 1 > θ 0 Suppose φ(x) is MP level α for Problem 2. Then it must satisfy (i) sup θ θ0 E θ [φ(x)] α and (ii) E θ1 [φ(x)] E θ1 [ψ(x)] ψ of level α The power function of φ obtained in Problem 1 is: β φ (θ) = E 1 θ [φ 1 (x)] = P θ[ x θ 0 + σ/ nφ 1 (1 α)] = Φ(Φ 1 n(θ θ0 ) (α) + ) σ which is in θ, so that sup θ θ0 E θ [φ 1 (x)] = E θ 0 [φ 1 (x)] = α. 32

A Random Sample from Normal, Problem 2: H 0 : θ θ 0 vs H 1 : θ = θ 1 > θ 0 (continued) Thus φ 1 is level α. Now if ψ satisfies (i), then i.e., ψ is level α for H 0 vs H 1. α sup θ θ0 E θ [ψ(x)] E θ0 [ψ(x)] Because φ 1 (x) is a MP for H 0 : θ = θ 0 vs H 1 : θ = θ 1 (> θ 0 ), E θ1 [φ (x)] E θ1 [ψ(x)] Thus, φ is a MP level α for H 0 : θ θ 0 vs H 1 : θ = θ 1. 33

A Random Sample from Normal, Problem 3: H 0 : θ θ 0 vs H 1 : θ > θ 0 For this problem, we want (if possible) a uniformly most power (UMP) level α test for H 0 vs H 1. In other words, we want a test φ such that (i) sup θ θ0 E θ [φ(x)] α and (ii) E θ [φ(x)] E θ [ψ(x)] θ > θ 0 whereas ψ satisfies (i). Since φ 1 obtained above is the same for θ 1 > θ 0, it is UMP at level α for H 0 vs H 1. 34

Uniformly Most Powerful (UMP) A test is a uniformly most powerful (UMP) test at level α for H 0 : θ Ω 0 vs H 1 : θ Ω 1 if (i) sup θ Ω0 E θ [φ(x)] α and (ii) E θ [φ(x)] E θ [ψ(x)] θ Ω 1 whereas ψ satisfies (i). Although requirement (ii) is very stringent, UMP tests do exist in a certain type of situations. We have seen such situation in the normal example. 35

Monotone Likelihood Ratio (MLR) Family A family of pdfs or pmfs for a univariate random variable X is said to be a monotone likelihood ratio (MLR) family if there exists a real-valued T (x) such that for any θ 1 < θ 2 in Ω, f θ2 (x)/f θ1 (x) is a nondecreasing function of T (x). If f θ1 (x) = 0 and f θ2 (x) > 0, define f θ2 (x)/f θ1 (x) = + 36

Some Examples of MLR Suppose X Binomial(n, θ). Then f(x θ 2 )/f(x θ 1 ) = (θ 2 /θ 1 ) x ( 1 θ 2 ) n x = ( θ 2(1 θ 1 ) 1 θ 1 θ 1 (1 θ 2 ) )x ( 1 θ 2 ) n 1 θ 1 Thus, the family is MLR. Any regular exponential family with f(x θ) = h(x)g(θ)exp{η(θ)t (x)} has an MLR. N(θ, σ 2 ) with known σ 2. N(µ, θ = σ 2 ) with known µ and θ > 0. P oisson(θ) for θ > 0 37

Some Examples of MLR The hypergeometric distribution H(N, n, θ) with p θ (x) = ( )( ) θ N θ x n x ( ) N, n where x = max(0, θ+n N),, min(n, θ). [θ white and N θ black balls in a box from which n balls are drawn at random, P θ (x) is the probability of x white balls in the sample.] Unif(0, θ). [We define a/0 = for a > 0.] The Cauchy distributions C(θ, 1) with P θ (x) = 1/[π(1 + (x θ) 2 )], x R, θ R is NOT a MLR family. 38

Theorem of UMP Suppose that the family of pdfs/pmfs P θ, θ R has a MLR in T (X). Then 1. There exists a UMP level α test for H 0 : θ θ 0 vs H 1 : θ > θ 0 given by φ(x) = 1 if T (x) > c γ if T (x) = c 0 if T (x) < c where c and 0 γ 1 are determined by E θ0 [φ(x)] = α 2. The power function β(θ) = E θ [φ(x)] of this φ is strictly increasing at all θ for which β(θ) < 1. 39

Proof First consider H 0 : θ = θ 0 vs H 1 : θ = θ 1 where θ 1 > θ 0 is fixed. Then a MP level α test for H 0 vs H 1 rejects H 0 for large values of f θ1 (x)/f θ0 (x), i.e., for large values of T (x) (by the MLR property). Moreover, by the existence part of the N-P lemma, there exist c and 0 γ 1 such that the test φ(x) = I (c, ) (T (x)) + γi {c} (T (x)) satisfies E θ0 [φ(x)] = α. Since the forms: { 1 T (x) > c φ(x) = 0 T (x) < c and φ(x) = { 1 fθ (x) > kf θ (x) 0 f θ (x) < kf θ (x) are equivalent for any θ < θ, this test is MP at level α = β(θ ) for testing H0 : θ = θ vs H1 : θ = θ whenever θ < θ (by the sufficiency part of the N-P lemma). 40

Proof (continued) Next note that by the Corollary to the N-P lemma, β(θ ) > α = β(θ if α < 1, which proves that β(θ) is strictly increasing, so long as it is < 1. Now note that for this test, β(θ) = E θ [φ(x)] α θ θ 0, which makes φ a level α test for H 0 : θ θ 0. Let and Ψ α = {all tests ψ such that sup θ θ0 E θ [ψ(x)] α} Ψ α = {all tests ψ such that E θ0 [ψ(x)] α} Then Ψ α Ψ α. We have shown that E θ1 [φ(x)] E θ1 [ψ(x)] ψ Ψ α. Hence E θ 1 [φ(x)] E θ1 [ψ(x)] ψ Ψ α (by the N-P Lemma). This makes φ a MP test at level α for H 0 : θ θ 0 vs H1 : θ = θ 1 > θ 0. 41

Proof (continued) Finally, since φ is independent of θ 1 > θ 0, it is a UMP test at level α for H 0 : θ θ 0 vs H 1 : θ > θ 0. Note, the theory for H 0 : θ θ 0 vs H 1 : θ < θ can be stated and proved analogously. 42

Example: Uniform Distribution Let X = (X 1,, X n ) be a random sample from Unif(0, θ). (i) Find a MP test for H 0 : θ = θ 0 vs H 1 : θ = θ 1, θ 1 > θ 0. (ii) Show that any level α test with the rejection region {X (n) > θ 0 } is a UMP level α test for H 0 : θ θ 0 vs H 1 : θ > θ 0. Solution :(i) The likelihood ratio is λ(x) = f ( ) θ 1 (x) n θ0 f θ0 (x) = I(x (n) θ 1 ) I(x (n) θ 0 ) = θ 1 { x (n) > θ 0 (θ 0 /θ 1 ) n 0 < x (n) θ 0 43

Solution (continued) By the Neyman-Pearson Lemma, a UMP level α satisfies φ(x) = which is equivalent to 1 λ(x) > (θ 0 /θ 1 ) n γ λ(x) = (θ 0 /θ 1 ) n 0 λ(x) < (θ 0 /θ 1 ) n this cannot happen φ(x) = { 1 x(n) > θ 0 γ 0 < x (n) θ 0 Because it is a level α test, γ satisfies α = E θ0 [φ(x)] = P θ0 [X (n) > θ 0 ] = γ. Note, because the ratio λ(x) is nondecreasing in X (n), the test is also a UMP for H 0 : θ θ 0 vs H 1 : θ > θ 0. 44

Solution (continued) Now, let s determine the rejection region using the theorem to construct UMP for MLR family. The fact that the family is a MLR family in X (n) implies that a UMP level α test for H 0 : θ < θ 0 vs H 1 : θ > θ 0 is: { 1 x(n) > k φ 2 (x) = 0 0 < x (n) < k where k satisfies α = P θ0 (X (n) > k) = (1 k/θ 0 ) n = 1 (k/θ 0 ) n, which gives k = θ 0 (1 α) 1/n. Since both φ and φ 2 are UMP level α test, UMP is not unique in this situation. This happens because the likelihood ratio λ(x) is constant on [0, θ 0 ]. In fact we can show that they have the same power function, i.e., for any θ > θ 0, β(θ) = β 1 (θ) = 1 (1 α) ( ) θ0 n θ 45

UMP Does Not Always Exist In many situations, because the class of level α tests is so large, no one dominates all the others in power. Example. Let (X 1,, X n ) be a random sample from N(θ, σ 2 ) with σ 2 known, H 0 : θ = θ 0 and H 1 : θ θ 0. Here the tests { 1 x > θ0 + σ φ 1 (x) = n Φ 1 { (1 α) 1 x < θ0 σ and φ 1 (x) = n Φ 0 otherwise 0 other are the UMP level α tests for H 0 : θ = θ 0 vs K 1 : θ > θ 0 and K 2 : θ < θ 0 respectively. 46

Consider an alternative parameter θ 1. E θ1 [φ 1 (x)] < α θ 1 < θ 0 and E θ1 [φ 2 (x)] < α θ 1 > θ 0. φ 1 is UMP level α when θ 1 > θ 0 ; φ 2 is UMP level α when θ 1 < θ 0. φ 1 is more powerful than φ 2 when θ 1 > θ 0, but φ 2 is more powerful when θ 1 < θ 0. As a result, a UMP test at level α for H : θ = θ 0 vs H 1 : θ θ 0 does not exist. Draw the power functions here (p394 of Casella and Berger) 47

Unbiased Test In the above example, UMP does not exist. To find/define a goog test in the situation, we then narrow the class of level α test to the class of unbiased level α tests. Definition. A test φ is said to be an unbiased test at level α for H 0 : θ Ω 0 vs H 1 : θ Ω 1 if (i) E θ [φ(x)] α θ Ω 0 (ii) E θ [φ(x)] α θ Ω 1 An unbiased test rejects the null hypothesis H 0 with at least as much probability when it is false as when it is true. 48

Uniformly Most Powerful Unbiased (UMPU) Test A test φ is said to be a uniformly most powerful unbiased (UMPU) test at level α for H 0 : θ Ω 0 vs H 1 : θ Ω 1, if (i) φ is an unbiased test at level α for H 0 vs H 1 (ii) if E θ [φ(x)] E θ [ψ(x)] θ Ω 1, where ψ is also an unbiased level α test for H 0 vs H 1. UMPU tests usually exist for several types of hypothesis tests for a natural exponential family. 49

UMPU (continued) For testing H : θ = θ 0 vs H 1 : θ θ 0 at level α, an unbiased test must satisfy E θ0 [φ(x)] α and E θ [φ(x)] α θ θ 0. Neither φ 1 nor φ 2 in the above example is an unbiased test. If we restrict to the class of unbiased level α tests, then tests such as φ 1 and φ 2 would not quality, and in the restricted class, a UMP test does exist. The following test is an UMPU: { 1 x θ0 > σ φ 3 (x) = n Φ 1 (1 α/2) 0 otherwise 50

Note. φ 3 (x) = I [0, ) ( x θ 0 > σ n Φ 1 (1 α/2)) In practice, MP, UMP, or UMPU test often does not exist. We often seek for intuitively reasonable solutions. solutions frequently coincide with optimal tests. These 51