Last Lecture Biostatistics 602 - Statistical Iferece Lecture 19 Likelihood Ratio Test Hyu Mi Kag March 26th, 2013 Describe the followig cocepts i your ow words Hypothesis Null Hypothesis Alterative Hypothesis Hypothesis Testig Procedure Rejectio Regio Type I error Type II error Power fuctio Size α test Level α test Likleihood Ratio Test Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 1 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 1 Example of Hypothesis Testig Power fuctio Let X 1,, X be chages i blood pressure after a treatmet The rejectio regio Truth { x : H 0 : θ 0 H 1 : θ 0 } x s X / > 3 Decisio Accept H 0 Reject H 0 H 0 Correct Decisio Type I error H 1 Type II error Correct Decisio Defiitio - The power fuctio The power fuctio of a hypothesis test with rejectio regio R is the fuctio of θ defied by βθ X R θ reject H 0 θ If θ Ω c 0 alterative is true, the probability of rejectig H 0 is called the power of test for this particular value of θ Probability of type I error βθ if θ Ω 0 Probability of type II error 1 βθ if θ Ω c 0 A ideal test should have power fuctio satisfyig βθ 0 for all θ Ω 0, βθ 1 for all θ Ω c 0, which is typically ot possible i practice Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 3 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 1
Sizes ad Levels of Tests Size α test A test with power fuctio βθ is a size α test if sup βθ α θ Ω 0 I other words, the maximum probability of makig a type I error is α Level α test A test with power fuctio βθ is a level α test if sup βθ α θ Ω 0 I other words, the maximum probability of makig a type I error is equal or less tha α Ay size α test is also a level α test Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 1 Likelihood Ratio Tests LRT Defiitio Let Lθ x be the likelihood fuctio of θ The likelihood ratio test statistic for testig H 0 : θ Ω 0 vs H 1 : θ Ω c 0 is λx sup θ Ω 0 Lθ x sup θ Ω Lθ x Lˆθ 0 x Lˆθ x where ˆθ is the MLE of θ over θ Ω, ad ˆθ 0 is the MLE of θ over θ Ω 0 restricted MLE The likelihood ratio test is a test that rejects H 0 if ad oly if λx c where 0 c 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 6 / 1 Example of LRT Problem iid Cosider X 1,, X N θ, σ 2 where σ 2 is kow H 0 : θ θ 0 H 1 : θ > θ 0 For the LRT test ad its power fuctio Solutio 1 Lθ x exp x i θ 2 ] 2πσ 2 1 2πσ 2 exp x i θ 2 ] We eed to fid MLE of θ over Ω, ad Ω 0, θ 0 ] Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 1 MLE of θ over Ω, To maximize Lθ x, we eed to maximize exp x i θ ], 2 or equivaletly to miimize x i θ 2 x i θ 2 x 2 i + θ 2 2θx i θ 2 2θ The equatio above miimizes whe θ ˆθ x i + x 2 i x i x Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 8 / 1
MLE of θ over Ω 0, θ 0 ] Likelihood ratio test Lθ x is maximized at θ x i x if x θ 0 However, if x θ 0, x does ot fall ito a valid rage of ˆθ 0, ad θ θ 0, the likelihood fuctio will be a icreasig fuctio Therefore ˆθ 0 θ 0 To summarize, λx Lˆθ 0 x Lˆθ x 1 if X θ 0 exp exp x i θ 0 2 ] x i x 2 ] if X > θ 0 2σ { 2 1 if X θ0 exp ] x θ 0 2 if X > θ 0 ˆθ 0 { X if X θ0 θ 0 if X > θ 0 Therefore, the likelihood test rejects the ull hypothesis if ad oly if exp x θ 0 2 ] c ad x θ 0 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 9 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 10 / 1 Specifyig c Specifyig c cot d exp x θ 0 2 ] x θ 0 2 log c x θ 0 2 2σ2 log c x θ 0 2σ2 log c c x > θ 0 So, LRT rejects H 0 if ad oly if x θ 0 x θ 0 σ/ Therefore, the rejectio regio is { x : x θ } 0 σ/ c 2σ2 log c 2σ2 log c σ/ c Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 11 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 12 / 1
Power fuctio X θ0 βθ reject H 0 X θ + θ θ0 σ/ X θ σ/ θ 0 θ Sice X 1,, X iid N θ, σ 2, X N where Z N 0, 1 σ/ + c θ, σ2 σ/ c c Therefore, X θ σ/ N 0, 1 βθ Z θ 0 θ σ/ + c Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 1 Makig size α LRT To make a size α test, sup βθ α θ Ω 0 sup Pr Z θ 0 θ θ θ 0 σ/ + c α Pr Z c α c z α Note that Pr Z θ 0 θ σ/ + c is maximized whe θ is maximum ie θ θ 0 Therefore, size α LRT test rejects H 0 if ad oly if x θ 0 σ/ z α Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 1 Aother Example of LRT Problem iid X 1,, X fx θ e x θ where x θ ad < θ < Fid a LRT testig the followig oe-sided hypothesis H 0 : θ θ 0 Solutio Lθ x H 1 : θ > θ 0 e xi θ Ix i θ e x i +θ Iθ x 1 Solutio cot d Whe θ Ω c 0, the likelihood is still a icreasig fuctio, but bouded by θ mix 1, θ 0 Therefore, the likelihood is maximized whe θ ˆθ 0 mix 1, θ 0 The likelihood ratio test statistic is λx { e x i +θ 0 e x i +x 1 if θ 0 < x 1 1 if θ 0 x 1 { e θ 0 x 1 if θ 0 < x 1 1 if θ 0 x 1 The likelihood fuctio is a icreasig fuctio of θ, bouded by θ x 1 Therefore, whe θ Ω R, Lθ x is maximized whe θ ˆθ x 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 16 / 1
Solutio cot d LRT based o sufficiet statistics The LRT rejects H 0 if ad oly if e θ 0 x 1 c ad θ 0 < x 1 θ 0 x 1 log c x 1 θ 0 log c So, LRT reject H 0 is x 1 θ 0 log c ad x 1 > θ 0 The power fuctio is βθ X 1 θ 0 log c X 1 > θ 0 Theorem 824 If TX is a sufficiet statistic for θ, λ t is the LRT statistic based o T, ad λx is the LRT statistic based o x the λ Tx] λx for every x i the sample space To fid size α test, we eed to fid c satisfyig the coditio sup βθ α θ θ 0 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 18 / 1 Proof Proof cot d By Factorizatio Theorem, the joit pdf of x ca be writte as fx θ gtx θhx ad we ca choose gt θ to be the pdf or pmf of Tx The, the LRT statistic based o TX is defied as λ t sup θ Ω 0 Lθ Tx t sup θ Ω Lθ Tx t sup θ Ω 0 gt θ sup θ Ω gt θ LRT statistic based o X is λx sup θ Ω 0 Lθ x sup θ Ω Lθ x sup θ Ω 0 fx θ sup θ Ω fx θ sup θ Ω 0 gtx θhx sup θ Ω gtx θhx sup θ Ω 0 gtx θ sup θ Ω gtx θ λ Tx The simplified expressio of λx should deped o x oly through Tx, where Tx is a sufficiet statistic for θ Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 19 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 1
Example Problem iid Cosider X 1,, X N θ, σ 2 where σ 2 is kow H 0 : θ θ 0 H 1 : θ θ 0 Fid a size α LRT Solutio - Usig sufficiet statistics TX X is a sufficiet statistic for θ T N θ, σ2 λt sup θ Ω 0 Lθ t sup θ Ω Lθ t 1 2πσ 2 / exp t θ 0 2 / sup θ Ω 1 2πσ 2 / exp ] t θ2 / Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 1 ] Solutio cot d The umerator is fixed, ad MLE i the deomiator is ˆθ t Therefore the LRT statistic is λt exp t θ 0 2 ] LRT rejects H 0 if ad oly if λt exp t θ 0 2 ] c t θ 0 σ/ 2 log c c Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 1 Solutio cot d Note that A size α test satisfies T X N T θ 0 σ/ sup Pr θ Ω 0 θ, σ2 N 0, 1 T θ σ/ c α T θ 0 Pr σ/ c α Pr Z c α PrZ c + PrZ c α Z T θ σ/ z α/2 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 1 LRT with uisace parameters Problem iid X 1,, X N θ, σ 2 where both θ ad σ 2 ukow Betwee H 0 : θ θ 0 ad H 1 : θ > θ 0 1 Specify Ω ad Ω 0 2 Fid size α LRT Solutio - Ω ad Ω 0 Ω {θ, σ 2 : θ R, σ 2 > 0} Ω 0 {θ, σ 2 : θ θ 0, σ 2 > 0} Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 24 / 1
Solutio - Size α LRT Solutio - Maximizig Numerator λx sup {θ,σ 2 :θ θ 0,σ 2 >0} Lθ, σ 2 x sup {θ,σ 2 :θ R,σ 2 >0} Lθ, σ 2 x For the deomiator, the MLE of θ ad σ 2 are { ˆθ X σ 2 Xi X 2 1 s2 X Step 1, fix σ 2, likelihood is maximized whe x i θ 2 is miimized over θ θ 0 { x if x θ0 ˆθ 0 θ 0 if x > θ 0 For umerator, we eed to maximize Lθ, σ 2 x over the regio θ θ 0 ad σ 2 > 0 1 Lθ, σ 2 x exp x i θ 2 ] 2πσ 2 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 26 / 1 Solutio - Maximizig Numerator cot d Step 2 : Now, we eed to maximize likelihood or log-likelihood with respect to σ 2 ad we substitute ˆθ 0 for θ lˆθ, σ 2 x log 2π + log σ 2 xi ˆθ 0 2 2 log l σ 2 + xi ˆθ 0 2 2σ 2 2 0 ˆσ 0 2 x i ˆθ 0 2 Combiig the results together λx { 1 if x θ0 ˆσ 2 ˆσ 2 0 /2 if x > θ 0 Solutio - Costructig LRT LRT test rejects H 0 if ad oly if x > θ 0 ad ˆσ 2 /2 c ˆσ 2 0 xi x 2 / xi θ 0 2 / /2 c xi x 2 xi θ 0 2 c xi X 2 xi X 2 + x θ 0 2 c 1 1 + x θ c 0 2 xi x 2 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 27 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 1
Solutio - Costructig LRT cot d Ubiased Test x θ 0 s X / c x θ 0 2 xi x 2 c x θ 0 s X / c LRT test reject if The ext step is specify c to get size α test omitted Defiitio If a test always satisfies Prreject H 0 whe H 0 is false Prreject H 0 whe H 0 is true The the test is said to be ubiased Alterative Defiitio Recall that βθ reject H 0 A test is ubiased if βθ βθ for every θ Ω c 0 ad θ Ω 0 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 29 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 1 Example X 1,, X iid N θ, σ 2 where σ 2 is kow, testig H 0 : θ θ 0 vs H 1 : θ > θ 0 LRT test rejects H 0 if x θ 0 σ/ > c X θ0 βθ σ/ > c X θ + θ θ0 σ/ X θ > c σ/ + θ θ 0 σ/ > c X θ σ/ > c + θ 0 θ σ/ Example cot d Therefore, for Z N 0, 1 βθ Z > c + θ 0 θ σ/ Because the power fuctio is icreasig fuctio of θ, βθ βθ always holds whe θ θ 0 < θ Therefore the LRTs are ubiased Note that X i N θ, σ 2, X N θ, σ 2 /, ad X θ σ/ N 0, 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 1 Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 32 / 1
Summary Today Examples of LRT LRT based o sufficiet statistics LRT with uisace parameters Ubiased Test Next Lecture Uiformly Most Powerful Test Hyu Mi Kag Biostatistics 602 - Lecture 19 March 26th, 2013 33 / 1