Eco 60 Three Classical Tests; Wald, MScore, ad R tests Suppose that we have the desity l y; θ of a model with the ull hypothesis of the form H 0 ; θ θ 0. et θ be the lo-likelihood fuctio of the model ad θ be the ME of θ. Wald test is based o the very ituitive idea that we are willi to accept the ull hypothesis whe θ is close to θ 0. The distace betwee θ ad θ 0 is the basis of costructi the test statistic. O the other had, cosider the followi costraied maximizatio problem, max θ Θ θ s.t. θ θ 0 If the costrait is ot bidi the ull hypothesi is true, the araia multiplier associated with the costrait is zero. We ca costruct a test measuri how far the araia multiplier is from zero. - M test. Fially, aother way to check the validity of ull hypothesis is to check the distace betwee two values of maximum likelihood fuctio like l θ θ 0 lo y; θ l y; θ 0 If the ull hypothesis is true, the above statistic should ot be far away from zero, aai. Asymptotic Distributios of the Three Tests Assume that the observed variables ca be partitioed ito the edoeous variables X ad exoeous variables Y. To simplify the presetatio, we assume that the observatios Y i,x i are i.i.d. ad we ca obtai coditioal distributio of edoeous variables ive the exoeous variables as f y i x i ; θ with θ Θ R p. The coditioal desity is kow up to ukow parameter vector θ. By i.i.d. assumptio, we ca write dow the lo-likelihood fuctio of observatios of Y i,x i as θ lo f y i x i ; θ We assume all the reularity coditios for existece, cosistecy ad asymptotic ormality of ME ad deote ME as θ. The hypotheses of iterest are ive as H 0 ; θ 0 0 H A ; θ 0 0 where ;R p R r ad the rak of Wald test Propositio 1 is r. ξ W θ θ I θ θ θ χ r uder H 0. where I E X E θ lo fy X;θ ad I θ is the iverse of I evaluated at θ θ. From the asymptotic characteristics of ME, we kow that d θ θ 0 N 0, I θ 0 1 1
The first order Taylor series expasio of θ aroud the true value of θ 0, we have θ θ θ 0 θ 0 + θ 0 θ θ 0 + o p 1 θ 0 θ θ 0 + o p 1 Hece, combii 1 ad ives d θ θ 0 N 0, θ 0 I θ 0 θ 0 3 Uder the ull hypothesis, we have θ 0 0. Therefore, d θ N 0, θ 0 I θ 0 θ 0 4 By formi the quadratic form of the ormal radom variables, we ca coclude that θ θ 0 I θ 0 θ 0 θ χ r uder H 0. 5 The statistic i 5 is useless sice it depeds o the ukow parameter θ 0. However, we ca cosistetly approximate the terms i iverse bracket by evaluati at ME, θ. Therefore, ξ W θ θ I θ θ θ χ r uder H 0. A asymptotic test which rejects the ull hypothesis with probability oe whe the alterative hypothesis is true is called a cosistet test. Namely, a cosistet test has asymptotic power of 1. The Wald test we discussed above is a cosistet test. A heuristic arumet is that if the alterative hypothesis is true istead of the ull hypothesis, p θ θ0 0. Therefore, θ θ I θ θ θ is coveri to a costat istead of zero. By multiplyi a costat by, ξ W implies that we always reject the ull hypothesis whe the alterative is true. as, which Aother form of the Wald test statistic is ive by - cautio: this is quite cofusi - ξ W θ θ θ I θ θ χ r uder H 0. where I E X E θ θ E X E θ loy i x i;θ ad I θ is the iverse of I evaluated at θ θ. Note that I I. A quite commo form of the ull hypothesis is the zero restrictio o a subset of parameters, i.e., H 0 ; θ 1 0 H A ; θ 1 0 where θ 1 is a q 1 subvector of θ with q<p.the, the Wald statistic is ive by ξ W θ 1 I 11 θ θ1 χ q uder H 0. where I 11 θ is the upper left block of the iverse iformatio matrix, [ ] I11 θ I I θ 1 θ I 1 θ I θ the, I 11 θ I 11 θ I 1 θ I θ by the formula for partitioed iverse. I 11 evaluated at ME. θ is I 11 θ
M test Score test If we have a priori reaso or evidece to believe that the parameter vector satisfies some restrictios i the form of θ 0, icorporati the iformatio ito the maximizatio of the likelihood fuctio throuh costraied optimizatio will improve the efficiecy of estimator compared to ME from ucostraied maximizatio. We solve the followi problem; max θ s.t. θ 0 FOC s are ive by θ θ + λ 0 6 θ 0 7 where θ is the solutio of costraied maximizatio problem called costraied ME ad λ is the vector of arae multiplier. The M test is based o the idea that properly scaled λ has a asymptotically ormal distributio. Propositio ξ S θ I θ θ 1 θ λ I θ θ λ χ r uder H 0. First order Taylor expasios of θ ad θ aroud θ 0 ives, iori o p 1 terms, θ θ 0 + θ 0 θ θ 0 θ θ 0 + θ 0 θ θ 0 Note that θ 0 from 7 ad substracti 9 from 8, we have θ θ 0 θ θ O the other had, taki first order Taylor series expasios of θ ad θ iori o p 1 terms, θ θ θ θ 0 + θ 0 θ θ 0 θ 0 + 1 θ 0 θ θ 0 θ 0 Iθ 0 θ θ 0 8 9 10 aroud θ 0 ives, 11 ote that 1 θ 0 1 p loy i x i;θ Iθ 0 by the law of lare umbers. Similarly, θ θ 0 Iθ 0 θ θ 0 1 3
Cosideri the fact that θ 0 by FOC of the ucostraied maximizatio problem, we take the differece betwee 11 ad 1. The, θ I θ 0 θ θ I θ 0 θ θ 13 Hece, θ θ I θ 0 θ From 10 ad 14, we obtai θ θ 0 I θ 0 θ 14 Usi 6, we deduce θ θ θ 0 λ I θ 0 θ 0 I θ 0 θ 0 λ 15 sice θ p θ0 hece θ p θ0. Therefore, λ θ0 I θ 0 θ 0 θ 16 From 4, uder the ull hypothesis, d θ N 0, θ0 I θ 0 θ 0. Cosequetly, we have λ d θ0 N 0, I θ 0 θ 0 17 Aai, formi the quadratic form of the ormal radom variables, we obtai 1 λ θ0 I θ 0 θ 0 λ χ r uder H 0. 18 Alteratively, usi 6, aother form of the test statistic is ive by θ θ I θ 0 χ r uder H 0 19 Note that 18 ad 19 are useless sice they deped o the ukow parameter value θ 0. We ca evaluate the terms ivolved i θ 0 at the costraied ME, θ to et a usable statistic. Aai, aother form of M test is ξ S θ I θ 0 θ We ca approximate I θ 0 with either 1 loy i x i; θ or 1 λ θ0 I θ 0 θ 0 λ. loy i x i; θ loy i x i; θ. If 4
we choose the secod approximatio, the M test statistic becomes ξ S θ 1 lo lo θ 1 lo 1 lo lo lo lo lo lo lo this expressio seems quite familiar to us - looks like a projectio matrix -. The ituitio is correct. The ucetered Ru from the reressio of 1 o loyi xi; θ is ive by [ where X p The, R u 1 X X X X X X X 1 1 1 loy 1 x 1; θ loy x ; θ 1 X X X X 1 1 1 loy x ; θ ] ad 1 1 [ 1 1 1 ]. Hece, R u loy i x i; θ loy i x i; θ loy i x i; θ loy i x i; θ ξ S R u This is quite a iteresti result sice the computatio of M statistic is othi but a OS reressio. We reress 1 o the scores evaluated at costraied ME ad compute ucetered R ad the multiply it with the umber of observatios to et M statistic. Oe thi to be cautious is that most software will automatically try to prit out cetered R, whichisimpossibleithiscasesice the deomiator of cetered R is simply zero. M test is also a asymptotically cosistet test. From 16 ad 18, ξ W θ θ I θ θ θ θ θ 0 I θ 0 θ 0 θ ξ S ikelihood ratior test Propositio 3 ξ R θ θ χ r uder H 0. We cosider the secod order Taylor expasios of θ ad θ aroud θ 0. Uder the ull 5
hypothesis, iori stochastically domiated terms, θ θ 0 + θ 0 θ θ 0 + 1 θ θ 0 θ 0 θ 0 + θ 0 θ θ 0 + θ 0 θ θ 0 + 1 θ θ 0 θ θ 0 + 1 θ θ 0 1 θ θ 0 θ 0 θ θ 0 θ 0 + 1 θ 0 θ θ 0 + 1 θ θ 0 1 θ 0 θ θ 0 θ 0 θ θ 0 Taki differeces ad multiplyi by, we obtai θ θ θ 0 θ θ + 1 θ θ 0 θ 0 θ θ 0 1 θ θ 0 θ 0 θ θ 0 θ θ 0 I θ0 θ θ θ θ 0 I θ0 θ θ 0 + θ θ 0 I θ0 θ θ 0 sice θ 0 I θ 0 θ θ 0 from 11 ad 1 θ 0 p Iθ0. Cotiui the derivatio, θ θ θ θ 0 I θ0 θ θ θ θ 0 I θ0 θ θ 0 + θ θ + θ θ 0 I θ0 θ θ + θ θ 0 θ θ 0 I θ0 θ θ θ θ 0 I θ0 θ θ 0 + θ θ I θ0 θ θ + θ θ I θ0 θ θ 0 + θ θ 0 I θ0 θ θ + θ θ 0 I θ0 θ θ 0 θ θ 0 I θ0 θ θ + θ θ I θ0 θ θ θ θ I θ0 θ θ 0 θ θ 0 I θ0 θ θ θ θ I θ0 θ θ ote that θ θ 0 I θ0 θ θ Now, from 13 ad 0, we have θ θ θ θ I θ0 θ θ 0. θ θ I θ0 θ θ θ θ I θ 0 I θ 0 I 1 θ 0 θ θ I θ 0 ξ S χ r uder H 0. Calculati R test statistic requires two maximizatios of likelihood fuctio oe with ad the other without costrait. R test is also a asymptotically cosistet test. As show above, Wald, M ad R test are asymptotically equivalet with χ r. 0 6
Examples of tests i the liear reressio model Suppose the reressio model such as The hypotheses are ive by H 0 ; The lo-likelihood fuctio is ive by R β r p p 1 y i β x i + ε i ε i i.i.. 0,σ γ H 0 ; Rβ γ β lo σ 1 σ y Xβ y Xβ The, the ucostraied ME is ive by Iformatio matrix is ive by β X X X y σ 1 y X β y X β I θ 0 The Wald statistic is, from Propositio 1, ξ W [ 1 σ X X 0 0 σ 4 [ [ ] R β γ R 0 I θ [ ]] R R β 0 γ [ [ ] R β γ R 0 I θ [ ]] R R β 0 γ [ R β γ R σ ] X X R R β γ 1 σ [ ] R β γ R X X R R β γ χ r uder H 0. ] Deote the costraied ME as β ad σ, respectively. The, σ σ 1 y X β y X β 1 y X β y X β 1 X β X β X β X β 1 β β X X β β 1 [ ] R β γ R X X R R β γ sice β β ] +X X R [R X X R R β γ. Therefore, ξ W [ σ ] σ σ [ ] R β γ R X X R R β γ y X β y X β [ [ ] R β γ R X X R R β γ ] /r [ y X β y X β ] / K 1 r K r K F 7
O the other had, the arae multiplier of the costraied maximizatio problem is λ σ R X X R γ R β Uder H 0, the distributio of the arae multiplier is 4 λ N 0, R X X R sice γ R β N σ 0, σ R X X R. The, the M test statistic is ξ S σ 4 λ R X X R λ 1 σ R β γ R X X R R β γ σ σ σ To obtai R test statistic, ote that θ O the other had, 1 1+ σ σ σ lo π lo σ 1 σ lo π lo σ σ 1 lo π lo σ σ σ lo π lo σ θ lo π lo σ 1 σ lo π lo σ σ 1 lo π lo σ σ σ lo π lo σ 1+ σ σ σ 1+ +K rf y X β y X β y X β y X β y X β y X β y X β y X β Hece, ξ R θ θ lo σ σ lo 1 1+ σ σ lo 1+ rf K lo σ + lo σ lo 1+ σ σ σ A iteresti result ca be obtaied usi the followi iequalities, x lo 1 + x x x > 1+x et x ad applyi the above iequalities,we obtai rf K ξ S ξr ξw 8