Pobably heoy Mgf of s [ E M e h: E M [ If Y eee he M + Y[ M[ MY[ Chaacesc fuco: φ [ E e fomaos: Y g If scee he fy[ y f[ x I[ g[ x y If couous v he ao Ξ A A ( P[ A ) efe g[ x g[ x x A so ha each g s moooous he fy[ y f [ { : } g y y g y y Ψ y x A y g x If co s veco v a Y g[ he efe ao Ξ A A ( P[ A ) so ha g [ x g[ x x A a each g s oe-o-oe he fy[ y f [ g y J whee J s a Jacoba of vese fomao: J g y y Covoluo: f Y eee he f ± Y[ z f[ w fy[ ± ( zw) w [ z f Y z f w fy[ w w f w / Y[ z f[ zw fy[ w w w If ~ [ µ Y ~ ν [ τ eee he Y µ ν τ LIE: E[ Y E[E[ Y Va[ Y E[Va[ Y + Va[E[ Y Dsbuos + ~ [ + + If ~ [ Po Y ~ P Bomal: P[ C ( ) E Va ( ) M ( e + ) Posso: P[ e λ λ! E λ Va λ M e λ Ufom: f [ x ( ba) x [ a; b α x α Γ[ α α E ( b+ a) Va ( b a) ( e ) α o λ eee he + Y ~ Po [ + λ Gamma: f x x e x > > > E α Va α M ( ) < Ces: γ[ χ γ[ ex[ Ch-squae: z Σ z ~ χ [g Σ x f[ x Γ [ e x x> E Va If z ~ [ I A emoe he zaz ~ χ [g A If z ~ [ Σ he x λ Exoeal: f[ x e x> λ > E Va E λ λ λ! omal: Bea: Logomal: λ f[ x e E Va E[ µ µ ()!! π ( xµ ) ( ) α f x x x x > > E Va E Γ [ α+ α Γ[ α Γ[ (;) α Exoeal famly: Mulomal omal: Iequales f x e x > E e Va e e (l ) ( xµ ) πx α Γ [ α+ Γ [ α+ α+ ( α+ ) ( α+ + ) Γ[ a Γ [ α+ + µ + ( + ) + f [ x h[ x cex w x w w he E l Va l E c c w ( x µ Σ ( x µ ) x π Σ e If ~ [ he y µ Σ Σ f Chebychev: P[ g ε ε E g[ x µ Σ Σ y x ~ [ µ + Σ Σ x µ Σ Σ Σ Σ ξ 9 ε ε ξ 8 3 Vysochĭ-Peu: ~ f umoal efe ξ E[( α) fo ay α he P[ α > ε ξ 9 ε 3 ε ξ 8 3 Se s lemma: Höle: f µ g µ g ~ [ E[ E[ q q + q he E Y (E ) (EY ) Cauchy-Schwaz: E Y E E Y Mows: E[ + Y E[ + E[ Y (?): E[ + Y max[ (E + E Y ) Jese: f g[ x covex he E g g[e s Laouov: Covaace equaly-ii: f g h boh o-ceg o o-eceg he E[ g[ h E[ g E[ h Covegece { } coveges almos suely o v f P ω lm [ ω [ ω { } coveges L o v f lm E[ { } coveges obably o v f fo ε > lm P[ > ε { } coveges sbuo o v f lm P[ x P[ x fo all x whee F [ x s couous L L E E < s elaosh: q fo q L h: f fo λ sequece of scala v s λ λ he Couous mag heoem: f a h[ s couous he h [ h Ma&Wal: If g : l s couous he g[ g[ ; g[ g[ ; g[ g Slusy: f Y α he + Y + α ; Y α Y α ; Y α Y α whe P[e Y Dela-meho: f cos [ l Σ g : s coff a he ( g[ g) [ GΣG whee G g[ x WLL (wea law of lage umbes): Le be wh E µ < he µ LL Kolmogoov (): Le { } be a E[ exss he E[
LL Kolmogoov (): Le { } be e wh Va a < he E LL Bhoff-Khch (egoc): Le { } be saoay a egoc he E[ CL Lebeg-Lévy: Le { } be wh E[ µ a Va[ he ( µ ) [ CL Laouov: Le { } be e wh E[ µ Va[ CL Lebeg-Felle: Le { } be e wh E[ µ ( ) ( µ ) [ a E[ µ ν Va[ CL Bllgsley: If { } saoay & egoc magale ffeece sequece he 3 3 If ( ) ( ν) lm Defe C ( ) If fo ε > E[ < he CL Aeso: Le { } be saoay & egoc wh w Cov[ < he Daa euco [ + z ( ) wz E [ ( µ ) ( ) [ x µ F x lm C xµ εc Paamec moel: P { : Θ} Paamee: ay mag ν : P (aual aameezao: ν ) Paamee ν s efable f P P P ν[ P ν[ P Paamee s efable f P P o equvalely P P P Sc ay meuable fuco of he aa [ Sc s suffce fo f [ oes o ee o Sc S s acllay fo f fs[ oes o ee o Sc S s oe acllay fo f E S oes o ee o Sc s mmal suffce f fo ay ohe suffce sc S we ca f fuco such ha [ S[ Sc [ s comlee f! g such ha g [ s fs-oe acllay Facozao h: sc : Τ s suffce fo f g : Θ a h : such ha f [ x g[ [ x h[ x fo x Θ h: f [ s such ha xy ao f[ x f[ y oes o ee o ff [ x [ y he s mmal suffce fo h: fo ex famly f[ x h[ xex h sc x B [ s comlee f η[ Θ co a oe se Bu h: f s comlee a mmal suffce he s eee of ay acllay sc h: f mmal suffce sc exss he ay comlee sc wll also be mmal suffce f ( ) Esmao oees Esmao ay meuable fuco of he aa φ [ Esmao φ[ s ube fo aamee g[ f fo Θ E[ g[ Ube esmao φ[ s a UMVUE (ufomly mmum vaace ube esmao) f Va [ φ < a fo ay ohe ube δ [ we have Va φ[ Va δ[ Camé-ao eq: Le { } ~ f[ x a φ[ be ube fo g[ s E[ φ oe of eg & ff s echageable he I CLB whee l f[ x l f[ x l f[ x I[ E [ E [ (Fshe Ifomao max) ψ ψ Va [ φ CLB aame: Le { } ~ a W s ube fo τ[ he a CLB ff a W[ x τ l L[ x fo some a[ Hausma cle: W s UMVUE of τ[ ff W s ucoelae wh all ube esmaos of ao-blacwell h: Le W be ube es of τ[ a W be suffce sc fo he φ E[ W s UMVUE of τ[ Lehma-Scheffé h: Le be a comlee suffce sc fo he φ [ be oly o s he uque UMVUE of E φ [ Hyoheses esg Hyohess ay saeme abou moel aamee ull hyohess: Θ aleave hyohess: Θ whee φ P he Θ Θ Aco sace A {} whee s eeco of ull Loss fuco: l[ a I [ Θ es fuco: δ : {} Ccal ego: C { x : δ[ x } ye-i eo o a eec H whe Θ ye-ii eo o acce H whe Θ Powe fuco: P [ δ[ Θ (obably o coecly eec whe Θ ) Sze of es: sze su Θ δ Level of es s α f su Θ δ α P-value of es: ˆ[ fα (;): C α If es s δ I [ c he efe α[ c su Θ P[[ c a -value s α[ [ x es φ s ube of level α f φ α Θ a es φ α Θ c C s ufomly mos oweful cls C f c fo c C Θ Famly { P Θ } s moooe lelhoo ao famly f fo > P P f[ x a f[ x s a moooe fuco of some [ x es φ s α-smla o Θ Θ f φ α Θ Mag S : s ( α) cofece ego fo aamee ν[ f P [ S[ { ν} α eyma-peo h: cose H: vs H : a lelhoo-ao es fuco [ [ [ { f f x f f x [; f f x φ x > < } he ) φ s MP cls of all level δ f [ x f[ x f[ x α E φ[ x ess; ) fo α exss MP level α of he fom φ ; 3) f a es φ s MP he h fom of φ Kal-ub h: suose { P Θ } s ML ceg [ x Defe δ [ x I [ [ x > he ) b δ [ s ceg ; ) δ s UMP level α E [ δ[ x fo esg H : ag H: > h: cose a ex famly f[ x e [ x A a a es level α fo esg H: vs H: ff E [ φ α a E [ φ αe φ [ x { f x < c x > c f x ( c c) γ [ xf x c} he hs es s UMPU Dualy h: Le δ be level α es of : a A[ { x : δ [ x } Defe S[ x { Θ: x A[ } he S[ s ( α) cofece se H Covesely f S[ s ( α) cofece se he A [ { x : S[ x} s acceace ego of a level α es of H: α H
Dualy h fo ML: suose { P Θ } s ML ceg [ x a F [ s co s If F [ α h soluo l[ α Θ a F [ α h soluo u[ α Θ he α α: α+ α < eval [ l[ α u[ α s a ( α α) cofece eval fo OLS Moel: y x + ε ; sace fom: y + ε Assumos: E[ ε (sc exogeey); E[ εε I (homoscecy); P[g (o mulcolleay) OLS esmaos: ˆ ( ) y ˆ εε ˆ ˆ s ˆˆ εε whee yˆ ˆ Py εˆ y ˆ My P ( ) M IP Paoe egesso: y + + ε he ˆ M M y ˆ ( M ) My Fsch-Waugh h: ˆ fom y s he same fom egesso y whee y ae esuals y a ae esuals Coollay: f egesso co ece you ca fs emea a he cay ou egesso h: f z s oe of egessos he aal coelao zy zz yy + #f sg[ whee z s -sc fo z yz z z SSegesso sum of squaes SSoal --"-- ESSeos --"-- If co ι he L I ιι a ( ) Whe z s ae o egesso he z + ( ) yz Fe samle oees E[ ˆ ˆ Va[ ( ) x x x a SS SS > wll cee oly f z y My Ause y Ly : E[ s Cov[ ε Gauss-Maov h: ˆ s BLUE (bes lea ube esmao) Ue omaly sumo: ˆ ~ [ s ~ χ [ ( Va[ s ) a ˆ s ae eee es H : b usg ( ) ~ [ ˆ b s es H : (exce ece) usg SS F ( ) ~ F[ es H : q usg ESSESS ( ) q ESS q qs ~ [ ESS F ˆ ˆ Fq Cosae esmao: + ( ) λ λ ( ( ) ) ( ˆ) Peco: bes eco: BP[ y x E[ y x bes lea eco: BLP[ y x x E[ E[ x y Lage samle oees Deoe Q E[ > m E[ ε he ysbuos: ( ) ( ) [ ˆ [ Q ( ˆ ) [ m es H : b usg ˆ b ˆ es : H g q usg Wal s es: ˆ W ˆ g[ ( G[ ( ) G ) g[ χ [ q Lelhoo ao es: L L L χ q Lagage mulle es: ˆ ˆ g[ LM λ ( G[ ( ) G ) λ χ [ q whee G (l l ) ˆ ˆ q Heeoscecy ce: eoe Q E[ ε he ˆ [ Q e Q Q e If E[( ) exss a fe fo he HCSE (heeoscecy-cosse saa eos): AVa[ QQ Q e Qˆ x x ˆ ˆ ε Q ε x x ε e x x x x o ( x ( ) x ) Whe s heeoscecy es: egess ˆ ε ψ whee ψ co uque o-cosa elemes of ; he χ [m ψ ue H GLS WLS Moel: y + ε whee E[ ε E[ εε Σ (ow) OLS esmao h oees E[ ˆ OLS a ˆ Va[ OLS ( ) Σ ( ) Geealze le squaes (GLS) esmao: ˆ ( Σ ) Σ y wh E[ ˆ Va[ ˆ ( Σ ) ; hs esmao s BLUE fo hs moel h: OLS ~ GLS f ( ) Σ B fo some o-sgula B ; ( ) ΣZ fo Z: Z ; ( ) Σ Γ + ZΘZ + I fo some ΓΘΖ : Z Cooal heeoscecy Assumo: Σ ag[ [ x [ x ; E[ > Q x x x he ˆ ( ) ( xy GLS [ x ) x ˆ [ Q Feble GLS: ehe esmae [ x o-aamecally o u auxlay egesso ˆε Z a he use ˆ [ x z γˆ : xy IV GLS ˆ ( ˆ ) ( FGLS x ˆ ) x Moel: y x + ε whee { y x z } s saoay&egoc E[ z ε Q zx E[ z x a coo fo ID: g Qzx oe coo fo ID: l l Esmao woul be yomal whe { z ε} s ms a Q E[ ε zz > zzε GMM Moel: Secal ces: OLS: E[ x ε WLS: E[ x [ x ε SU Moel: y + u whee E[ u E[ uu ; sace fom: y + u whee ag[ K + + K K GLS esmao: ˆ ( ( Σ I) ) ( Σ I) y (f Σ ow) FGLS esmao buls uo uˆ y ˆ (hee ˆ ( ) y ) ˆ ˆ uu ˆ ˆ ˆ Ω Σ I ( ) Asymoc sbuo: h: OLS ~ GLS f ( ) Σ s agoal; ( ) ( ˆ ) ( ˆ ) [ lm ( Σ I ) GLS FGLS
SEM Moel: y() Γ x() B + u() K K ; sace fom: Y Γ Β + U ; sumos: ~ [ u() Σ P[g K lm > e Γ Γ K K euce fom: Y Π + V whee Π ΒΓ V UΓ Λ Γ ΣΓ Deoe γ ge{ : } Γ Γ Y ge{ : } y Γ ge{ : } K K Β Β ge{ x : Β } Z Y ( ) α γ L + K y ( y y α ( α α Z ag[ Z Z Coveoal fom: y Y γ + + u Z α + u ; sace fom: Iefcao K K L L ΣL Σ L y Z α + u whee Ω E[ uu Σ I (/) K (/) We ΠΓ Β π Π γ π Π γ whee π ge{ Π : Β ( / )} Π ge{ Π : ( / ) } K Β Γ K K K K K K( ) I wos Π cosss les eseco of hose colums of Π whch coeso o clue ( h equao) eogeous vaables a hose ows whch coeso o exclue exogeous vaables Oe coo: K a coo: Π () Full Ifomao Moel g l L l[ π + l Γ l Σ Σ ( YΓ Β) ( YΓ Β) coceae log-l: ll ˆ l ( YΒΓ )( YΒΓ ) αfiml 3SLS: ) oba α fo esmae ˆ uû ˆ whee ˆ 3) f ˆ αˆ Z ( Σ IZ ) Z ( Σ Iy ) αˆ ˆ SLS Lme Ifomao Moel ˆ SLS u y Zα Moel: y Yγ + + u Zα + u Y Π + V whee ( u V )~ [ Σ { Σ Σ} a gπ αˆ Z ( IλM) Z Z ( IλM) y whee ( ) M I M I ( ) W ( y Y M( y Y ) LIML esmao: LIML W ( y Y M ( y Y ) a λ s smalles chaacesc oo of WW Π Deoe A lm[ he I Π ( ) Π y Y αsls ( )( AVa[ α ˆ ) SLS ( y Zα )( y Zα ) ZPZ LLS Moel: Bay choce moels Moel: P[ y F [ x whee F [ x f[ x > x f x F[ x [ x e π Π I 3SLS A SLS esmao: αˆ ( ZPZ ) ZPy whee P ( ) SLS α α α α A Ieeao of SLS: ) Y Yˆ ) LIML SLS [ x ~ E[ > Secal ces: lea obably moel: F [ x x ob moel: FIML x Φ log moel: Fx Λ [ x ( + e ) Log-lelhoo: l L ( y l l[ ) F x + y F x hs fuco s globally cocave fo log a ob secfcaos ob ˆ l [ I whee L[ lm E lm f I x F[ x ( F[ x) ye-i moel: { y x + u y max[ y} ; sumos: u ~ [ obseve: { y x } x ~ E[ > Lelhoo fuco: L ( Φ[ x ) [ φ y x ucae moel: aa fo y < uobseve [ [ φ y φ[ z L Φ x x Deoe λ[ z Φ[ z Hecma wo-se: ) esmae α ob P[ y > Φ[ x α by MLE ) egess y [ ˆ x λ xα usg samle y > Seos mus be comue wh Whe s HSCE fomula LS: aly o y x + λ[ x + ε LWLS: aly o same eq wh Va[ ε x xα λ[ x α λ[ x α log-lelhoo globally cocave ems of α a All esmaos ae cosse f aa seally coelae bu cosse ue heeoscecy o o-omaly eo em ye-ii moel: { y x + u y y >? x + u : } ; sumos: ( u u)~ [ ( ρ) obseve: { y sg y x x} Lelhoo fuco: L Φ[ x Φ x + ρ( y x ) φ ( y x ) y If hee ae o cos o y ρ aamees he s uefe α Hecma wo-se: y x + ρλ[ x α + ε ( y ) Vaε ρ x αλ[ x α ρ λ[ x α ye-iii moel: { y x + u y max[ y y y >? x + u : } L P[ y f[ y y y y> MLE: ye-iv moel: { y x + u y max[ y y y >? x + u : y y? x + u :} L f [ y y y f [ y y 3 3 3 3 y 3 3 y> ye-v moel: { y x + u y y >? x + u : y y? x + u : } L f [ y y y f [ y y y me sees 3 3 3 3 y y3 3 3 Pocess { z } s scly saoay f f [ z z ees oly o bu o o I s wealy saoay (o -oe saoay) f E z µ cos a Cov[ z z s Γ Γ s s fo s Fo scala ocesses auocoelao fuco: ρ γ γ Seco-oe saoay ocess s whe ose f µ a Γ fo s s Pocess { z } s calle magale f E[ z z z z Saoay ocess { z } s egoc f m l g lm E [ f[ g[ E [ f[ E [ g[ fo f : : : z z z z z z z z + m + + + l + m + l
Samle auocovaace: ˆ γ ( z )( z z z) samle ACF: ˆ ρ + γ γ h: If z µ + ε whee ε s saoay ms wh E[ ε ε ε he: γˆ [ I ρˆ [ I whee γ ˆ ( ˆ γ ˆ γ ρ ˆ ( ˆ ρ ˆ ρ Box-Pece Q: ˆ ρ Lug-Box Q: χ + ˆ ρ χ Suose y + ε εε + ˆ γ ˆ ρ γ γ he x { y x} s saosy&egoc ε ε x ~ v [ a E[ > If we calculae: γ I Φ ρˆ [ I Φ whee Φ E[ xε Q E[ x ε ˆ [ ( )