Variance Covariance Matrices for Linear Regression with Errors in both Variables. by J.W. Gillard and T.C. Iles

Transcript

1 Variace Covariace Matrices for Liear Regressio with Errors i both Variables by J.W. Gillard ad T.C. Iles School of Mathematics, Segheydd Road, Cardiff Uiversity, October 2006

2 Cotets 1 Itroductio 2 2 The Simple Liear Errors i Variables Model The Method of Momets Estimatig Equatios Estimatio of the Liear Structural Model Asymptotics for the Method of Momets 8 4 Derivig the Variace Covariace Matrices for Restricted Cases Derivatio of V ar[s xx ] Derivatio of Cov[s xx, s xy ] Costructig the Variace Covariace Matrices Shortcut Formulae for Variaces Shortcut Formulae for Covariaces Descriptio of Variace Covariace Matrices for Restricted Cases The Variace Covariace Matrices Itercept α kow Error variace σδ 2 kow Error variace σε 2 kow Reliability ratio κ = σ2 kow σ 2 +σδ Ratio of the error variaces λ = σ2 ε kow σδ Both variaces σδ 2 ad σ2 ε kow Variaces ad Covariaces for Higher Momet Estimators Estimator based o Third Momets Estimator based o Fourth Momets

3 Chapter 1 Itroductio I the techical report by Gillard ad Iles [9], several method of momets based estimators for the errors i variables model were itroduced ad discussed. Broadly speakig, the method of momets estimators ca be divided ito two separate classificatios. These are: Restricted parameter space estimators Higher momet based estimators Restricted parameter space estimators These estimators are based o first ad secod order momets, ad are derived by assumig that a parameter (or a fuctio of parameters) is kow a priori. Higher momet based estimators If the data displays sufficiet skewess ad kurtosis, oe may estimate the parameters of the model by appealig to estimators based o third ad fourth order momets. Although these estimators have the iitial appeal of avoidig a restrictio o the parameter space, they must be used with cautio. Details of how these estimators were derived, ad how they may be used i practise were agai give by Gillard ad Iles [9]. This preset techical report aims to provide the practitioer with further details cocerig asymptotic variace ad covariaces for both the restricted cases, ad higher momet based estimators. I this way, this report ca be viewed as a direct sequel to the techical report by Gillard ad Iles [9]. 2

4 Chapter 2 The Simple Liear Errors i Variables Model Cosider two variables, ξ ad η which are liearly related i the form η i = α + βξ i, i = 1,..., However, istead of observig ξ ad η, we observe x i = ξ i + δ i y i = η i + ε i = α + βξ i + ε i where δ ad ε are cosidered to be radom error compoets, or oise. It is assumed that E[δ i ] = E[ε i ] = 0 ad that V ar[δ i ] = σδ 2, V ar[ε i] = σε 2 for all i. Also the errors δ ad ε are mutually ucorrelated. Thus Cov[δ i, δ j ] = Cov[ε i, ε j ] = 0, i j Cov[δ i, ε j ] = 0, i, j (2.1) There have bee several reviews of errors i variables methods, otably Casella ad Berger [2], Cheg ad Va Ness [3], Fuller [7], Kedall ad Stuart [14] ad Spret [15]. Ufortuately the otatio has ot bee stadardised. This report closely follows the otatio set out by Cheg ad Va Ness [3] but for coveiece, it has bee ecessary to modify parts of their otatio. 3

5 Errors i variables modellig ca be split ito two geeral classificatios defied by Kedall [12], [13], as the fuctioal ad structural models. The fudametal differece betwee these models lies i the treatmet of the ξ is The fuctioal model This assumes the ξ is to be ukow, but fixed costats µ i. The structural model radom variable with mea µ ad variace σ 2. This model assumes the ξ is to be a radom sample from a It is the liear structural model that is the mai focus of this techical report. 2.1 The Method of Momets Estimatig Equatios The method of momets estimatig equatios follow from equatig populatio momets to their sample equivalets. By usig the properties of ξ, δ ad ε detailed above, the populatio momets ca be writte i terms of parameters of the model. This was also doe by Kedall ad Stuart [14], amogst others. E[X] = E[ξ] = µ E[Y ] = E[η] = α + βµ V ar[x] = V ar[ξ] + V ar[δ] = σ 2 + σδ 2 V ar[y ] = V ar[α + βξ] + V ar[ε] = β 2 σ 2 + σε 2 Cov[X, Y ] = Cov[ξ, α + βξ] = βσ 2 The method of momets estimatig equatios ca ow be foud by replacig the 4

6 populatio momets with their sample equivalets x = µ (2.2) ȳ = α + β µ (2.3) s xx = σ 2 + σ δ 2 (2.4) s yy = β 2 σ 2 + σ ε 2 (2.5) s xy = β σ 2 (2.6) Here, a tilde is placed over the symbol for a parameter to deote the method of momets estimator. From equatios (2.4), (2.5) ad (2.6) it ca be see that there is a hyperbolic relatioship betwee the method of momets estimators. This was called the Frisch hyperbola by va Motfort [16]. (s xx σ δ)(s 2 yy σ ε) 2 = (s xy ) 2 This is a useful equatio as it relates pairs of estimates ( σ δ 2, σ2 ε) to the data i questio. Oe of the mai problems with fittig a errors i variables model is that of idetifiability. It ca be see from equatios (2.2), (2.3), (2.4), (2.5) ad (2.6) a uique solutio caot be foud for the parameters; there are five equatios, but six ukow parameters. A way to proceed with this method is to assume that there is some prior kowledge of the parameters that eables a restrictio to be imposed. The method of momets equatios uder this restrictio ca the be readily solved. Other tha this, additioal estimatig equatios may be derived by derivig equatios based o the higher momets. There is a compariso with this idetifiability problem ad the maximum likelihood approach. The oly tractable assumptio to obtai a maximum likelihood solutio is to assume that the distributios of ξ, δ ad ε are all Normal. Otherwise, the algebraic maipulatio required becomes a eormous task. Further details will be preseted i Gillard [8]. If all the distributios are assumed Normal, this leads to the bivariate radom variable (x, y) havig a bivariate Normal distributio. This distributio 5

7 has five parameters, ad the maximum likelihood estimators for these parameters are idetical to the method of momets estimators based o the momet equatios (2.2), (2.3), (2.4), (2.5), ad (2.6) above. I this case therefore it is ot possible to fid uique solutios to the likelihood equatios without makig a additioal assumptio, effectively restrictig the parameter space. A fuller treatmet of restrictios o the parameter space ad method of momets estimators ca be foud i Gillard ad Iles [9]. 2.2 Estimatio of the Liear Structural Model It is possible to write estimators for the parameters of the liear structural model i terms of β ad first or secod order momets. Oce a estimator for β has bee obtaied, the followig equatios ca be used to estimate the remaiig parameters. Equatio (2.2) immediately yields the ituitive estimator for µ µ = x (2.7) The estimators for the remaiig parameters ca be expressed as fuctios of the slope, β, ad other sample momets. A estimator for the itercept may be foud by substitutig (2.2) ito (2.3) ad rearragig to give α = ȳ β x (2.8) This shows, just as i simple liear regressio, that the errors i variables regressio lie also passes through the cetroid ( x, ȳ) of the data. Equatio (2.6) gives with β ad s xy sharig the same sig. σ 2 = s xy β (2.9) If the error variace σ 2 δ is ukow, it may be estimated usig (2.4) σ 2 δ = s xx σ 2 (2.10) 6

8 Fially, if the error variace σ 2 ε is ukow, it may be estimated usig (2.5) σ 2 ε = s yy β 2 σ 2 (2.11) I order to esure that the estimators for the variaces are o egative, admissibility coditios must be placed o the equatios. The straightforward coditios are icluded below s xx > σ 2 δ s yy > σ 2 ε. More precisely, the estimate of the slope must lie betwee the slopes of the regressio lie of y o x ad that of x o y for variace estimators usig (2.4), (2.5) ad (2.6) to be o egative. As stated i Gillard ad Iles [9], there is a variety of estimators for the slope. This techical paper will cocetrate o the asymptotic theory regardig some of these estimators. The estimators whose variaces are derived here will be stated later, but for more detail cocerig practical advice ad also the derivatio of the estimators, the reader is oce agai referred to Gillard ad Iles [9]. 7

9 Chapter 3 Asymptotics for the Method of Momets A commo misuderstadig regardig the method of momets is that there is a lack of asymptotic theory associated with the method. This however is ot true. Cramer [4] ad subsequetly other authors such as Bowma ad Sheto [1] detailed a approximate method commoly kow as the delta method (or the method of statistical differetials) to obtai expressios for variaces ad covariaces of fuctios of sample momets. The method is sometimes described i statistics texts, for example DeGroot [5], ad is ofte used i liear models to derive a variace stabilisatio trasformatio (see Draper ad Smith [6]). The delta method is used to approximate the expectatios, ad hece also the variaces ad covariaces of fuctios of radom variables by makig use of a Taylor series expasio about the expected values. The motivatio of the delta method is icluded below. Cosider a first order Taylor expasio of a fuctio of a sample momet x, f(x) where E[x] = µ. f(x) f(µ) + (x µ)f (µ) (3.1) Upo takig the expectatio of both sides of (3.1) the usual approximatio E[f(x)] f(µ) 8

10 is foud. Additioally, The otatio V ar [f(x)] = E [ {f(x) E[f(x)]} 2] {f (µ x )} 2 E[(x µ x ) 2 ] { f } x = f x = {f (µ x )} 2 V ar[x] { f } 2V = ar[x] x x=e[x] was itroduced by Cramer to deote a partial derivative evaluated at the expected values of the sample momets. This ca be aturally exteded to fuctios of more tha oe sample momet. For a fuctio f(x, y) V ar[f(x, y)] { f x } 2V ar[x] + { f y ad for a fuctio of p sample momets, x 1,..., x p, } 2V ar[y] + 2 { f x }{ f } Cov[x, y] y V ar[f(x 1,..., x p )] T V where T = ( f,... f ) x 1 x p is the vector of derivatives with each sample momet substituted for its expected value, ad V ar[x 1 ] Cov[x 1, x 2 ]... Cov[x 1, x p ] V =..... Cov[x 1, x p ] Cov[x 2, x p ]... V ar[x p ] is the p p matrix cotaiig the variaces of ad covariaces betwee sample momets. Covariaces betwee fuctios of sample momets ca be derived i a similar maer. 9

11 Chapter 4 Derivig the Variace Covariace Matrices for Restricted Cases Essetially, use of the method outlied above requires the prior computatio of the variace of each relevat sample momet, ad the covariaces betwee each sample momet. For each of the restricted cases discussed by Gillard ad Iles [9], the followig variaces ad covariaces are used. The variaces ad covariaces eeded to compute the asymptotics for the higher momet based estimators will be stated later o i this report. The variaces of the first ad secod order momets are: V ar[ x] σ2 + σ 2 δ V ar[ȳ] β2 σ 2 + σ 2 ε V ar[s xx ] (µ ξ4 σ 4 ) + (µ δ4 σ 4 δ ) + 4σ2 σ 2 δ V ar[s xy ] β2 (µ ξ4 σ 4 ) + σ 2 σ 2 ε + β 2 σ 2 σ 2 δ + σ2 δ σ2 ε V ar[s yy ] β4 (µ ξ4 σ 4 ) + (µ ε4 σ 4 ε) + 4β 2 σ 2 σ 2 ε (4.1) (4.2) (4.3) 10

12 The covariaces betwee all first ad secod order momets are: Cov[ x, ȳ] βσ2 Cov[ x, s xx ] µ ξ3 + µ δ3 Cov[ȳ, s xx ] βµ ξ3 Cov[ x, s xy ] βµ ξ3 Cov[ȳ, s xy ] β2 µ ξ3 Cov[ x, s yy ] β2 µ ξ3 Cov[ȳ, s yy ] β3 µ ξ3 + µ ε3 Cov[s xx, s xy ] β(µ ξ4 σ 4 ) + 2βσ 2 σδ 2 Cov[s xx, s yy ] β2 (µ ξ4 σ 4 ) Cov[s xy, s yy ] β3 (µ ξ4 σ 4 ) 2βσ 2 σ 2 ε (4.4) (4.5) Expressios (4.1), (4.2) ad (4.4) follow from the defiitio of the liear structural model. To show how these may be derived, the algebra uderlyig expressios (4.3) ad (4.5) shall be outlied. For brevity the otatio ξi = ξ i ξ ad ηi = η i η is itroduced. 4.1 Derivatio of V ar[s xx ] Sice ξ i ad δ i are ucorrelated we ca write [ 1 E[s xx ] = E ] (x i x) 2 i=1 [ 1 = E ] {(ξi ) + (δi )} 2 i=1 = 1 E [ i=1 (ξ i ) σ 2 + σ 2 δ. (ξi )(δi ) + i=1 ] (δi ) 2 i=1 The above result also follows from the method of momet estimatig equatio stated earlier, s xx = σ 2 + σ 2 δ. 11

13 E[(s xx ) 2 ] = 1 2 E [{ i=1 = 1 2 E [{ i=1 (x i x) 2 } 2 ] ( (ξ i ) + (δi ) ) } 2 ] 2 ((ξ i ) 4 + 4(ξ i ) 3 (δ i ) + 6(ξ i ) 2 (δ i ) 2 + 4(ξ i )(δ i ) 3 + (δ i ) 4 )] = 1 [ E 2 i=1 + 1 [ E 2 Hece it follows that i j ( (ξ i ) 2 (ξ j ) 2 + 2(ξ i ) 2 (ξ j )(δ j ) + (ξ i ) 2 (δ j ) 2 + 2(ξ i )(ξ j ) 2 (δ i ) )] + 4(ξi )(ξj )(δi )(δj ) + 2(ξi )(δi ) 2 (δj ) + (ξj ) 2 (δi ) 2 + 2(ξj )(δi )(δj ) 2 + (δi ) 2 (δj ) 2 1 ) ((µ 2 ξ4 + 6σ 2 σδ 2 + µ δ4 ) + ( 1)(σ 4 + 2σ 2 σδ 2 + σδ) 4 V ar[s xx ] = E[(s xx ) 2 ] E 2 [s xx ] (µ ξ4 σ 4 ) + (µ δ4 σ 4 δ ) + 4σ2 σ 2 δ 4.2 Derivatio of Cov[s xx, s xy ] E[s xx s xy ] = 1 [ ] E (x 2 i x) 2 (x i x)(y i ȳ) i=1 Now, (x i x) = (ξ i ) + (δ i ) ad (y i ȳ) = β(ξ i ) + (ε i ). Substitutig these ito the i=1 12

14 above summatio, ad multiplyig out leads to E[s xx s xy ] = 1 [ ( E β(ξ 2 i ) 4 + 2β(ξi ) 3 (δi ) + β(ξi ) 2 (δi ) 2 + β(ξi ) 3 (δi ) + 2β(ξi ) 2 (δi ) 2 i=1 +β(ξi )(δi ) 3 + (ξi ) 3 (ε i ) + 2(ξi ) 2 (δi )(ε i ) + (ξi )(δi ) 2 (ε i ) + (ξi ) 2 (δi )(ε i ) )] +2(ξi )(δi ) 2 (ε i ) + (δi ) 3 (ε i ) + 1 [ ( E β(ξ 2 i ) 2 (ξj ) 2 + 2β(ξi ) 2 (ξj )(δj ) + β(ξi ) 2 (δj ) 2 + β(ξi )(ξj ) 2 (δi ) Hece, i j +2β(ξi )(ξj )(δi )(δj ) + β(ξi )(δi )(δj ) 2 + (ξi )(ξj ) 2 (ε i ) + 2(ξi )(ξj )(δj )(ε i ) )] +(ξi )(δj ) 2 (ε i ) + (ξj ) 2 (δj )(ε j) + 2(ξj )(δi )(δj )(ε i ) + (δi )(δj ) 2 (ε i ) 1 ) ((βµ 2 ξ4 + βσ 2 σδ 2 + 2βσ 2 σδ) 2 + ( 1)(βσ 4 + βσ 2 σδ) 2 Cov[s xx, s xy ] = E[s xx s xy ] E[s xx ]E[s xy ] β(µ ξ4 σ 4 ) + 2βσ 2 σ 2 δ. 13

15 Chapter 5 Costructig the Variace Covariace Matrices For each restricted case, ad for the estimators of the slope based o the higher momets a variace-covariace matrix ca be costructed. As there are six parameters i the liear structural model µ, α, β, σ 2, σδ 2 ad σ2 ε the maximum size of the variace covariace matrix is 6 6. If the parameter space is restricted, the the size of the variace covariace matrix will decrease i accordace with the umber of assumed parameters. It is possible to use the delta method i order to costruct shortcut formulae or approximatios to eable quicker calculatio of each elemet of the variace covariace matrix. Usually, these shortcut formulae deped o the variability of the slope estimator ad the covariace of the slope estimator with a first or secod order sample momet. I some cases the variaces ad covariaces do ot deped o the slope estimator used, ad as a result are robust to the choice of slope estimator. These shortcut formulae are stated below, ad repeatig the style of the previous sectio, example derivatios will be give. For brevity, the otatio Σ = σδ 2σ2 ε + β 2 σ 2 σδ 2 + σ2 σε 2 is itroduced. This is the determiat of the variace covariace matrix of the bivariate distributio of x ad y. 14

16 5.1 Shortcut Formulae for Variaces Firstly, the shortcut formulae for the variaces will be cosidered. V ar[ α] will be the example derivatio provided. V ar[ µ] = σ2 + σ 2 δ V ar[ α] = µ 2 V ar[ β] + β2 σ 2 δ + σ2 ε + 2µ(βCov[ x, β] Cov[ȳ, β]) V ar[ σ 2 ] = σ4 β V ar[ β] + Σ + β2 (µ ξ4 σ 4 ) 2σ2 2 β 2 V ar[ σ δ] 2 = σ4 β V ar[ β] + Σ + β2 (µ δ4 σδ 4) + 2σ2 2 β 2 β β 2 Cov[s xy, β] ( Cov[s xx, β] Cov[s xy, β] β V ar[ σ 2 ε] = β 2 σ 4 V ar[ β] + 2βσ 2 (βcov[s xy, β] Cov[s yy, β]) + β2 Σ + (µ ε4 σ 4 ε) Derivatio of V ar[ α] { α } 2V { α } 2V { α } 2V V ar[ α] = V ar[ȳ β x] = ar[ȳ]+ ar[ ȳ β β]+ ar[ x] x { α }{ α } + 2 ȳ β Cov[ȳ, β] { α }{ α } + 2 x β Cov[ x, β] { α }{ α } + 2 Cov[ x, ȳ] x ȳ ) = µ 2 V ar[ β] + β2 σ 2 δ + σ2 ε + 2µ(βCov[ x, β] Cov[ȳ, β]) A similar shortcut formula was provided i the paper by Hood et al [11]. As outlied i Gillard [8], they assumed that (ξ, δ, ε) follow a trivariate Normal distributio. They the used the theory of maximum likelihood to obtai the iformatio matrices required for the asymptotic variace covariace matrices for the parameters of the model. Applyig various algebraic maipulatios to V ar[ α] they showed that V ar[ α] = µ 2 V ar[ β] + β2 σ 2 δ + σ2 ε The shortcut formula derived above is a geeralisatio of that derived by Hood et al. [11] to cope with o-normal ξ. Ideed, if (ξ, δ, ε) do follow a trivariate Normal distributio, the as β is a fuctio oly of secod order momets (or higher), β is statistically idepedet of the first order sample momets. As a result Cov[ x, β] = Cov[ȳ, β] = 0 15

17 ad the shortcut formula derived above collapses to that suggested by Hood et al. [11] 5.2 Shortcut Formulae for Covariaces Now, the shortcut formulae for the covariaces of µ with the remaiig parameters will be provided. Cov[ µ, α] = βσ2 δ µcov[ x, β] Cov[ µ, β] = Cov[ x, β] Cov[ µ, σ 2 ] = µ ξ3 σ2 Cov[ x, β] β Cov[ µ, σ δ] 2 = µ δ3 + σ2 Cov[ x, β] β Cov[ µ, σ ε] 2 = βσ 2 Cov[ x, β] The shortcut formulae for the covariaces of α with all other parameters are listed here. Cov[ α, β] = Cov[ȳ, β] βcov[ x, β] µv ar[ β] ( ) Cov[ α, σ 2 µσ 2 ] = β V ar[ β] + σ 2 Cov[ x, β] Cov[ȳ, β] µ β β Cov[s xy, β] Cov[ α, σ δ] 2 µ = β Cov[s xy, β] βµ δ3 µcov[s xx, β] µσ2 V ar[ β] β ( ) σ 2 Cov[ x, β] Cov[ α, σ 2 ε] = Cov[ȳ, β] β µ ε3 + βµσ2 V ar[ β] + βσ 2 (βcov[ x, β] Cov[ȳ, β]) +µ(βcov[s xy, β] Cov[s yy, β]) The shortcut formulae for the covariaces of β with the remaiig parameters are listed here. Cov[ β, σ 2 ] = 1 β Cov[s xy, β] σ2 V ar[ β] β Cov[ β, σ δ] 2 = Cov[s xx, β] Cov[s xy, β] + σ2 V ar[ β] β β Cov[ β, σ ε] 2 = Cov[s yy, β] βcov[s xy, β] βσ 2 V ar[ β] 16

18 The shortcut formulae for the covariaces of σ 2 with the remaiig parameters are listed here. ( ) Cov[ σ 2, σ δ] 2 = σ4 β V ar[ β] + σ2 2 2 β β Cov[s xy, β] Cov[s xx, β] + Σ 2σ2 σε 2 2σδ 2σ2 ε β 2 Cov[ σ 2, σ ε] 2 = σ 4 V ar[ β] σ2 β Cov[s yy, β] + Σ 2β2 σ 2 σδ 2 2σ2 δ σ2 ε Fially, the covariace betwee the error variace estimates is Cov[ σ δ, 2 σ ε] 2 = σ 4 V ar[ β] + σ2 β Cov[s yy, β] βσ 2 Cov[s xx, β] + Σ 2β2 σ 2 σδ 2 2σ2 σε 2. Agai, a example derivatio is provided. Derivatio of Cov[ β, σ 2 ] We have that σ 2 = s xy β. A first order Taylor expasio of σ 2 aroud the expected values of s xy ad β is σ 2 = σ 2 + (s xy βσ 2 ) 1 β ( β β) σ2 β. Hece, Cov[ β, σ 2 ] = E[( β β)( σ 2 σ 2 )] = 1 β Cov[s xy, β] σ2 V ar[ β]. β 5.3 Descriptio of Variace Covariace Matrices for Restricted Cases The complete asymptotic variace covariace matrices for the differet slope estimators uder varyig assumptios are icluded i the followig pages. For ease of use, the matrices are expressed as the sum of three compoets, A, B ad C. The matrix A aloe is eeded if the assumptios are made that ξ, δ ad ε all have zero third momets ad zero measures of kurtosis. These assumptios would be valid if all three of these variables are ormally distributed. The matrix B gives the additioal terms that are ecessary if ξ has o zero third momet ad a o zero measure of kurtosis. It ca be see that i most cases the B 17

19 matrices are sparce, eedig oly adjustmet for the terms for V ar[ σ 2 ] ad Cov[ µ, σ 2 ]. The exceptios are the cases where the reliability ratio is assumed kow, ad slope estimators ivolvig the higher momets. The C matrices are additioal terms that are eeded if the third momets ad measures of kurtosis are o zero for the error terms δ ad ε. It is likely that these C matrices will prove of less value to practitioers tha the A ad B matrices. It is quite possible that a practitioer would ot wish to assume that the distributio of the variable ξ is Normal, or eve that its third ad fourth momets behave like those of a Normal distributio. Ideed, the ecessity for this assumptio to be made i the likelihood approach may well have bee oe of the obstacles agaist a more widespread use of errors i variables methodology. The assumptio of Normal like distributios for the error terms, however, is more likely to be acceptable. Thus, i may applicatios, the C matrix may be igored. As a check o the method employed the A matrices were checked with those give by Hood [10] ad Hood et al. [11], where a differet likelihood approach was used i derivig the asymptotic variace covariace matrices. I all cases exact agreemet was foud, although some simplificatio of the algebra has bee foud to be possible. As discussed earlier, the limitatio of the likelihood approach is that it is limited to the case where all radom variables are assumed to be Normally distributed. The momets approach described by Gillard ad Iles [9] does ot have this limitatio. 18

20 Chapter 6 The Variace Covariace Matrices This sectio cotais the variace covariace matrices for each of the restricted case slope estimators outlied by Gillard ad Iles [9]. These are Itercept α kow Error variace σδ 2 kow Error variace σε 2 kow Reliability ratio κ = σ2 σ 2 +σ 2 δ kow Ratio of the error variaces λ = σ2 ε σ 2 δ kow Both variaces σ 2 δ ad σ2 ε kow For brevity, the otatio U = σ 2 + σ 2 δ, V = β2 σ 2 δ + σ2 ε, e 1 = µ δ4 3σ 4 δ, e 2 = µ ε4 3σ 4 ε ad e 3 = βλµ δ3 + µ ε3 shall be used. 19

21 6.1 Itercept α kow The method of momets estimator for the slope based o this assumptio is β = ȳ α x. Sice α is assumed to be kow, the variace covariace matrix for µ, β, σ 2, σ 2 δ ad σ2 ε is required. A 1 = 1 U βσ2 δ µ σ 2 σδ 2 σ2 σδ 2 µ µ β 2 σ 2 σ 2 δ µ V µ 2 σ2 βµ 2 V σ 2 βµ 2 V βσ2 µ 2 V Σ β 2 + σ4 β 2 µ 2 V + 2σ 4 Σ β 2 Σ β 2 σ4 β 2 µ 2 V 2σ 2 σ 2 δ + σ4 β 2 µ 2 V + 2σ 4 δ Σ + σ4 µ 2 V + 2σ 2 σ 2 ε Σ σ4 µ 2 V 2β 2 σ 2 σ 2 δ β 2 Σ + β2 σ 4 µ 2 V + 2σ 4 ε B 1 = µ ξ µ ξ4 3σ 4 0 2σ2 β 2 µ ξ3 µ C 1 = µ δ βµ δ3 µ ɛ3 µ 0 σ 2 µ µ δ3 σ2 βµ µ ε3 µ δ4 3σ 3 δ 2 σ2 µ µ δ3 βσ 2 µ (βµ δ3 + µ 2 µ ε3 ) µ ε4 3σ 3 ε 2 βσ2 µ µ ε3 20

22 6.2 Error variace σ 2 δ kow The method of momets estimator for the slope based o this assumptio is β = s xy. s xx σδ 2 Sice σ 2 δ is assumed kow, the variace covariace matrix for µ, α, β, σ 2 ad σ 2 ε is required. A 2 = 1 U βσ 2 δ µ 2 ( Σ + 2β 2 σ 4 σ 4 δ ) + V µ ( Σ + 2β 2 σ 4 σ 4 δ ) 2µβσδ 2 U σ 2 1 ( Σ + 2β 2 σ 4 σ 4 δ ) 2βσ2 δ 2µβσδ 2 V σ 2 U 2βσ2 σ 2 δ V σ 2 2U 2 2β 2 σ 4 δ 2V 2 B 2 = µ ξ µ ξ4 3σ C 2 = 1 0 βµµ δ3 σ 2 2β2 µµ δ3 σ 2 βµ δ3 σ 2 µ δ3 β 2 µ δ3 β 2 µ δ3 σ 2 βµ δ3 µ ε3 β 3 µ δ3 β 2 σ 4 e 1 β σ 2 e 1 β3 σ 2 e 1 e 1 β 2 e 1 e 2 + β 4 e 1 21

23 6.3 Error variace σ 2 ε kow The method of momets estimator for the slope based o this assumptio is β = s yy σ 2 ε s xy. Sice σ 2 ε is assumed kow, the variace covariace matrix for µ, α, β, σ 2 ad σ 2 δ is required. For brevity, the otatio W = (β 2 Σ + 2σ 2 ε) + V is itroduced. A 3 = 1 U βσ 2 δ µ 2 β 2 σ 4 W µ β 2 σ 4 W 2µ β 3 σ 2 (σ 2 εv + β 4 σ 2 σ 2 δ ) 1 W 2 (σ β 2 σ 4 β 3 σ εv 2 + β 4 σ 2 σ 2 2 δ ) 2µσ2 εv β 3 σ 2 2σ 2 εv β 3 σ 2 2 (β 4 U 2 + V 2 2β 4 σ 4 β 4 δ ) 2 (σ β εv 2 + 2β 2 σ 2 4 δ σ2 ε) 2V 2 β 4 B 3 = µ ξ µ ξ4 3σ C 3 = µ δ3 2µµ ε3 βσ 2 µ ε3 βσ 2 µ ε3 β 2 µ ε3 β 2 βµ δ3 1 e β 2 σ e β 3 σ 2 2 e β 3 σ e β e β (β 4 e β e 2 ) 22

24 6.4 Reliability ratio κ = σ2 σ 2 +σ 2 δ kow The method of momets estimator for the slope based o this assumptio is β = s xy κs xx. The variace covariace matrix for µ, α, β, σ 2 ad σ 2 δ is required. For brevity, the otatio ϖ = 1 κ is itroduced. A 4 = 1 U βσ 2 δ µ 2 Σ + V µ Σ 0 2µβ (1 κ) Σ σ 4 σ 4 σ 2 Σ 0 2β (1 κ) Σ σ 4 σ 2 2σ 4 2β 2 κσ 2 σ 2 δ 4β 2 (1 κ) Σ + 2σ 4 ε B 4 = 1 0 µ βϖ σ 2 µ ξ3 β 2 ϖ σ 2 µ ξ3 µ ξ3 β 2 ϖµ ξ β 2 ϖ 2 σ 4 (µ ξ4 3σ 4 ) βκϖ σ 2 (µ ξ4 3σ 4 ) β3 ϖ 2 σ 2 (µ ξ4 3σ 4 ) κ 2 (µ ξ4 3σ 4 ) β 2 κϖ(µ ξ4 3σ 4 ) β 4 ϖ 2 (µ ξ4 3σ 4 ) C 4 = 1 0 µ βκ σ 2 µ δ3 2µ β2 κ σ 2 µ δ3 βκ σ 2 µ δ3 κµ δ3 β 2 κµ δ3 β 2 κ σ 2 µ δ3 βκµ δ3 β 3 κµ δ3 + µ ε3 β 2 κ 2 σ 4 e 1 βκ2 σ 2 e 1 β3 κ 2 σ 2 e 1 β 2 κ 2 e 1 β 2 κ 2 e 1 β 4 κ 2 e 1 + e 2 23

25 6.5 Ratio of the error variaces λ = σ2 ε σ 2 δ kow The method of momets estimator for the slope based o this assumptio is β = (s yy λs xx ) + (s yy λs xx ) 2 + 4λ(s xy ) 2 2s xy. The variace covariace matrix for µ, α, β, σ 2 ad σ 2 δ is required. A 5 = 1 U βσ 2 δ µ 2 Σ σ 4 + V µ Σ σ 4 2µβ (β 2 +λ)σ 2 Σ 0 Σ σ 4 2β (β 2 +λ)σ 2 Σ 0 2σ Σ (β 2 +λ) 2σ2 δ σ2 ε (β 2 +λ) 2σ 4 δ B 5 = µ ξ µ ξ4 3σ C 5 = 1 0 µλβ (β 2 +λ)σ 2 µ δ3 λβ (β 2 +λ)σ 2 µ δ3 λ (β 2 +λ) µ δ3 2 µβ β e (β 2 +λ)σ 2 3 (β 2 +λ)σ 2 e 3 e 3 (β 2 +λ) β 2 (β 2 +λ) µ δ3 β (β 2 +λ) e 3 βµ δ3 β 2 e 2 +λ 2 β 2 e 1 (βe 2+λ 2 βe 1 ) βe 2 λβ 3 e 1 (β 2 +λ) 2 σ 4 (β 2 +λ) 2 σ 2 (β 2 +λ) 2 σ 2 e 2 +λ 2 e 1 (e 2+λβ 2 e 1 ) (β 2 +λ) 2 (β 2 +λ) 2 e 2 +β 4 e 1 (β 2 +λ) 2 24

26 6.6 Both variaces σ 2 δ ad σ2 ε kow The method of momets estimator for the slope based o this assumptio is s yy σε β = sg(s xy ) 2. s xx σδ 2 The variace covariace matrix for µ, α, β ad σ 2 is required. A 7 = 1 U βσ 2 δ Σ + (β2 σ 2 σ 4 δ σ2 ε) 2β 2 σ 4 βσ2 δ σ 2 (U + σ 2 ) µ ξ4 3σ 4 B 7 = µ ξ µ ξ4 3σ 4 C 7 = 1 0 µβµ δ3 2σ 2 βµ δ3 2σ 2 β2 µ σ 2 µ δ3 µ βσ 2 µ ε3 β 2 2σ 2 µ δ3 + µε3 2βσ 2 β 2 4σ 4 e 1 + e 3 β 2 σ 4 µ δ3 βµ δ3 β 2σ 2 e 1 e 1 25

27 Chapter 7 Variaces ad Covariaces for Higher Momet Estimators The methodology uderlyig the derivatio of the asymptotic variaces ad covariaces for estimators based o higher momets is idetical to that outlied previously. However, the algebraic expressios for the variaces ad covariaces of higher momet based estimators are loger ad more cumbersome tha those for the restricted parameter space. As a result, the full variace covariace matrices for higher momet estimators will ot be reported here. However, the expressios eeded to work out the full variace covariace matrices for the slope estimator based o third momets will be provided. These expressios ca the be substituted ito the shortcut formulae to derive the full variace covariace matrices. 7.1 Estimator based o Third Momets The estimator for the slope β based o the third order momets derived by Gillard ad Iles [9] is β 8 = s xyy s xxy. I order to use the shortcut equatios outlied earlier, the quatities Cov[ x, β 8 ], Cov[ȳ, β 8 ], Cov[s xx, β 8 ], Cov[s xy, β 8 ] ad Cov[s yy, β 8 ] are eeded. Further, to obtai these quatities, the covariaces betwee each of the first ad secod order momets ( x, ȳ, s xx, s xy, s yy ) ad the third order momets that occur i β 8 (s xxy, s xyy ) must be obtaied. Also, the variaces of these third order momets must be obtaied, as well as the covariace 26

28 betwee them. Usig the method illustrated i derivig V ar[s xx ] ad Cov[s xx, s xy ], the required covariaces betwee the secod order ad third order momets are: V ar[s xxy ] = V ar[s xyy ] = Cov[ x, s xxy ] = Cov[ x, s xyy ] = Cov[ȳ, s xxy ] = Cov[ȳ, s xyy ] = Cov[s xx, s xxy ] = Cov[s xy, s xxy ] = Cov[s yy, s xxy ] = Cov[s xx, s xyy ] = Cov[s xy, s xyy ] = Cov[s yy, s xyy ] = Cov[s xxy, s xyy ] = β 2 (µ ξ6 µ 2 ξ3 ) + 6β 2 µ ξ4 σ 2 δ + µ ξ4 σ 2 ε + 4β 2 µ ξ3 µ δ3 + β2 σ 2 µ δ4 + µ δ4 σ 2 ε + 6σ 2 σ 2 2 δ σ ε β 4 (µ ξ6 µ 2 ξ3 ) + 6β 2 µ ξ4 σ 2 ε + β 4 µ ξ4 σ 2 δ + 4βµ ξ3 µ ε3 + σ2 µ ε4 + σ 2 δ µ ε4 + 6β 2 σ 2 σ 2 2 δ σ ε β (µ ξ4 + 3σ 2 σ 2 δ ) β 2 µ ξ4 + σ 2 δ σ 2 ε + β 2 σ 2 σ 2 δ + σ 2 2 σ ε β 2 µ ξ4 + σ 2 δ σ 2 ε + β 2 σ 2 σ 2 δ + σ 2 2 σ ε β (β 2 µ ξ4 + 3σ 2 σ 2 ε ) β (µ ξ5 σ 2 µ ξ3 ) + 5βµ ξ3 σ 2 δ + 4βσ 2 µ δ3 β 2 (µ ξ5 σ 2 µ ξ3 ) + 3β 2 µ ξ3 σ 2 δ + σ 2 ε (µ ξ3 + µ δ3 ) + β 2 σ 2 µ δ3 β 3 (µ ξ5 σ 2 µ ξ3 ) + σ 2 µ ε3 + σ 2 δ µ ɛ3 + β 3 µ ξ3 σ 2 2 δ + 2βµ ξ3 σ ε β 2 (µ ξ5 σ 2 µ ξ3 ) + 2β 2 µ ξ3 σ 2 δ + β 2 σ 2 µ δ3 + µ ξ3 σ 2 2 ε + µ δ3 σ ε β 3 (µ ξ5 σ 2 µ ξ3 ) + 3βµ ξ3 σ 2 ε + σ 2 µ ε3 + σ 2 δ µ ε3 + β 3 2 µ ξ3 σ δ β 4 (µ ξ5 σ 2 µ ξ3 ) + 5β 2 µ ξ3 σ 2 ε + 4βσ 2 µ ε3 β 3 µ ξ6 β 3 µ 2 ξ3 + 3βµ ξ4 σ 2 ε + 3β 3 2 µ ξ4 σ δ + µ ξ3µ ε3 + β 3 µ ξ3 µ δ3 + 9βσ 2 σ δ 2 σ ε 2 + µ δ3 µ ε3 By usig the methodology outlied earlier, we ca ow obtai the variace of our slope estimator β 8, ad the covariaces of our slope estimator with the first ad secod order 27

29 momets. V ar[ β 8 ] = Cov[ x, β 8 ] = Cov[ȳ, β 8 ] = Cov[s xx, β 8 ] = β 2 µ ξ4 σ ε 2 + β 4 µ ξ4 σ δ 2 + 2βµ ξ3 µ ε3 + σ 2 µ ε4 + σ δ 2 µ ε4 6β 2 σ 2 σ δ 2 σ ε 2 β 2 µ ξ32 + 2β4 µ ξ3 µ δ3 + β 4 σ 2 µ δ4 + β 2 µ δ4 σ ε 2 2βµ δ3 µ ε3 β 2 µ 2 ξ3 σ 2 δ σ 2 ε 2β 2 σ 2 σ 2 δ + σ 2 2 σ ε βµ ξ3 2σ 2 σ 2 ε σ 2 δ σ 2 ε β 2 σ 2 2 σ δ µ ξ3 3β 2 µ ξ3 σ 2 δ 3β 2 σ 2 µ δ3 + µ ξ3 σ 2 2 ε + µ δ3 σ ε βµ ξ3 Cov[s xy, β 8 ] = Cov[s yy, β 8 ] = 2βµ ξ3 σ 2 ε + σ 2 µ ε3 + σ 2 δ µ ε3 2β 3 µ ξ3 σ 2 δ βµ δ3 σ 2 ε β 3 σ 2 µ δ3 βµ ξ3 3βµ ξ3 σ 2 ε + 3σ 2 µ ε3 σ 2 δ µ ε3 β 3 2 µ ξ3 σ δ µ ξ3 We ow have each of the compoets eeded to use the shortcut formulae to obtai the followig variace covariace matrix for the parameters µ, α, β, σ 2, σ 2 δ ad σ2 ε whe the estimator β 8 is used. 7.2 Estimator based o Fourth Momets The estimator for the slope β based o fourth order momets derived earlier is s xyyy 3s xy s yy β 9 = s xxxy 3s xx s xy I order to use the shortcut equatios, the quatities Cov[ x, β 9 ], Cov[ȳ, β 9 ], Cov[s xx, β 9 ], Cov[s xy, β 9 ] ad Cov[s yy, β 9 ] are eeded. Further, to obtai these quatities, the covariaces betwee each of the first ad secod order momets ( x, ȳ, s xx, s xy, s yy ) ad the fourth order momets that occur i β 9 (s xyyy, s xxxy ) must be obtaied. Also, the variaces of these fourth order momets must be obtaied, as well as the covariace betwee them. The formulae for these quatities are very legthy ad full details will be give i Gillard [?]. However, there is a potetial difficulty. As ca be see from the shortcut formulae, a key compoet of the variace covariace matrices is V ar[ β]. For the estimator of the slope based o fourth momets, V ar[ β] 28

30 depeds o the sixth momet of ξ. High order momets may be difficult to estimate reliably, so the authors believe further work is eeded to establish whether this formula is of practical value. Agai, full details will be give by Gillard [8]. 29

31 Bibliography [1] K. O. Bowma ad L. R. Sheto. Method of Momets. Ecyclopedia of Statistical Scieces (Volume 5). Joh Wiley & Sos, Ic, Caada, [2] G. Casella ad R. L. Berger. Statistical Iferece. Duxbury Press, Califoria, first editio, [3] C-L. Cheg ad J. W. Va Ness. Statistical Regressio with Measuremet Error. Kedall s Library of Statistics 6. Arold, Lodo, [4] H. Cramer. Mathematical Methods of Statistics. Number 9 i Priceto Mathematical Series. Priceto Uiversity Press, Priceto, [5] M. H. DeGroot. Probability ad Statistics. Addiso-Wesley, Secod editio, [6] N. R. Draper ad H. Smith. Applied Regressio Aalysis. Wiley-Itersciece, Caada, Third editio, [7] W. A. Fuller. Measuremet error models. Wiley Series i Probability ad Mathematical Statistics: Probability ad Mathematical Statistics. Joh Wiley & Sos Ic., New York, [8] J. W. Gillard. Errors i variables regressio: What is the appropriate model? PhD thesis, Cardiff Uiversity, [9] J. W. Gillard ad T. C. Iles. Method of momets estimatio i liear regressio with errors i both variables. Cardiff Uiversity School of Mathematics Techical Paper, [10] K. Hood. Some Statistical Aspects of Method Compariso Studies. Ph.d Thesis, Cardiff Uiversity, [11] K. Hood, A. B. J. Nix, ad T. C. Iles. Asymptotic iformatio ad variacecovariace matrices for the liear structural model. The Statisticia, 48(4): , [12] M. G. Kedall. Regressio, structure ad fuctioal relatioship. I. Biometrika, 38:11 25, [13] M. G. Kedall. Regressio, structure ad fuctioal relatioship. II. Biometrika, 39:96 108,

32 [14] M. G. Kedall ad A. Stuart. The Advaced Theory of Statistics Volume Two. Charles Griffi ad Co Ltd, Lodo, Third editio, [15] P. Spret. Models i Regressio ad Related Topics. Methue s Statistical Moographs. Matheu & Co Ltd, Lodo, [16] K. Va Motfort. Estimatig i Structural Models with No-Normal Distributed Varibles: Some Alterative Approaches. DSWO Press, Leide,