Deming regression MethComp package May 2007 Anders Christian Jensen Steno Diabetes Center, Gentofte, Denmark acjs@steno.dk
Contents 1 Introduction 1 2 Deming regression 1 3 The likelihood function 1 4 Solving for ξ i 2 5 Solving for α 2 6 Solving for β 3 7 Solving for ξ i - again 6 8 Solving for σ 2 6 9 Summing up 7 10 The Deming function 8
Deming regression 1 1 Introduction This document is related to the Deming function in the package MethComp and contains the derivation of the maximum likelihood estimates related to the Deming regression model. It is based on the book Models in regression and related topics chapter three, from 1969 by Peter Sprent, but with more detailed calculations included. 2 Deming regression The mathematical model η = α + βξ describes a linear relationship between two variables ξ and η. Observations x and y of two variables are usually desribed by a regression of y on x where x is assumed to be observed without error or, equivantly using the conditional distribution of y given x. In linear regression with observations subject to additive random variation on both x and y and observed values for individuals x i, y i, i = 1,..., n, a model may be written x i = ξ i + e xi, y i = η i + e yi = α + βξ i + e yi, where e xi and e yi denotes the random part of the model. This is known as a functional relationship because the ξ i s are assumed to be fixed parameters, as oppposed to a structural relationship where some distribution for the ξ i s is assumed. In the following it is assumed that the e xi s are iid with e yi N0, σ 2, and that the e yi s are iid with e yi N0, λσ 2, for some λ > 0. Furthermore e xi is assumed to be independent of e yi. The aim of this document is to derive the maximum likelihood estimates for α, β, ξ i and σ 2 in the functional model stated above. 3 The likelihood function The likelihood function f x1,x 2,...,x n,y 1,y 2,...,y n α, β, ξ 1, ξ 2,..., ξ n, σ 2 denoted f is f = n 2πσ 2 1 2 exp x i ξ i 2 2πλσ 2 1 2 exp y i α βξ i 2 2σ 2 and the loglikelihood, denoted L, is L = 1 2 log 2πσ 2 x i ξ i 2 1 2σ 2 2 log 2πλσ 2 y i α βξ i 2 = n 2 log 4π 2 n 2 log λσ 4 n x i ξ i 2 2σ 2 y i α βξ i 2. It follows that the likelihood function is not bounded from above when σ 2 goes to 0, so in the following it is assumed that σ 2 > 0.
2 4 Solving for ξ i 4 Solving for ξ i Differentiation of L with respect to ξ i gives = x i ξ i 2 ξ i ξ i 2σ 2 = x i ξ i σ 2 + βy i α βξ i λσ 2. y i α βξ i 2 Setting ξ i equal to zero yields ξ i = 0 ξ i = λσ2 x i + βσ 2 y i βασ 2 λσ 2 + β 2 σ 2 = λx i + βy i α λ + β 2. 1 So to estimate ξ i, estimates for β and α are needed. Therefore focus is turned to the derivation of ˆα. 5 Solving for α Differentiation of L with respect to α gives α = y i α βξ i 2 α = y i α βξ i λσ 2, and putting α equal to zero yields α = 0 α = 1 n y i βξ i.
Deming regression 3 Now one can use 1 to dispense with ξ i α = 1 n = 1 n = 1 n y i βξ i y i β λx i + βy i α λ + β 2 y i β λx i + βy i + β2 α λ + β 2 λ + β 2 α 1 β2 λ + β 2 = 1 n = 1 n y i β λx i + βy i λ + β 2 y i 1 β2 βλ x λ + β 2 i λ + β 2 α = 1 n = 1 n βλ λ + β 2 y i x i λ + β 2 λ y i x i β = y xβ. Hence the estimate for α becomes ˆα = y x ˆβ. 6 Solving for β Differentiation of L with respect to β gives β = y i α βξ i 2 = β y i α βξ i ξ i λσ 2. Setting β equal to zero yields β = 0 y i α βξ i ξ i = 0,
4 6 Solving for β and using 1 0 = = y i α βξ i ξ i y i α β λx i + βy i α λxi + βy i α. λ + β 2 λ + β 2 This implies that 0 = = y i αλ + β 2 βλx i β 2 y i α λx i + βy i α λ 2 x i y i α + β 2 λx i y i α βλ 2 x 2 i β 2 λx i y i α + βλy i α 2 + β 3 λy i α 2 β 2 λx i y i α β 3 y i α 2 = β 2 λ x i y i βλ 2 x 2 i + λ 2 x i y i +β 2 λα x i + βλ y i α 2 λ 2 α x i. Dividing with λ and using the fact that α = y. x.β it is seen that 0 = β 2 x i y i βλ x 2 i + λ x i y i + β 2 y. x.β x i +β yi y. x.β 2 λy. x.β x i = β 2 x i y i βλ β 3 x.β x i + β x 2 i y 2 i + λ x i y i + β 2 y. x i + β y. x.β 2 2β y i y. x.β λy. x i + λx.β x i.
Deming regression 5 Splitting up the sums even more gives 0 = β 2 x i y i βλ +β y 2 i x 2 i + λ x i y i + β 2 y. x i β 3 x.β x i + β y. 2 + β x.β 2 2β y.x.β 2β +2β y i x.β λy. x i + λx.β x i. Finally the terms are sorted and collected according to powers of β: 0 = β 3 x. 2 x. x i Since +β 2 y. x i +β yi 2 λ x i y i 2 x 2 i + +λ x i y i y. x.2 x. x i = 0 y.x. + 2 y. 2 2 x i. y i x. y i y. + λx. y. x i x iy i 2 y.x. + 2 y ix. = SPD xy x i y2 i λ x2 i + y.2 2 y iy. + λx. x i = SSD y λssd x x iy i y. x i = SPD xy it is clear that the derivation of β comes down to solve For SPD xy 0 this implies that y i y. β 2 SPD xy + βssd y λssd x + λspd xy = 0. 2 β = SSD y λssd x ± SSD y λssd x 2 4SPD xy λspd xy 2SPD xy = SSD y λssd x ± SSD y λssd x 2 + 4λSPD 2 xy 2SPD xy.
6 7 Solving for ξ i - again Since SSD y λssd x SSD y λssd x 2 + 4λSPD 2 xy there is always a positive and a negative solution to 2. The desired solution should always have the same sign as SPD xy, hence the solution with the positive numerator is selected. Therefore ˆβ = SSD y λssd x + SSD y λssd x 2 + 4λSPD 2 xy 2SPD xy. 7 Solving for ξ i - again With estimates for β and α it is now possible to estimate ξ i using 1: ˆξ i = λx i + ˆβy i ˆα λ + ˆβ 2. 8 Solving for σ 2 Differentiation of L with respect to σ 2 gives and setting σ 2 = n σ 2 σ 2 2 logλσ4 x i xi i 2 2σ 2 = nσ2 + x i xi i 2 n + y i α βξ i 2 σ 4 2σ 4 2λσ 4 y i α βξ i 2 = 2λnσ2 + λ x i xi i 2 + y i α βξ i 2 2λσ 4, equal to zero yields σ 2 = 0 2λnσ2 + λ x i xi i 2 + y i α βξ i 2 = 0 σ 2 = λ x i ξ i 2 + y i α βξ i 2. 2λn To get a central estimate of σ 2 one must divide by n 2 instead of 2n since there are n + 2 parameters to be estimated, namely ξ 1, ξ 2,..., ξ n, α and β. Hence the degrees of freedom are 2n n + 2 = n 2. Therefore ˆσ 2 = λ x i ˆξ i 2 + y i ˆα ˆβ ˆξ i 2. 2λn 2
Deming regression 7 9 Summing up ˆα = y x ˆβ ˆβ = ˆσ = SSD y λssd x + SSD y λssd x 2 + 4λSPD 2 xy 2SPD xy λ x i ˆξ i 2 + y i ˆα ˆβ ˆξ i 2 2λn 2 ˆξ i = λx i + ˆβy i ˆα λ + ˆβ 2 These formula are implemented in the Deming function in the MethComp package.
8 10 The Deming function 10 The Deming function Deming <- function x, y, vr=sdr^2, sdr=sqrtvr, boot=false, keep.boot=false, alpha=0.05 { if missing vr & missing sdr var.ratio <- 1 else var.ratio <- vr vn <- c deparse substitute x, deparse substitute y pn <- c "Intercept", "Slope", paste "sigma", vn, sep="." alfa <- alpha dfr <- data.frame x=x, y=y dfr <- dfr[complete.casesdfr,] x <- dfr$x y <- dfr$y n <- nrow dfr SSDy <- var y *n-1 SSDx <- var x *n-1 SPDxy <- cov x, y *n-1 beta <- SSDy - var.ratio*ssdx + sqrt SSDy - var.ratio*ssdx ^2 + 4*var.ratio*SPDxy^2 / 2*SPDxy alpha <- mean y - mean x * beta ksi <- var.ratio*x + beta*y-alpha /var.ratio+beta^2 sigma.x <- var.ratio*sum x-ksi^2 + sum y-alpha-beta*ksi^2 / # The ML-estiamtes requires 2*n at this point bu we do not think we have that # many observations so we stick to n-2. Any corroboation from litterature? n-2*var.ratio sigma.y <- var.ratio*sigma.x sigma.x <- sqrt sigma.x sigma.y <- sqrt sigma.y if!boot { res <- c alpha, beta, sigma.x, sigma.y names res <- pn res else { if is.numeric boot N <- boot else N <- 1000 res <- matrix NA, N, 4 for i in 1:N { wh <- sample 1:n, n, replace=true res[i,] <- Deming x[wh], y[wh], vr=var.ratio, boot=false ests <- cbind calpha,beta,sigma.x, sigma.y, se <- sqrt diag cov res, t apply res, 2, quantile, probs=c0.5,alfa/2,1-alfa/2, na.rm=t colnames res <- rownames ests <- pn colnames ests <- c"estimate", "S.e.boot", colnamesests[3:5] ifkeep.boot { print ests invisible res else { cat vn[2], " = alpha + beta*", vn[1], "\n" ests