Solutions: Homework 3

HTML
DOWNLOAD

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Solutions: Homework 3"

Καλλιγένεια Παπακωνσταντίνου
7 χρόνια πριν
Προβολές:

1 Solutios: Homework 3 Suppose that the radom variables Y,, Y satisfy Y i = βx i + ε i : i,, where x,, x R are fixed values ad ε,, ε Normal0, σ ) with σ R + kow Fid ˆβ = MLEβ) IND Solutio: Observe that Y i Normalβxi, σ ) : i,, With this iformatio, the log-likelihood fuctio relative to a observed sample y = y,, y ) R is lβ; y) = lπσ ) σ y i βx i ) The fuctio lβ; y) : β R is cotiuously differetiable i β I particular Notice that l β; y) σ x i y i βx i ) [ ] σ x i y i β : = l β; y) > 0 β < l β; y) < 0 β > x i y i x i y i x i Also lim lβ; y) = β ± This meas that lβ; y) : β R has a global maximum which depeds o y) at β MAX y) x i y i Fially, the MLE of β will be the fuctio β MAX : R R evaluated at the sample Y = Y,, Y ), ie ˆβ = β MAX Y ) x i Y i Fid the distributio of ˆβ Solutio: Observe that ˆβ ca be expressed as ˆβ = c i Y i : c i = x i The last relatio says that ˆβ is a liear combiatio of idepedet radom variables with Normal distributio The ˆβ Normalµ, λ ) where µ = E[ ˆβ] = c i E[Y i ] λ = Var[ ˆβ] = c i Var[Y i ] S xx x i βx i ) = β x i σ ) = σ

2 Fid the CRLB for estimatig β with ˆβ Solutio: Because Y,, Y are radom variables whose distributio belogs to the Expoetial Family CRLBβ) = E[l β; Y )] : where l β; y) = σ for ay observed sample y R This shows that l β; Y ) is costat as a fuctio of Y The E[l β; Y )] = σ CRLBβ) = σ Show that ˆβ = UMVUEβ) Solutio: We proved earlier that Show that ) ˆβ Normal β, σ From this it follows that ˆβ is a ubiased estimator for β Also ˆβ has a distributio that belogs to the Expoetial Family ad This proves that ˆβ = UMVUEβ) Var[ ˆβ] = σ = CRLBβ) : β R T is the UMVUE for σ R + if X,, X Normal0, σ ) Solutio: Give that the distributios for X,, X belog to the Expoetial Family CRLBσ ) = X i E[l σ ; X)] where X = X,, X ) The associated log-likelihood fuctio relative to a observed sample x = x,, x ) R is Notice that With this iformatio lσ ; x) = lπ) lσ ) σ l σ ; x) = σ ) σ ) 3 x i x i E[l σ ; X)] = σ ) 3 Var[X ] σ ) = σ ) CRLBσ ) = σ ) O the other had T = σ U : U = with W i = σ ) X i Because W,, W χ ), U χ ) The T has a distributio that belogs to the Expoetial Family Chage of Variables) ad W i E[T ] = σ E[U ] = σ

3 ie, a ubiased estimator for σ Also Var[T ] = σ ) Var[U ] = σ ) = CRLBσ ) : σ R + From this calculatios it follows that T = UMVUEσ ) Let X,, X Beroulliθ) with θ 0, ) Fid the Bayes Estimator ˆθB of θ with respect to the Uiform0, ) prior uder the Loss Fuctio Lt, θ) = t θ) : t R, θ 0, ) θ θ) Solutio: The first step is to deduce the posterior distributio for θ give a observed sample x = x,, x ) {0, } With the provided iformatio [ πθ x) θ xi θ) ]Iθ xi 0, )) = θ s θ) s Iθ 0, )) : s = x i Notice that this expressio is the fuctioal part of a Betaα, β ) desity with parameters α = s + ad β = s + The θ x Betaα, β ) The ext step is to calculate the posterior expected loss for our observed sample I this case E[Lt, θ) x] = Bα, β ) 0,) t θ) θ s θ) s dθ : t R Observe that fθ) = θ s θ) s Iθ 0, )) : θ R is proportioal to a Betaδ, γ ) desity with parameters δ = s ad γ = s Because of this observatio E[Lt, θ) x] = Bδ, γ ) Bα, β ) E[t θ ) x] with θ x Betaδ, γ ) As a cosequece of the above relatio, the value t MIN x) R which depeds o x) that miimizes E[Lt, θ) x] is the same that miimizes E[t θ ) x] This last problem has a explicit solutio Squared Error Loss) t MIN x) = E[θ x] = δ δ + γ s = x Fially, the Bayes Estimator will be the fuctio t MIN : {0, } R evaluated at the sample X = X,, X ), ie ˆθ B = t MIN X) = X Solutios: Homework 4 Let X,, X be a radom sample from a Normalθ, σ ) populatio σ kow) Cosider estimatig θ usig Squared Error Loss ad a Normalµ, τ ) prior distributio for θ Let δ π be the Bayes Estimator for θ Show that the posterior distributio of θ is Normalm, v ) with parameters m = τ x τ + σ + σ µ τ + σ v = τ σ τ + σ 3

4 Solutio: Our task it s reduce to idetify the fuctioal part of the posterior desity for θ give a observed sample x = x,, x ) R Igorig all terms that do t deped o θ [ πθ x) exp x i θ) )] θ ) µ) σ exp τ Iθ R) = exp [ x i θ) θ ]) µ) σ + τ Iθ R) Notice that exp [ x i θ) θ ]) µ) σ + τ exp [ σ + ) τ = exp [ τ + σ ) τ σ = exp [ v θ m ]) v θ exp θ m ) ) Fially which meas that θ x Normalm, v ) v v πθ x) exp θ m ) ) Iθ R) θ x σ + µ ) ]) τ θ τ θ x + σ µ ) τ σ Fid the Bayes Estimator ˆθ B of θ uder Squared Error Loss Solutio: I geeral, whe the posterior distributio admits at least secod-order momets, the Bayes Estimator uder Squared Error Loss is the Posterior Mea I our case, the Normal distributio has fiite momets for all orders The with δ π X) = E[θ X] = τ X τ + σ + σ µ τ + σ = a X + b a = τ τ + σ b = σ µ τ + σ Complete parts a)-c) of Problem 76 Solutio: For ay costats a R ad b R\{0}, the estimator δx) = ax +b has a Normalaθ + b, a σ ) distributio give θ ivariace uder liear trasformatios) The associated Risk Fuctio Squared Error Loss) is Rδ, θ) = E[δX) θ] θ) + Var[δX) θ] = aθ + b) θ) + a σ = θa ) + b) + a σ = b a)θ) + a σ I particular, for δ π X) = a X + b the coefficiets a R + ad b R satisfy the relatio b a )µ The Rδ π, θ) = b a )θ) + a σ a ) µ θ) + a σ = c µ θ) + c ) σ : c σ a = τ + σ ]) θ 4

5 Fially, the associated Bayes Risk for the Bayes Estimator is Bδ π, π) = E[Rδ π, θ)] = c E[µ θ) ] + c ) σ = c τ + c ) σ = σ ) τ τ + σ ) + τ ) σ τ + σ ) = τ σ τ + σ [ = τ σ τ + σ = τ c σ τ + σ + τ τ + σ ] Solutios: Homework 5 Cosider a radom sample of size from a distributio with discrete pdf fx p) = p p) x for x = 0,,, ad 0 otherwise Before proceedig otice that EX ) p)/p ad V arx ) p)/p The MLE of p is the value that solves the followig d log fx p) = 0 d log p p) i xi = 0 p i x ) i = 0 p) p) = p i x i p = i x i + Now sice d log fx p) = p i x i < 0, ˆp is ideed a maximum Further, the p) likelihood at p ad p = 0 is zero Thus, the MLE of p is ˆp = i x i + By the ivariace properties of MLEs, the MLE of θ p)/p is ˆθ ˆp = ˆp X 3 The CRLB associated with ubiased estimators of θ is [ ] dθ I p) Now dθ = d p)/p = /p ad I p) = E [ d ) log fx p)] [ d = E = E ] log fx p) [ p i x ] i p) = p + p p p) = p p) Thus the CRLB associated with ubiased estimators of θ is [ ] dθ I p) = [ /p ] p p) p p 5

6 4 Sice the variace of X attais the CRLB ie, V ar X) /)V arx ) p)/p )), the ˆθ is UMVUE for θ 5 First otice that ˆθ is ubiased for θ That is E X) p)/p The sice lim E[ˆθ θ) ] = lim V ar X) p = lim p = 0 ˆθ is MSEC 6 From the asymptotic results of MLEs we have ˆp p) d N0, /I p)) ˆp p) d N0, p p)) The from the Delta Method results we have ˆθ θ) d N0, dθ/) /I p)) ˆθ θ) d N0, /p 4 )p p)) ˆθ θ) d N0, p)/p ) Thus, we ca say somewhat iformally) that ˆθ Nθ, p)/p )) 7 First the risk fuctio associated with ˆθ = X Notice that i what follows that the risk fuctios are fuctios of p If so desired, they could be made fuctios of θ) [ ] Rˆθp) ˆθ θ) = E[Lˆθ, θ)] = E θ + θ [ θ + θ E ˆθ θ) ] θ V ar X) sice + θ p p = θ + θ /p)θ θ + θ /p) θ + ˆθ is ubiased for θ p)/p 6

7 Now for the risk fuctio associated with θ = X/ + ) [ ] R θp) θ θ) = E[L θ, θ)] = E θ + θ [ θ + θ E θ θ) ] θ + θ [V ar X/ + )) + bias X/ + )) ] [ ) p θ + θ + p + p + p p ) ] p [ ) ) ] θ + θ + p θ + θ + [ ) ) ] θ + + p + θ + + p + ) Rp) R θ ~ p) R θ^p) p 7

Σχετικά έγγραφα

Homework for 1/27 Due 2/5

Homework for 1/27 Due 2/5 Name: ID: Homework for /7 Due /5. [ 8-3] I Example D of Sectio 8.4, the pdf of the populatio distributio is + αx x f(x α) =, α, otherwise ad the method of momets estimate was foud to be ˆα = 3X (where