Name: ID: Homework for /7 Due /5. [ 8-3] I Example D of Sectio 8.4, the pdf of the populatio distributio is + αx x f(x α) =, α, otherwise ad the method of momets estimate was foud to be ˆα = 3X (where X is the sample mea of the radom sample X,..., X ). I this problem, you will cosider the samplig distributio of ˆα. (a) Show that the estimate ˆα is ubiased. (b) Fid Var[ ˆα]. [Hit: What is Var[X]?] (b) Use the cetral limit theorem to deduce a ormal approximatio to the samplig distributio of ˆα. Accordig to this approximatio, if = 5 ad α =, what is the P( ˆα >.5)? (a) We otice that Therefore, (b) First, we have ad Thus, E [ X ] = E[X ] = ( x = 4 + αx3 6 x + αx dx = ) = α 3. E[ˆα] = E [ 3X ] = 3E [ X ] = 3 α 3 = α. E[X ] = x + αx dx = ( ) x 3 = 6 + αx4 = 8 3, ( x ( ) x + αx + αx3 Var[X ] = E[X ] E[X ] = ( α ) 3 3 α =. 3 9 Var [ X ] = Var [ ] [ X i = ] Var X i = Var [X i ] = Var[X ] = 3 α 9. )
Therefore, we have Var[ˆα] = Var [ 3X ] = 9Var [ X ] = 3 α. ( α (c) Accordig to the cetral limit theorem, we have X N 3, 3 ) α, 9 approximately. Therefore, ˆα = 3X implies that ˆα N (α, 3 ) α, approximately. I the case α = ad = 5, we have ˆα N(,.), approximately. Thus P( ˆα >.5) = P(ˆα >.5) + P(ˆα <.5) ( ˆα = P >.5 ) ( ˆα + P <.5 ).... P(Z >.44) + P(Z <.44) =.749 +.749 =.498.. [ 8-53] Let X,..., X be i.i.d. uiform o [, θ]. (a) Fid the method of momets estimate of θ, ad the mea, variace, bias, ad MSE of the MME. (b) The mle of θ is ˆθ = max X i. The pdf of max i X i (How do we i fid this?) is x f(x θ) = θ < x < θ. otherwise Calculate the mea ad variace of the mle. Compare the variace, the bias, ad the mea squared error to those of the method of momets estimate. (c) Fid a modificatio of the mle that reders it ubiased.
(a) Sice we have ad the MME for θ is where X = ad Thus µ = E[X ] = θ x θ x dx = θ θ = θ, θ = µ, θ = X, X i. Furthermore, we have µ = E[X ] = θ x x3 dx = θ 3θ Var[X ] = µ µ = θ 3 θ = θ 3, ( ) θ = θ. E [ X ] = E[X ] = θ, ad Var [ X ] = Var[X ] = θ. It follows that ad E[ θ] = E [ X ] = E [ X ] = θ, Var[ θ] = Var [ X ] = 4Var [ X ] = θ 3. I particular, the MME θ is ubiased. The bias ad MSE of θ are ad (b) The mea of ˆθ is b( θ) =, MSE( θ) = Var[ θ] + b( θ) = Var[ θ] = θ 3. Sice E[ˆθ] = θ ( ) x x x + θ θ dx = ( + )θ = + θ. E[ˆθ ] = θ ( ) x x x + θ θ dx = ( + )θ = + θ, 3
the variace of ˆθ is Var[ˆθ] = E[ˆθ ] E[ˆθ] = The bias of ˆθ is ad the MSE of ˆθ is MSE(ˆθ) = Var[ˆθ] + b(ˆθ) = = ( ) θ θ + θ = + ( + ) ( + ). b(ˆθ) = E[ˆθ] θ = θ +, θ ( + )( + ). θ ( + ) ( + ) + θ ( + ) By compariso, although the MLE ˆθ is biased while the MME θ is ubiased, we see that MSE(ˆθ) < MSE( θ) whe is large. I fact, MSE(ˆθ) decreases much faster tha MSE( θ). (c) Let it follows that Thus θ is ubiased. θ = + ˆθ = + max X i, i E [ θ] [ ] + = E ˆθ = + ] [ˆθ E = θ. 4
3. [ 8-57] This problem is cocered with the estimatio of the variace of a ormal distributio with ukow mea from a sample X,..., X of i.i.d. ormal radom variables N(µ, σ ). I aswerig the followig questios, use the fact that (from Theorem B of Sectio 6.3) ( )s χ σ ad that the mea ad variace of a chi-square radom variable with r df are r ad r, respectively. (a) Which of the followig estimates is ubiased? s = (X i X) ad ˆσ = (X i X) (We discussed this i class. However, we do ot assume ormality. Whe the distributio is ot ormal, the argumet is much more complicated, as see i class. A techical detail is provided at the ed of this homework.) (b) Which of the estimates give i part (a) has the smaller MSE? (c) For what value of ρ does ρ (X i X) have the miimal MSE (as a estimate for σ )? (a) Sice ad ( )s χ, σ E[U] = r ad Var[U] = r, where U χ r, we have [ ] [ ] ( )s ( )s E = ad Var = ( ). Therefore, ad σ E [ s ] = ( ) σ σ ( ) = σ, Var [ s ] ( ) σ = ( ) = σ4 ( ). Furthermore, sice ˆσ = s, we have E[ˆσ ] = E[s ] = σ, 5
ad ( ) Var[ˆσ ] = Var[s ] = ( ) σ 4. Thus s is a ubiased estimate for σ while ˆσ is ot. (b) The bias of the two estimates are b(s ) = ad b(ˆσ ) = E[ˆσ ] σ = σ σ = σ, respectively. Thus the MSE of the two estimates are ad MSE(s ) = Var[s ] + b(s ) = MSE(ˆσ ) = Var[ˆσ ] + b(ˆσ ) = = σ 4, respectively. Sice σ4 + = σ4, ( ) σ 4 + ( ) σ 3 > ( )( ) < <, we have MSE(ˆσ ) < MSE(s ). (c) Let Y := ρ (X i X). The Y = ρ( )s. By a similar argumet as i (a), we have ad E[Y ] = ρ( )E[s ] = ρ( )σ, Var[Y ] = (ρ( )) Var[s ] = ρ ( ) σ4 = ρ ( )σ 4. Thus MSE(Y ) = Var[Y ] + b(y ) = ρ ( )σ 4 + ( ρ( )σ σ ) Sice = σ 4 [ρ ( ) + (ρ ρ ) ] f(ρ). f (ρ) = σ 4 [4ρ( ) + (ρ ρ )( )] = σ 4 [4ρ + (ρ ρ )]( ) = ( )σ 4 (ρ + ρ ), ad f (ρ) = ( )( + )σ 4 >, we see that f(ρ) achieve its miimum at ρ = +. (f ( ( + ) ) =.) 6
Name: ID: Homework for /9 Due /5. [ 8-7] Suppose that X follows a geometric distributio, P(X = k) = p( p) k ad assume X,..., X is a i.i.d. sample of size. Fid the asymptotic variace of the mle. (The momets of geometric distributio ca be foud i P7.) We have log f(x p) = log p + (X ) log( p), p log f(x p) = p X p, ad p log f(x p) = p X ( p). Therefore, the Fisher iformatio is [ ] I(p) = E log f(x p) = p = ( ) p + ( p) p = ad the asymptotic variace of the mle is I(p) ( ) p (E[X] ) ( p) ( p)p, ( p)p =.. [ 8-6] Cosider a i.i.d. sample of radom variables with desity fuctio f(x σ) = ( σ exp x ), < x <, σ >. σ Fid the asymptotic variace of the mle.
We have log f(x σ) = log log σ X σ, σ log f(x σ) = σ + X σ, E[ X ] = = σ = σ. x σ exp x σ exp ( x σ Therefore, the Fisher iformatio is [ ] I(σ) = E log f(x σ) σ ad the asymptotic variace of the mle is σ log f(x σ) = σ X σ 3, ad ( x ) x dx = ( σ σ exp x ) dx σ ) d x σ = σ ye y dy ( = σ ) σ 3 E[ X ] = σ, I(σ) = σ. 8
3. [ 8-47] The Pareto distributio has bee used i ecoomics as a model for a desity fuctio with a slowly decayig tail: f(x x, θ) = θx θ x θ, x x, θ >. Assume that x > is give ad that X, X,..., X is a i.i.d. sample. (a) Fid the method of momets estimate of θ. (b) Fid the mle of θ. (c) Fid the asymptotic variace of the mle. (a) Sice we have µ = E[X ] = x θx θ x θ dx = θx θ x = θx θ x θ θ = θx x θ, ad the MME is where X = X i. (b) The log likelihood fuctio is Thus log f(θ) = θ = µ µ x, θ = X X x, x (log θ + θ log x (θ + ) log X i ) = log θ + θ log x (θ + ) l (θ) = θ + log x l (θ) = θ <. log X i, x θ dx log X i. ad Sice l =, log X i log x 9
the mle of θ is (c) We have ˆθ =. log X i log x log f(x θ) = log θ + θ log x (θ + ) log X, θ log f(x θ) = θ + log x log X Therefore, the Fisher iformatio is [ ] I(θ) = E log f(x θ) θ ad ad the asymptotic variace of the mle is I(θ) = θ. θ log f(x θ) = θ. = ( θ ) = θ,