Solution Series 9. i=1 x i and i=1 x i.

Transcript

1 Lecturer: Prof. Dr. Mete SONER Coordinator: Yilin WANG Solution Series 9 Q1. Let α, β >, the p.d.f. of a beta distribution with parameters α and β is { Γ(α+β) Γ(α)Γ(β) f(x α, β) xα 1 (1 x) β 1 for < x < 1, otherwise. Suppose that X 1,, X n form a random sample from the Bernoulli distribution with parameter θ, which is unknown ( < θ < 1). Suppose also that the prior distribution of θ is the beta distribution with parameters α > and β >. Show that the posterior distribution of θ given that X i x i (i 1,..., n) is the beta distribution with parameters α + n i1 x i and β + n n i1 x i. In particular the family of beta distributions is a conjugate family of prior distributions for samples from a Bernoulli distribution. If the prior distribution of θ is a beta distribution, then the posterior distribution at each stage of sampling will also be a beta distribution, regardless of the observed values in the sample. First we calculate the joint p.f. of X 1,, X n, θ: f X1,,X n,θ(x 1,, x n, x) x y n y (1 x) Γ(α)Γ(β) xα 1 (1 x) β 1 Γ(α)Γ(β) xα+y 1 (1 x) β+n y 1 where y x x n. So that the marginal p.f. of X 1,, X n at (x 1,, x n ) is 1 Γ(α)Γ(β) xα+y 1 (1 x) β+n y 1 dx Thus the conditional p.d.f. of θ given x 1,, x n is f θ x1,,x n (x) Γ(α)Γ(β) Γ(α + y)γ(β + n y). Γ(α + β + n) Γ(α + β + n) Γ(α + y)γ(β + n y) xα+y 1 (1 x) β+n y 1, which is the Beta distribution with parameters α + y and β + n y. 1

2 Q2. Let ξ(θ) be defined as follows: for constants α > and β > : ξ(θ) { Γ(α) θ (α+1) e β/θ for θ >, for θ. A distribution with this p.d.f. is called an inverse gamma distribution. (a) Verify that ξ(θ) is actually a p.d.f.. (b) Consider the family of probability distributions that can be represented by a p.d.f. ξ(θ) having the given form for all possible pairs of constants α > and β >. Show that this family is a conjugate family of prior distributions for samples from a normal distribution with a known value of the mean µ and an unknown value of the variance θ. (a) The integral of ξ is βα Γ(α) 1 Γ(α) 1 Γ(α) θ (α+1) e β/θ dθ β y 2 β (α+1) y α+1 e y dy y α 1 e y dy (b) As we did before, let Θ be r.v. following the inverse gamma distribution with parameter α and β, let X be a random variable such that the conditional p.d.f. given Θ θ is The conditional p.d.f. of Θ given X x is f Θ x (θ) f X θ (x) 1 2πθ e (x µ)2 /2θ. ξ(θ)f X θ (x) ξ(θ)f X θ (x)dθ ξ(θ)f X θ (x) Γ(α) 2π Γ(α) 2π θ (α+1/2) 1 e (β+(x µ)2 /2)/θ θ (α+1/2) 1 e (β+(x µ)2 /2)/θ dθ Γ(α) Γ(α+1/2) 2π [β+(x µ) 2 /2] α+1/2 [β ] α Γ(α ) θ α 1 e β /θ. Where we set α α + 1/2 and β β + (x µ) 2 /2. The conditional distribution of Θ given X x is an inverse Gamma distribution with parameter α and β. Thus the family of inverse Gamma distribution is a family of prior distributions for samples from a normal distribution with a known mean µ and an unknown value of the variance. 2

3 Q3. Suppose that the number of defects in a 12-foot roll of magnetic recording tape has a Poisson distribution for which the value of the mean θ is unknown, and the prior distribution of θ is the gamma distribution with parameters α 3 and β 1. When five rolls of this tape are selected at random and inspected, the numbers of defects found on the rolls are 2, 2, 6, and 3. (a) What is the posterior distribution of θ? (b) If the squared error loss function is used, what is the Bayes estimate of θ? (a) Recall that X follows the Poisson distribution with mean θ then And θ has the p.d.f. θ θk P(X k) e k!. f θ (x) x3 1 e x Γ(3) The conditional p.d.f. of θ is obtained as before: x 2 e x /2. f θ 2,2,6,,3 (x) x2 e x /2 e 5x x /(2!2!6!!3!) s 2 e s /2 e 5s s /(2!2!6!!3!)ds e 6x x 15 e 6s s 15 ds e 6x 15 Γ(16) x : ξ(x 2, 2, 6,, 3) 6 16 Remark also that the posterior distribution of θ is the Gamma distribution with parameter (α 16, β 6). (b) The squared error loss function is L(θ, a) (θ a) 2. The Bayes estimator of θ is the a which minimizes E[L(θ, a) observation] L(x, a)ξ(x observation)dx, where ξ(θ observation) is the posterior p.d.f. of θ given the observation. We have computed the k-th moments of Gamma distribution X with parameter α and β in previous series, in particular E(X) α β, E(X2 ) α(α + 1) β 2. 3

4 Hence E[(θ a) 2 2, 2, 6,, 3] E[θ 2 2, 2, 6,, 3] 2aE[θ 2, 2, 6,, 3] + a a a2 ( a 8 ) The Bayes estimate for the observation 2, 2, 6,, 3 is 8/3. Q4. Let c > and consider the loss function L(θ, a) Assume that θ has a continuous distribution. (a) Let a q be two real numbers, show that { c θ a if θ < a, θ a if θ a. E[L(θ, a) L(θ, q)] (q a)[p(θ q) cp(θ q)], and E[L(θ, a) L(θ, q)] (q a)[p(a θ) cp(θ a)]. (b) Prove that a Bayes estimator of θ will be any 1/(1 + c) quantile of the posterior distribution of θ. Let θ follows a continuous distribution, (a) let a q, E[L(θ, a) L(θ, q)] E[(L(θ, a) L(θ, q))(1 θ a + 1 a θ q + 1 q θ )] Similarly we have E[c(a θ q + θ)1 θ a ] + E[(θ a c(q θ))1 a θ q ] + E[(θ a θ + q)1 q θ ] c(a q)p(θ a) + (1 + c)e(θ1 a θ q ) (a + cq)p(a θ q) + (q a)p(q θ) (q a)[p(q θ) cp(θ a)] + a(1 + c)p(a θ q) (a + cq)p(a θ q) (q a)[p(q θ) cp(θ q)]. E[L(θ, a) L(θ, q)] (q a)[p(a θ) cp(θ a)]. 4

5 (b) Let q be a 1/(1 + c)-quantile of θ. Then For all a q, by the first inequality, For all a q, by the second inequality, P(θ q) 1/(1 + c) and P(θ q) c/(1 + c). E(L(θ, a) L(θ, q)). E(L(θ, a) L(θ, q)) (q a)[p(q θ) cp(θ q)]. Thus q minimizes E[L(θ, a)] among all a R, is the Bayes estimator with loss function L. 5