Stat 50 Homework Solutions Spring 005. (a λ λ λ 44 (b trace( λ + λ + λ 0 (c V (e x e e λ e e λ e (λ e by definition, the eigenvector e has the properties e λ e and e e. (d λ e e + λ e e + λ e e 8 6 4 4 6 (e ( ( ( λ e e + λ e e + 5/6 /6 /6 /6 /6 7/6 /6 7/6 /6 (f / λ e e + λ e e λ e e + λ e e 0.684 0.099 0.099 0.099 0.578 0.89 0.099 0.89 0.578 (g Let µ (µ,µ,µ denote the mean vector for X. Then E(Y e e e µ µ µ 6 6,V(Y 0 0 0 6 0 0 0 (h y y (Γ X (Γ X X (Γ Γ X X ΓΓ X X X because Γ is an orthogonal matrix, that is, ΓΓ Γ ΓI.
(i i. Yes, use result A.9.(e on page 97 of the text and I. Then, y Γ X Γ Γ X Γ Γ Γ X Γ Γ X I X. This makes intuitive sense because y and X are proportional to the square of the volume of the same ellipse. Rotations do not change volume. ii. Yes, use result A..(c on page 98 of the text. Then trace( y trace(γ X Γ trace(γγ X trace(i X trace( X. iii. No, Y is a diagonal matrix for any X regardless of whether or not X is a diagonal matrix. iv. Yes, note that Γ Y Γ is the spectral decomposition of X. Then, X Γ Y Γ, X Γ Y Γ and Γ X ΓΓ (Γ Y Γ Γ (Γ Γ Y (Γ Γ I Y I Y. Consequently, (Y µ Y Y (Y µ (Γ X Γ µ Γ X X X Γ(Γ Γ µ X Y Γ (X µ Γ X X (X ΓΓ µ X (X µ (Γ Γ X X (X ΓΓ µ X (X µ (ΓΓ X X (X (ΓΓ µ X (X µ X X (X µ X v. Yes, (Y µ Y (y µ Y (Γ X Γ µ X (Γ X Γ µ X Γ (X µ X Γ (X µ X (X µ X (Γ Γ (X µ X (X µ X (X µ X because ΓΓ I Hence, both Euclidean distance and Mahalanobis distance are invariant to rotations.. (a f(x,x exp π / (X µ (X µ X X 6, ( 9 6 f(x,x π 6 exp ( X 5 ( 6 ( 9 5 ( X
π exp 5 8 ( x 5 + 5 ( x 5 ( x +( x (b λ 7+ 0.6056, λ 7.944, e 0.479 0.887 0.887,e 0.479 (c 6, trace( 4 (d The ellipse given by 9(x +6(x (x + 5(x χ (.50 6.86 (e χ(0.5 λ.8 χ(0.5 λ.7 (f area πχ (.50 / (π(.9(6 7.95 (g area πχ (.05 / (π(5.99(6.9. µ 4, 5/6 / / /4 4. (a (i,(v display independent pairs of random variables, you simply have to check if the covariance is zero in each case.
(b X ( N X 4 0, 0 5 (c A bivariate normal distribution with mean vector ( + 0 X (X + 4 and covariance matrix 4 0 0 5 0 (d ρ 0andρ 0 (e Since X are independent to X x,x N(, 5 0 7/ 0 0 5 X X, 5. (a Z N(6, 77 (b Z Z 5 0 0 X N ( 6 5 77 59, 59 89 (c A bivariate normal distribution with mean vector 4 4 + ( 0 9 and covariance matrix 4 4 (d ρ 4 4 4 5/ 7/ 7/ 5 7 0.74 9 4/9 4/9 4/9 4/9 4/9 7/9 + /9 /9 /9 /9 /9 /9 6. (a cov(x Y,X + Y cov(x,x +cov(x,y cov(x,y cov(y,y 4 4 4 0 (b No, because cov(x Y,X + Y is not a matrix of zeros. 4
(c Start with the fact that X and Y are independent. Then from result 4.5(c on page 60 of the text, 8 0 0 X N 6 Y, 4 0 0 0 0 4 4 0 0 4 then from result 4. on page 57 of the text X Y I I X X + Y I I Y 0 4 4 N 5, 0 8 4 0 4 4 0 0 4 0 0 8 7. (a d(p for any A. d(p(p A(p / > 0 for any (p ifais positive definite and it always true that d(p,p0. Nowweonlyhavetoconsiderproperty (iv. It easy to show that (iv istruewhena is both positive definite and symmetric. Then, the spectral decomposition matrix A ΓΛΓ, exists, where Λ is a diagonal matrix containing the eigenvalues for A and the columns of Γ are the corresponding λ eigenvector. Then, A / Γ Γ is also symmetric and positive λp definite and A / A / A. Define d A / (r anda (p r A /. Use the Cauchy-Schwarz inequality (page 80 in the text to show that Then, (p r A(r (a d d(p (p A(p (a a (d d d(p d(r. (p r + r A(p r + r (p r A(p r +(r A(p r +(p r A(r +(r A(r d(p,r +d(r +(p A(r 5
d(p,r +d(r +d(p,r d(r d(p,r +d(r Hence, (iv is established for any positive definite symmetric matrix A. (b Show that (iv holds for any positive definite matrix A, note that d(p (p A(p is a scalar, so it is equal to its transpose. Hence, (p A(p (p A(p (p A (p Then, d(p (p A(p +(pq A(p (p (A + A (p satisfies (iv by the previous argument because (A + A is symmetric and positive definite. Consequently, A dose not have to be symmetric, but it must be positive definite for (p A(p to satisfy properties (i-(iv of a distance measure. If A was not positive definite, p 0 would exist for which (p A(p 0and this would violate condition (ii. (c Simply apply the triangle inequality to each component of (p, i.e., d(p max( p, p max( p r + r, p r + r max( p r + r, p r + r max( p r + max( r, r, p r + max( r, r max( p r, p r +max( r, r d(p,r +d(r 8. (a i. Iq q B Iq q B I q q B q r + B q r I r r 0 r q + I r r Iq q 0 q r I 0 r q I (q+r (q+r r r 6
ii. Ir r 0 r q Ir r 0 r q B B I r r 0 r q B q r I r r 0 r q +0 r q B q r I r r B q r B q r 0 r q + Ir r 0 r q 0 q r I (r+q (r+q (b The determinant of a block triangle matrix can be obtained by simply multiplying the determinants of the diagonal blocks. Consequently, B 0 I r r I r r and I r r 0 B I r r To formly prove the latter case, start with the martix with q and expand across the first row to compute the determinant. The determinant of each minor in the expansion is multiplied by zero, except for the determinant of the r r identity matrix which is multiplied by one. Cosequently, the determinant is one. Using an inductive proof, assume the result is true for any q and use a similar expansion acroos the first row to show that it is also true when the upper left block has dimesion q+. Iq q A (c A A A 0 q r A A A A I r r Iq q A A A A A A A A 0 q r 0 r q A + I r r A 0 r q A + I r r A A A I r r A A A A 0 0 q r A A A A I r r A A A A 0 q r 0 r q A (d from 8(c, A A A A 0 q r 0 r q A A A A A A A 0 q r A A I r r LHS A A A A A for A 0 RHS A A from 8(b. Hence, A A A A A for A 0 7
(e Take inverse on both sides of 8(c, then A A A A 0 q r 0 r q A then we easily get A Iq q A A ( Iq q A A Iq q A A A A A A A A A A 0 q r 0 r q A from result of 8(a, we know that Iq q A A I q q 0 q r A A I r r 0 q r Iq q A A A A I r r ( X 9. Define X µ µ ( X µ f X (x exp (π p/ / (X µ (X µ (π q/ (π r/ / A A A A A 0 q r A A I r r 0 q r A A I r r 0 q r A I r r exp / (X µ (X µ Expand the quadratic form (X µ (X µ (X µ ( (X µ (X µ (X µ ( (X µ (X µ 0 q r I r r Iq q (X µ (X µ ( 0 q r 0 r q (X µ (X µ (X µ ( 0 q r 0 r q 8
(X µ (X µ (X µ (X µ (X µ ( (X µ (X µ + (X µ (X µ B + B f X (x (π q/ (π r/ / ( (π q/ (π q/ / ( exp / (X µ (X µ exp (π r/ / (B + B exp / B ( exp (π r/ / B f X x (x f X (x If X and X are uncorrelated, then f X X x (x f X (x Therefore, f X x(x f X (x f X (x implyx and X are independent. The review of properties for partitioned matrices in problem 8 was done to give you the tools to factor the density for a multivariate normal distribution as a product of a marginal density and a conditional density. You may find some of these martix properties to be useful in future work. 9