Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-57: Στατιστική Επεξεργασία Σήµατος 15 ιδάσκων : Α. Μουχτάρης Πρώτη Σειρά Ασκήσεων Λύσεις Ασκηση 1. 1. Since a and b are independent and have zero mean, we have E[a] = and E[b] = and E[ab] =. Therefore, E[x ] = E[(an+b) ] = E[a n +nab+b ] = n E[a ]+ne[ab]+e[b ] = n E[a ]+E[b ] E[x ] = n σ a+σ b. x(n) is not stationary since its variance E[x ] depends on the absolute time n. For this reason it is not ergodic since ergodicity depends on stationarity. Ασκηση. H(z) = 1 3.5z 1 + 6z + 4z 3 1 1 z 1 + 1 H(z) = z3 (1 3.5z 1 + 6z + 4z 3 ) 4 z z 3 (1 1 z 1 + 1 4 z ) H(z) = z3 3.5z + 6z + 4 z 3 1 z + 1 4 z H(z) = z3 3.5z + 6z + 4 z(z 1 z + 1 4 ) Poles: z(z 1 z + 1 4 ) = z 1 = and z 1 z + 1 4 = z =.3536 + j.3536, z 3 =.3536 j.3536 Zeros: z 3 3.5z + 6z + 4 = z 1 =.5, z = + j, z 3 = j. Zeros and poles can be found either using the matlab function roots() or solving algebraically the equations (for the zeros of a 3rd order linear equation see: http://en.wikipedia.org/wiki/cubic_function). So the pole-zero map for H(z) is shown in Fig. 1
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 1.5 1 Imaginary Part.5.5 1 1.5 1.5 1.5.5 1 1.5.5 3 Real Part Figure 1: Pole-zero map for H(z) Magnitude and Phase of H(z)(see Figs and 3) % matlab code for the magnitude and phase of H(z) B=[1, -3.5, 6, 4]; A=[1, -1/sqrt(), 1/4]; fr=-pi:pi/1:pi; plot(fr/pi, *log1(abs(freqz(b, A, fr))));%magnitude plot(fr/pi, angle(freqz(b, A, fr)));%phase
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 3 4 H(z) Magnitude (db) 18 16 14 1 1 1.8.6.4...4.6.8 1 normalized frequency Figure : Magnitude of H(z) 4 H(ω) 3 Phase (rads) 1 1 3 4 1.8.6.4...4.6.8 1 normalized frequency Figure 3: Phase of H(z)
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 4 H(z) as a cascade connection of a minimum phase and an all-pass system. H(z) = z3 3.5z + 6z + 4 z(z 1 z + 1 4 ) H(z) = (z +.5)(z j)(z + j) z(z.3536 j.3536)(z.3536 + j.3536) z +.5 (z j)(z + j) H(z) = z(z.3536 j.3536)(z.3536 + j.3536) 1 z +.5 (z j)(z + j) H(z) = z(z.3536 j.3536)(z.3536 + j.3536) ( 1 4 + j 1 4 )( 1 4 j 1 4 ) ( 1 4 + j 1 4 )(1 4 j 1 4 ) (z +.5)( 1 4 H(z) = + j 1 4 )( 1 4 j 1 4 ) (z j)(z + j) z(z.3536 j.3536)(z.3536 + j.3536) ( 1 4 + j 1 4 )( 1 4 j 1 4 ) where H min (z) = (z +.5)( 1 4 + j 1 4 )( 1 4 j 1 4 ) z(z.3536 j.3536)(z.3536 + j.3536) and H ap (z) = (z j)(z + j) ( 1 4 + j 1 4 )( 1 4 j 1 4 ) Pole-zero plot of H min (z) and H ap (z) (see Figs 4 and 5) % matlab code for Pole-zero plot of H_min(z) and H_ap(z) ZerosHmin=[-.5, 1/4+1j*1/4, 1/4-1j*1/4]; A=[1, -1/sqrt(), 1/4]; zplane(poly(zeroshmin), A); ZerosHap=[+1j*, -1j*]; PolesHap=[1/4+1j*1/4, 1/4-1j*1/4]; zplane(poly(zeroshap), poly(poleshap)); Magnitude and Phase of H min (z)(see Figs 6 and 7) % matlab code for the magnitude and phase of H_min(z) fr=-pi:pi/1:pi; plot(fr/pi, *log1(abs(freqz(poly(zeroshmin), A, fr)))); figure; plot(fr/pi, angle(freqz(poly(zeroshmin), A, fr)))
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 5 Pole zero map of H min 1.8.6.4 Imaginary Part...4.6.8 1 1.5.5 1 Real Part Figure 4: Pole-zero map of H min (z) Pole zero map of H ap 1.5 1.5 Imaginary Part.5 1 1.5 1.5 1.5.5 1 1.5.5 3 Real Part Figure 5: Pole-zero map of H ap (z)
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 6 6 H min (ω) 4 magnitude (db) 4 6 8 1.8.6.4...4.6.8 1 normalized frequency Figure 6: Magnitude of H min (z).8 H min (ω).6.4. Phase (rads)..4.6.8 1.8.6.4...4.6.8 1 normalized frequency Figure 7: Phase of H min (z)
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 7 18.618 H ap (ω) 18.618 18.618 18.618 magnitude (db) 18.618 18.618 18.618 18.618 18.618 18.618 1.8.6.4...4.6.8 1 normalized frequency Figure 8: Magnitude of H ap (z) Magnitude and Phase of H ap (z)(see Figs 8 and 9) % matlab code for the magnitude and phase of H_ap(z) fr=-pi:pi/1:pi; plot(fr/pi, *log1(abs(freqz(poly(zeroshap), poly(poleshap), fr)))); figure; plot(fr/pi, angle(freqz(poly(zeroshap), poly(poleshap), fr)));
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 8 4 H ap (ω) 3 1 Phase (rads) 1 3 4 1.8.6.4...4.6.8 1 normalized frequency Figure 9: Phase of H ap (z) Ασκηση 3. 1. This is an AR process since the input (driving noise) is the signal ɛ(n) and the output y(n) is a linear combination of past output values and the input.. (a) For the first realization: This is an AR system of order M = 1, so from the Yule-Walker equations with M = 1 we get r()w 1 = r (1) w 1 = r (1) r() w 1 =.3785 3.186 w 1 =.9644 ˆα =.9654. Also for the variance of the driving white noise we have (see also Eq..71 Haykin): 1 σɛ = a k r(k) = 1 r() ˆαr(1) = 3.186.9654.3785 σɛ = 1.5764 k= (b) For the second realization: This is an AR system of order M = 1, so from the Yule-Walker equations with M = 1 we get r()w 1 = r (1) w 1 = r (1) r() w 1 =.71 3.671 w 1 =.949 ˆα =.949. Also for the variance of the driving white noise we have (see also Eq..71 Haykin): 1 σɛ = a k r(k) = 1 r() ˆαr(1) = 3.671.949.71 σɛ =.715 k=
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 9 4 First Realization 3 Power Spectrum (db) 1 1 Estimated Power Spectrum True Power Spectrum Periodogram 3 4.5 1 1.5.5 3 3.5 ω (rad) Figure 1: Power spectra for the first realization 3. Ŝ yy (ω) = ˆσ ɛ 1 âe jω, with ˆα =.9654 and σ ɛ = 1.5764 for the first realization and ˆα =.949 and σ ɛ =.715 for the second realization. For the plot see below. 4. The true power spectrum, with {a, σ ɛ } = {.95, 1}. 5. The periodogram spectrum ˆΣ yy (ω) = 1 N S yy (ω) = σ ɛ 1 ae jω, N 1 n= y ne jωn We plot all three spectra for the first realization in Fig. 1 and for the second realization in Fig. 11 6. For the results please refer to part 5. %Matlab code for parts -6
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 1 3 Second Realization 1 Power Spectrum (db) 1 Estimated Power Spectrum True Power Spectrum Periodogram 3 4.5 1 1.5.5 3 3.5 ω (rad) Figure 11: Power spectra for the second realization y1=[3.848, 3.5, 5.55, 4.976, 6.599,6.17,6.57, 6.388, 6.5, 5.564,... 5.683, 5.55, 4.53, 3.95, 3.668,3.668, 3.6, 1.945,.4,.14].'; R1=xcorr(y1, 'biased'); a_hat1=r1(1)/r1(); sigma_square1=r1()-r1(1)/r1()*r1(1); y=[5.431, 5.55, 4.873, 5.1, 5.7, 5.86, 6.133, 5.68, 6.479, 4.31,... 5.181, 4.79, 5.469, 5.87, 3.819,.968,.751, 3.36, 3.13, 3.694].'; R=xcorr(y, 'biased'); a_hat=r(1)/r(); sigma_square=r()-r(1)/r()*r(1); omega=linspace(, pi, ); Spectrum1=sigma_square1./(abs(1-a_hat1*exp(-1i*omega)).ˆ); Spectrum1true=1./(abs(1-.95*exp(-1i*omega)).ˆ);
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 11 n=(:length(y1)-1).'; for iomeg=1:length(omega) Periodogram1(iomeg)=1/(length(y1))*abs(sum(y1.*exp(-1i*omega(iomeg)*n))).ˆ; end Spectrum=sigma_square./(abs(1-a_hat*exp(-1i*omega)).ˆ); Spectrumtrue=1./(abs(1-.95*exp(-1i*omega)).ˆ); for iomeg=1:length(omega) Periodogram(iomeg)=1/(length(y))*abs(sum(y.*exp(-1i*omega(iomeg)*n))).ˆ; end figurea(1,8,4); hold on; box('on'); plot(omega, 1*log1(Spectrum1)); plot(omega, 1*log1(Spectrum1true), 'r'); plot(omega, 1*log1(Periodogram), 'g'); title('first Realization'); xlabel('\omega (rad)'); ylabel('power Spectrum (db)'); legend({'estimated Power Spectrum'; 'True Power Spectrum'; 'Periodogram'}, 'Location', 'Best'); saveas(gcf, 'FirstRealization.fig','fig'); print(gcf,'-depsc',['firstrealization.eps']); hold off; close; figurea(,8,4); hold on; box('on'); plot(omega, 1*log1(Spectrum)); plot(omega, 1*log1(Spectrumtrue), 'r'); plot(omega, 1*log1(Periodogram), 'g');
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 1 3 N=56 Power Spectrum (db) 1 1 Estimated Power Spectrum True Power Spectrum Periodogram 3.5 1 1.5.5 3 3.5 ω (rad) Figure 1: Power spectra for N = 56 samples title('second Realization'); xlabel('\omega (rad)'); ylabel('power Spectrum (db)'); legend({'estimated Power Spectrum'; 'True Power Spectrum'; 'Periodogram'}, 'Location', 'Best'); saveas(gcf, 'SecondRealization.fig','fig'); print(gcf,'-depsc',['secondrealization.eps']); hold off; close; 7. For N = 56 ˆα =.934 and σ ɛ =.9146. For N = 14 ˆα =.959 and σ ɛ =.9499. The corresponding plots can be seen in Fig. 1 and Fig. 13. Ασκηση 4. 1. The filter output is
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 13 3 N=14 Power Spectrum (db) 1 1 Estimated Power Spectrum True Power Spectrum Periodogram 3.5 1 1.5.5 3 3.5 ω (rad) Figure 13: Power spectra for N = 14 samples x(n) = w H u(n), where u(n) is the tap-input vector. The average power of the filter output is therefore E[ x(n) ] = E[w H u(n)u H (n)w] = w H E[u(n)u H (n)]w = w H Rw.. If u(n) is extracted from a zero mean white noise of variance σ, we have R = σ I, where I is the identity matrix. Hence, E[ x(n) ] = σ w H w
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 14 1 u(n) N=3 5 5 v(n) N=3 1 1 3 n (samples) 4 1 3 n (samples) 4 u(n) N=56 1 1 v(n) N=56 5 1 15 5 n (samples) 4 5 1 15 5 n (samples) 4 4 u(n) N=48 v(n) N=48 4 5 1 15 n (samples) 4 5 1 15 n (samples) Figure 14: Driving noise and 4th order AR process for N = {3, 56, 48} samples Ασκηση 5. 1. For all N cases plots of the driving noise and the AR process are shown in Fig. 14. The sample autocorrelation is given by: ˆr u (k) = 1 N N k 1 n= u (n)u(n + k), k =, 1,,..., N 1 For the true autocorrelation sequence, r u (k), from he definition of the autocorrelation and the definition of the 4th order AR process we are given, it holds: r u (k) = E[u(n)u (n k)] = E[(a 1 u(n 1) + a u(n ) + a 3 u(n 3) + a 4 u(n 4) + v(n)) u(n k)] = a 1 E[u(n 1)u(n k)]+a E[u(n )u(n k)]+a 3 E[u(n 3)u(n k)]+a 4 E[u)n 4)u(n k)]+e[v(n)u(n k)] The term E[v(n)u(n k)] is zero for k >, since u(n k) involves samples of the white noise process up to time n k, that is u(n k) is uncorrelated to v(n), so r u (k) becomes:
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 15 r u (k) = a 1 E[u(n 1)u(nk)]+a E[u(n )u(n k)]+a 3 E[u(n 3)u(n k)]+a 4 E[u)n 4)u(n k)] r u (k) = a 1 r u (k 1) + a r u (k ) + a 3 r u (k 3) + a 4 r u (k 4). (1) Since the autocorrelation function is a recurrence relation we have to estimate the initial conditions, i.e., r u (k) for k =, 1,, 3, 4. The autocorrelation at lag k = is equal to : r u () = E[u(n)u (n)] = E[(a 1 u(n 1) + a u(n ) + a 3 u(n 3) + a 4 u(n 4) + v(n)) u(n)] = a 1 r u ( 1) + a r u ( ) + a 3 r u ( 3) + a 4 r u ( 4) + E[v(n)u(n)] = a 1 r u (1) + a r u () + a 3 r u (3) + a 4 r u (4) + E[v(n) (a 1 u(n 1) + a u(n ) + a 3 u(n 3) + a 4 u(n 4) + v(n))] = a 1 r u (1) + a r u () + a 3 r u (3) + a 4 r u (4)+ a 1 E[v(n)u(n 1)] + a E[v(n)u(n )] + a 3 E[v(n)u(n 3)] + a 4 E[v(n)u(n 4)] + E[v(n)v(n)] = a 1 r u (1) + a r u () + a 3 r u (3) + a 4 r u (4) + σv r u () = a 1 r u (1) + a r u () + a 3 r u (3) + a 4 r u (4) + σ v r u () a 1 r u (1) a r u () a 3 r u (3) a 4 r u (4) = σ v, () since terms of the form a k E[v(n)u(n k)], k > are equal to zero. From Eq. (1) The autocorrelation at lag k = 1 is equal to : r u (1) = a 1 r u () + a r u ( 1) + a 3 r u ( ) + a 4 r u ( 3) r u (1) = a 1 r u () + a r u (1) + a 3 r u () + a 4 r u (3) a 1 r u () + (a 1)r u (1) + a 3 r u () + a 4 r u (3) + r u (4) =. (3) The autocorrelation at lag k = is equal to : r u () = a 1 r u (1) + a r u () + a 3 r u ( 1) + a 4 r u ( ) r u () = a 1 r u (1) + a r u () + a 3 r u (1) + a 4 r u () a r u () + (a 1 + a 3 )r u (1) + (a 4 1)r u () + r u (3) + r u (4) =. (4)
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 16 5 4 3 Estimated r u (k) True r u (k) 1 N=3 1 3 4 5 1 15 5 3 35 n (samples) Figure 15: Estimated and true autocorrelation sequence of the 4th order AR process for N = 3 samples The autocorrelation at lag k = 3 is equal to : r u (3) = a 1 r u () + a r u (1) + a 3 r u () + a 4 r u ( 1) r u (3) = a 1 r u () + a r u (1) + a 3 r u () + a 4 r u (1) a 3 r u () + (a + a 4 )r u (1) + a 1 r u () r u (3) + r u (4) =. (5) Finally, the autocorrelation at lag k = 4 is equal to : r u (4) = a 1 r u (3) + a r u () + a 3 r u (1) + a 4 r u () a 4 r u () + a 3 r u (1) + a r u () + a 1 r u (3) r u (4) =. (6) Equations () to (6) form a system of linear equations. Solving the system we get: r u () = 36.861, r u (1) = 5.178, r u () = 3, 7887, r u (3) = 1, 5576 and r u (4) = 3, 6389 and from Eq. (1) r u (k) = a 1 r u (k 1) + a r u (k ) + a 3 r u (k 3) + a 4 r u (k 4), k = 1,,..., N 1. The estimate and true autocorrelation functions for N = {3, 56, 48} can be seen in Fig. 15-17
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 17 4 3 1 N=56 1 3 Estimated r u (k) True r u (k) 4 5 1 15 5 n (samples) Figure 16: Estimated and true autocorrelation sequence of the 4th order AR process for N = 56 samples 4 3 1 N=48 1 3 Estimated r u (k) True r u (k) 4 4 6 8 1 1 14 16 18 n (samples) Figure 17: Estimated and true autocorrelation sequence of the 4th order AR process for N = 48 samples
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 18 1 PSD (db) 3 4 5 Blackman & Tukey PSD Parametric PSD True PSD 6.5 1 1.5.5 3 3.5 ω (rad) Figure 18: Blackman-Tukey, Parametric AR and True Power Spectral Density for the 4th order AR process for N = 3 samples 3. For the Blackman-Tukey method a hamming window of size N/5 samples has been used. All spectra have been normalized to 1 ( db in db scale). We see the comparative plots in Fig. 18- Ασκηση 6. We are given x(n) = v(n) +.75v(n 1) +.5v(n ). Taking the z-transform of both sides: X(z) = (1 +.75z 1 +.5z )V (z). Hence, the transfer function of the MA model is: X(z) V (z) = 1 +.75z 1 +.5z = 1 (1 +.75z 1 +.5z 1. (7) )
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 19 PSD (db) 5 1 15 5 3 35 4 45 Blackman & Tukey PSD Parametric PSD True PSD 5.5 1 1.5.5 3 3.5 ω (rad) Figure 19: Blackman-Tukey, Parametric AR and True Power Spectral Density for the 4th order AR process for N = 56 samples
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 PSD (db) 5 1 15 5 3 35 4 45 Blackman & Tukey PSD Parametric PSD True PSD 5.5 1 1.5.5 3 3.5 ω (rad) Figure : Blackman-Tukey, Parametric AR and True Power Spectral Density for the 4th order AR process for N = 48 samples
ΗΥ-57: Στατιστική Επεξεργασία Σήµατος Πρώτη Σειρά Ασκήσεων Λύσεις 15 1 Using long division, we may perform the following expansion of the denominator in Eq. (7): ( 1 +.75z 1 +.5z ) 1 = 1 3 4 z 1 + 5 16 z 3 64 z 3 11 56 z 4 + 45 14 z 5 91 496 z 6 + 93 1683 z 7 + 85 65536 z 8 67 6144 z 9 + 1541 148576 z 1 +... 1.75z 1 +.315z.469z 3.43z 4 +.439z 5.z 6 +.57z 7 +.13z 8.4z 9 +.15z 1. (8) 1. M= Retaining terms in Eq. (8) up to z, we may approximate the MA model with an AR model of order two as follows: X(z) V (z) 1 1.75z 1 +.315z. Retaining terms in Eq. (8) up to z 5, we obtain the following approximation in the form of an AR model of order five: X(z) V (z) 1 1.75z 1 +.315z.469z 3.43z 4 +.439z 5 3. M=1 Finally, retaining terms in Eq. (8) up to z 1, we obtain the following approximation in the form of an AR model of order ten: X(z) V (z) 1 D(z), where D(z) = 1.75z 1 +.315z.469z 3.43z 4 +.439z 5.z 6 +.57z 7 +.13z 8.4z 9 +.15z 1.