Linear System Response to Random Inputs M. Sami Fadali Professor of Electrical Engineering University of Nevada 1
Outline Linear System Response to deterministic input. Response to stochastic input. Characterize the stochastic output using: mean, mean square, autocorrelation, spectral density. Continuous-time systems: stationary, nonstationary. Discrete-time systems. 2
Response to Deterministic Input Analysis: Given the initial conditions, input, and system dynamics, characterize the system response. Use time domain or s-domain methods to solve for the system response. Can completely determine the output. 3
Response to Random Input Response to a given realization is useless. Characterize statistical properties of the response in terms of: Moments. Autocorrelation. Power spectral density. 4
Analysis of Random Response 1. Stationary steady-state analysis: Stationary input, stable LTI system After a sufficiently long period Stationary response, can solve in s-domain. 2. Nonstationary transient analysis: Time domain analysis. Can consider unstable or time-varying systems. 5
Calculus for Random Signals Dynamic systems: integration, differentiation. Integral and derivative are defined as limits. Random Signal: Limit may not exist for all realizations. Convergence to a limit for random signals (law, probability, q th mean, almost sure). Mean-square Calculus: using mean-square convergence. 6
Continuity Deterministic continuous function x t at t 0 Lim t t 0 x t = x t 0 Mean-square continuous random function X t at t 0 Lim E X t X t 2 0 = 0 t t 0 Can exchange limit and expectation for finite variance. 7
Interchange Limit & E{.} (Shanmugan & Breipohl, p. 162) For a mean square continuous process with finite variance and any continuous function g. Lim t t 0 E g X t = E g X t 0 8
Continuity Theorem (Shanmugan & Breipohl, Stark & Woods) WSS X(t) is mean-square continuous if its autocorrelation function R XX τ is continuous at τ = 0. Proof: E X t + τ X t 2 = R XX 0 + R XX 0 2R XX τ Lim E X t + τ X t 2 τ 0 = 2 R XX 0 Lim τ 0 R XX τ = 0 if R XX τ is continuous at τ = 0. 9
Mean-square Derivative Ordinary Derivative xሶ t = Lim ε 0 x t+ε x t For a finite variance stationary process. M.S. Derivative: Limit exists in a mean-square sense. ε ሶ X t = l. i. m. ε 0 X t+ε X t ε Lim E ε 0 ሶ X t X t + ε X t ε 2 = 0 10
Mean-square Stochastic Integral T 2X I = න t dt = l. i. m. In T 1 n I n = i=1 X t i Δt i Integral exists when lim n E I I n 2 = 0 Integral is a linear operator 11
Mean-square Calculus Provides definitions of integrals and derivatives that are valid for random processes. Allows us to use calculus for stochastic systems. Allows us to write integro-differential equations to describe stochastic systems. 12
Expectation of Output E x t E x t = E න = න = න G t, τ f τ dτ G t, τ E f τ G t, τ dτ m f The integral is bounded if the linear system is BIBO stable. dτ General result for MIMO time-varying case. 13
Stationary Steady-state Analysis Expectation of Output Assume Stable LTI system in steady-state. Stationary random input process. න E x t = න G t τ dτ = න G t τ dτ m f G τ e jωτ dτ m x = G jω ሿ ω 0 m f Mean is scaled by the DC gain ω 0 14
Stationary Steady-state Analysis Autocorrelation of Output R xx τ = E x t x T t + τ = E න G u f t u du න G v f t + τ v dv T = න න R xx τ = න G u E f t u f T t + τ v න G T v du dv G u R ff τ + u v G T v du dv 15
Change of variable R xx τ = න න Change of variable ξ = u + τ R xx τ = න G u R ff τ + u v G T v du dv න Change order of integration G ξ τ R ff ξ v G T v dξ dv 16
Autocorrelation & Power Spectral Density of Output R xx τ = න G ξ τ න R ff ξ v G T v dv dξ = න G τ ξ න = G τ R ff τ G T τ R ff ξ v G T v dv R ff ξ G T ξ dξ Laplace transform: S xx s = G s S ff s G T s 17
Stationary Steady-state Analysis Stable LTI system with transfer function G s X s = G s F s, S xx s = G s S ff s G T (s) For SISO case: S XX s = G s G s S FF s X jω = G jω F jω S XX jω = G jω 2 S FF jω 18
Example: Gauss-Markov Process Used frequently to model random signals R FF τ = σ 2 e βτ S FF s = 2σ2 β s 2 + β 2 = 2σ2 β s + β 2σ2 β s + β 19
Example: SDF of Output Spectral factorization G s = 1 Ts+1 S XX s = G s G s S FF s = 1 Ts + 1 1 Ts + 1 2σ2 β s + β 2σ2 β s + β = 2σ 2 β Ts + 1 s + β 2σ 2 β Ts + 1 s + β 20
Mean-square Value of Output j R XX 0 = 1 2πj න S XX s ds j = 1 j 2πj න S XX s S + XX s ds j S + XX s = 2σ 2 β s + β Ts + 1 = 2σ 2 β Ts 2 + βt + 1 s + β Use integration table for 2-sided LT I 2 = c 0 2 + 0 R 2d 0 d XX 0 = σ2 1 βt + 1 21
System with White Noise Input S FF jω = A, G s = 1 Ts + 1 S XX jω = G jω 2 A S FF jω = ωt 2 + 1 E X 2 = 1 2π න A ωt 2 + 1 dω A = 2πT tan 1 ωt = A 2T 22
Bandlimited White Noise Input S FF jω = ቊ A, ω ω c, G s = 1 0, ω > ω c Ts + 1 S XX jω = G(jω 2 S FF jω A = ቐ ωt 2 + 1, ω ω c 0, ω > ω c E X 2 = 1 ω c 2π න A ω c ωt 2 + 1 dω = A πt tan 1 ω c T AΤ 2T for ω c T 1 (ω c ω BW = 1/T) Approximately the same as white noise. 23
Response of Physical filter E X 2 ൧ physical filter j = 1 2πj න AG s G s ds j 24
Noise Equivalent BW Approximate physical filter with ideal filter E X 2 ൧ = 1 ideal filter 2πβ 2π න Adω = 2Aβ 2πβ j E X 2 ൧ = 1 physical filter 2πj න j 1 AG s G s ds BW of ideal filter =noise equivalent BW β = j 1 2 2πj න G s G s ds j Gain 2 Ideal filter: BW 25
Example Find the noise equivalent bandwidth for the filter 1 G s = T 1 s + 1 T 2 s + 1 = 1 T 1 T 2 s 2 + T 1 + T 2 s + 1 j 1 β = noise equivalent BW = 2 2πj න G s G s ds j I 2 = c 0 2 1 = 2d 0 d 1 2 T 1 + T 2 1 β = 4 T 1 + T 2 26
Shaping Filter Use spectral factorization to obtain filter TF S XX s = S XX s S + XX s = G s G s 1 F(s) Unity White Noise G s = S + XX s G(s) Shaping Filter X(s) Colored Noise 27
Example: Gauss-Markov S XX s = 2σ2 β s 2 + β 2 = = S XX s S + XX s 2σ2 β s + β 2σ 2 β s + β Spectral Factorization gives the shaping filter G s = 2σ2 β s + β 28
Nonstationary Analysis Linear system: superposition Random initial conditions & random input Total response = zero-input response + zerostate response Assume zero cross-correlation to add autocorrelations. 29
Natural (Zero-input) Response Zero-input response: response due to initial conditions. Response for state-space model xሶ t = Fx t y t = Cx t y t = Ce F t t 0 x t 0 e Ft = L 1 si F 1 30
Example: RC Circuit RC circuit in the steady state. Capacitor charged by unity Gaussian white noise input voltage source. Close switch and discharge capacitor. Random initial condition but deterministic discharge. Unity Gaussian R G s = 1/(Cs) white noise voltage source 2R + 1/(Cs) 1 = 2RCs + 1 = 1 2Ts + 1 31 R C
Steady-state Response G s = E v 2 c = 1 j 2πj න j 1 2Ts + 1 G s G s ds Use Table 3.1 (Brown & Hwang, pp. 109) with d 1 = 2T, d 0 = c 0 = 1 E v c 2 = I 1 = c 0 2 2d 0 d 1 = 1 4T 32
Close switch at t = 0 Tvሶ c t + v c t = 0 sv c s v c 0 + V c s T = 0 Unity Gaussian white noise voltage source V c s = v c 0 s + 1/T v c t = v c 0 e t/t E v c 2 t = E v c 2 0 e 2t/T = 1 4T e 2t/T R R C 33
Plot of Natural Response v c 2 t t 34
Forced (Zero-state) Response MIMO Time-Varying Case R XX t 1, t 2 = E x t 1 x T t 2 t x t = න G t, τ f τ dτ 0 = න 0 t 1 න 0 t 2G t1, u E f u f T v G T t 2, v du dv = න 0 t 1 න 0 Mean square with t 1 = t 2 t 2G t1, u R ff u, v G T t 2, v du dv 35
Forced (Zero-state) Response SISO Time-invariant Case Autocorrelation t X t = න g u F t u du 0 R XX t 1, t 2 = E X t 1 X t 2 t 2 t 1g = න න t1 u g t 2 v R FF u v dudv = න 0 0 0 t 2 න 0 t 1g u g v RFF u v + t 2 t 1 dudv 36
Mean Square: SISO Timeinvariant Case R XX t 1, t 2 = න 0 R XX t, t t 2 න 0 t 1g u g v RFF u v + t 2 t 1 dudv = E X 2 t t = න 0 t න g u g v R FF u v dudv 0 37
Example: RC Circuit g u = 1 T e Τ u T, u 0 Mean Square t t E X 2 = න 0 න 0 = A T 2 න 0 = A T 2 න 0 t R FF u = Aδ u g u g v R FF u v dudv t න e uτt e vτt δ u v dudv 0 t e Τ 2v T dv = A 2T 1 e Gaussian white noise voltage source f 2t Τ T R C 38
Example: Autocorrelation R XX t 1, t 2 = න 0 = t 2 t 1 න 1 0 g u = 1, u > 0, R FF u = δ u t 2 t 1g = න න u g v RFF u v + t 2 t 1 dudv 0 0 t 1du න, t2 t 1 0 න 0 1 δ u v + t2 t 1 dudv t 2dv, t1 > t 2 = min t 1, t 2 t 2 t 2 t 1 F(s) v 1/s 39 v=u+t 2 t 1 t 1 u X(s)
Discrete-Time (DT) Analysis Difference equations. z-transform solution. z = time advance operator (z 1 = delay) Similar analysis to continuous time but summations replace integrals. 40
Z-Transform (2-sided) G k =, G 1, G 0,, G i, G z = + G 1 z 1 + G 0 + + G i z i + Linear: Z af k + bg k = af z + bg z Convolution Theorem Z G k i f i = G z F z f k i= =, f 1, f 0,, f i, 41
Response of DT System x k = i= X z = G z F z, Convolution Summation G i impulse response G i f k i = Z-transform Z i= G z = G k i f i i= G i z i G(z) transfer function 42
Even Function G m = G m G z = G m z m = m= m= = G l z l = G z 1 l= G z = G z 1 G m z m 43
Expectation of the Output E x k = E G i f k i = G i E f k i i= i= Stationary G e jωt = frequency response m x = G i m f = G i z i m f i= i= = G 1 m f z=1 The mean is scaled by the DC gain 44
Discrete-time Processes R m = E x l x l + m Power spectrum: Discrete-time Fourier Transform (DTFT) of autocorrelation. S z = R m z m, S e jω = R m e jmω m= π m= R m = 1 2π න S e jω e jmω dω π 45
Properties of R and S R real, even: S real, even, cosine series R m = R m S z = R m z m = S z 1 m= = R 0 + R m (z m + z m ) m=1 S e jω = R 0 + 2 R m cos mω m=1 46
Autocorrelation of the Output E x k x T k + n = E G i f k i G j f k + n j i= j= = G i E f k i f T k + n j i= j= = G i R ff n + i j G T j i= j= T G T j 47
Autocorrelation of the Output E x k x T k + n = i= j= Substitute m = n + i, i = m n = m= G n m j= G i R ff n + i j G T j R ff m j G T j R ff m G T m R xx n = G n R ff n G T n 48
Z-Transform of G n G n =, G 1, G 0,, G i, G z = + G 1 z 1 + G 0 + + G i z i + G n =, G 1, G 0,, G i, Z G n = + G 1 z 1 + G 0 + + G i z i + = G z 1 49
Spectral Density Function S xx z = R xx n z n = G i R ff n + i j G T j n= n= i= i= m = n + i j n = m + j i S xx z = G i R ff m G T j z m+j i m= i= j= = G i z i R ff m z m G T j z j i= m= j= = G z 1 S ff z G T z Same result using the convolution theorem. 50 z n
Mean Square of Output R xx n = G i R ff n + i j G T j i= j= E x k x T k + n ൧ = R n=0 xx 0 R xx 0 = G i R ff i j G T j i= j= 51
Cross Correlation E x k f T k + n = E = i= i= G i f k i f T k + n G i E f k i f T k + n R xf n = i= G i R ff n + i 52
Cross Correlation (2) E f k x T k + n = E f k = i= i= G i f k + n i E f k f T k + n i R fx n = i= G T i R ff n i G T i T 53
Cross Spectral Density S xf z S xf z = S xf z = n= R xf n z n = m= i= = i= G i z i n= i= G i R ff n + i G i R ff m z m i, m = n + i m= R ff m z m S xf z = G z 1 S ff z, G z 1 = S xf z S 1 ff z z n 54
Cross Spectral Density S fx z S fx z = n= S fx z = R fx n z n = m= i= = m= n= i= R ff n i G T i R ff m G T i z m+i, m = n i R ff m z m i= S fx z = S ff z G T z G z = S 1 ff z S fx z T = S T fx G T i z i z S T ff z z n 55
Expressions for SISO Case Autocorrelation: use R xx n = G n R ff n G T n R xx n = g i g j R ff n + i j i= j= Power Spectral Density S xx z = G z 1 G z S ff z S xx e jωt = G e jωt 2 S ff e jωt Identification: S fx z = S ff z G z G z = S fx z /S ff z = S fx z for unity white noise 56
Example For the transfer function G z = bz z a, a < 1 Determine the PSD of the output due to a unity white noise sequence. S xx z = G z 1 G z S ff z S xx z = S ff z = 1 b 2 z a z 1 a, a < 1 57
Shaping Filter F(z) G(z) X(z) Colored noise with PSD S XX Spectral factorization of the PSD S xx z = r 2 L z L z 1, L = 1 Transfer function of the shaping filter G z = rl z 58
Example S xx z = b 2 z a z 1 a, a < 1 Find the transfer function of the shaping filter for colored noise with PSD S XX. S xx z = b 2 L z L z 1 = bz bz 1 z a z 1 a G z = bz F(z) X(z) z a G(z) 59
Conclusion Mean, autocorrelation, spectral density. Using white noise in analysis is acceptable. Model colored noise using white noise. Use to extend KF to cases where the noise is not white. Discrete case. 60
References 1. R. G. Brown & P. Y. C. Hwang, Introduction to Random Signals and Applied Kalman Filtering, J. Wiley, NY, 2012. 2. A. Papoulis & S. U. Pillai, Probability, Random Variables and Stochastic Processes, McGraw Hill, NY, 2002. 3. K. S. Shanmugan & A. M. Breipohl, Detection, Estimation & Data Analysis, J. Wiley, NY, 1988. 4. T. Söderström, Discrete-time Stochastic Systems : Estimation and Control, Prentice Hall, NY, 1994. 61