Stationary Stochastic Processes, 06 Stationary Stochastic Processes Table of Formulas, 06 Basics of probability theory The following is valid for probabilities: P(Ω), where Ω is all possible outcomes 0 P(A), where A is some event P(A c ) P(A), where A c is the complement of A P(A B) P(A) + P(B), if the events A and B are mutually exclusive The addition law of probability: P(A B) P(A) + P(B) P(A B) The conditional probability: P(B A) P(A B) P(A) A and B are independent P(A B) P(A) P(B) Stochastic variables p X (k) k x Distribution functions: F X (x 0 ) P(X x 0 ) 0 x0 f X (x) dx k p X (k) k Expected value: E[X] m X x f X (x) dx (X discrete) (X continuous) (X discrete) (X continuous) (k m X ) p X (k) (X discrete) Variance: V[X] E[X ] m X k (x m X ) f X (x) dx (X continuous) Rules for expected value and variance (a and b constants): E[aX + b] ae[x] + b V[aX] a V[X] V[X + b] V[X] E[X + Y] E[X] + E[Y] V[X + Y] V[X] + V[Y] + C[X, Y]
Stationary Stochastic Processes, 06 Covariance: C[X, Y] E[(X m X )(Y m Y )] E[XY] m X m Y Correlation coefficient: ρ[x, Y] C[X, Y] V[X] V[Y] Taylor series expansions ("Gauss approximations"): E[g(X,..., X n )] g(e[x ],..., E[X n ]) V[g(X,..., X n )] c i V[X i ] + c i c j C[X i, X j ] i<j where c i g (x,..., x n ) x i xk E[X k ], k i The two-dimensional probability density function of a jointly Gaussian random variable, (X, Y), with E[X] E[Y] 0, V[X] V[Y] and C[X, Y] ρ is { } f X,Y (x, y) π ρ exp ( ρ ) (x ρxy + y ) Stationary stochastic processes Estimation of expected value: ˆm n n t X t V [ ˆm n ] n n nv [ ˆm n ] τ τ n+ r X (τ) (n τ )r X (τ) If ˆm n N(m, V[ ˆm n ]), the confidence interval for m is for large n I m : { ˆm n λ α/ V[ ˆmn ], ˆm n + λ α/ V[ ˆmn ]} with confidence level α. For confidence level 0.95, α 0.05 and λ α/ λ 0.05.96. Estimation of covariance function: ˆr n (τ) n τ (X t m X )(X t+τ m X ) for τ 0 n t
Stationary Stochastic Processes, 06 3 where m X is replaced by ˆm n if m X is unknown. If X t is Gaussian, with mean m X and covariance function r X (τ), such that τ0 r X(τ) <, then for t s + τ, nc [ˆr n (s),ˆr n (t)] u The Poisson process and the Wiener process {r X (u)r X (u + τ) + r X (u s)r X (u + t)} when n A simply increasing process {X(t), t 0} is a homogeneous Poisson process, if X(0) 0 and X(t) has stationary, independent increments. If the intensity is λ, E[X(t)] λt V[X(t)] λt r X (s, t) λ min(s, t) The interarrival times are independent and exponentially distributed with mean value /λ. A Gaussian process {X(t), t 0} is a Wiener process, if X(0) 0, and X(t) has independent increments, where X(t) X(t + h) N(0, σ h), E[X(t)] 0 V[X(t)] σ t r X (s, t) σ min(s, t) Spectral representations Relations between covariance function r X (τ) and spectral density R X (f): Continuous time Discrete time r X (τ) R X(f)e iπfτ df r X (τ) / / R X(f)e iπfτ df R X (f) r X(τ)e iπfτ dτ R X (f) τ r X(τ)e iπfτ Folding (aliasing): Let {Z t, t 0, ±d, ±d,... } be the continuous time process Y(t) sampled with time interval d and sampling frequency f s /d: R Z (f) R Y (f + kf s ) for f s / < f f s / k
4 Stationary Stochastic Processes, 06 Sum of harmonic components with random phase and amplitude: X(t) A 0 + A k cos(πf k t + φ k ) where φ k Rect(0, π), A k, k 0,..., n, are independent and E[A 0 ] 0. Covariance function: k r X (τ) σ 0 + where σ 0 E [A 0] and σ k E [A k ] /. Spectral density: σ k cos πf k τ k R X (f) k n b k δ fk (f), where b 0 σ 0 E [A 0], and b k σ k / E [A k ] /4. Linear filters - general theory Impulse response h(u): Y(t) h(u)x(t u) du u h(u)x(t u) (continuous time) (discrete time) Relation between covariance functions: h(u)h(v) r X(τ + u v) du dv r Y (τ) v h(u)h(v) r X(τ + u v) u (continuous time) (discrete time) Relation between spectral densities: R Y (f) H(f) R X (f) where H(f) is the frequency function corresponding to the impulse response h(n). Differentiation: X (t) exists (in quadratic mean) if r X (t) exists. This is equivalent
Stationary Stochastic Processes, 06 5 to (πf) R(f)df <. If X (t) exists, the following relations hold: r X (τ) r X(τ) R X (f) (πf) R X (f) V [X (t)] (πf) R X (f) df r X,X (τ) r X(τ) r X (j),x (k)(τ) ( )j r (j+k) X (τ) Integration: [ E ] g(s)x(s) ds g(s)e[x(s)] ds [ C g(s)x(s) ds, ] h(t)y(t) dt g(s)h(t) C[X(s), Y(t)] ds dt Cross-covariance and cross-spectrum: r X,Y (τ) C[X(t), Y(t + τ)] e iπfτ R X,Y (f) df R X,Y (f) H(f)R X (f) A X,Y (f)e iφ X,Y(f) where A X,Y (f) is the amplitude spectrum and Φ X,Y (f) the phase spectrum. The squared coherence spectrum is κ X,Y(f) A X,Y (f) R X (f)r Y (f) AR- MA- and ARMA-models White noise in discrete time: {e t, t 0, ±,...}, E[e t ] 0 and V[e t ] σ : R e (f) σ for / f / AR(p)-process: (a 0 ) X t + a X t + a X t +... + a p X t p e t
6 Stationary Stochastic Processes, 06 Yule-Walker equations for covariance function: r X (k) + a r X (k ) +... + a p r X (k p) Spectral density: { σ for k 0 0 for k,,... MA(q)-process: (b 0 ) R X (f) σ p k0 a ke iπfk Covariance function: Spectral density: X t e t + b e t + b e t +... + b q e t q r X (τ) Matched filter and Wiener filter Matched filter: with white noise: with colored noise: s(t u) c { σ j kτ b jb k for τ q 0 for τ > q q R X (f) σ b k e iπfk k0 h(u) c s(t u) SNR N 0 s(t u) du SNR c h(v)r N (u v) dv h(u)h(v)r N (u v) du dv Wiener filter: H(f) SNR R S (f) R S (f) + R N (f) RS (f) df RS (f)r N (f) R S (f)+r N (f) df
Stationary Stochastic Processes, 06 7 Spectral estimation Periodogram of the sequence {x(t), t 0,,,... n }, where X (f) n t0 x(t)e iπft. ] E [ˆRx (f) ˆR x (f) X (f) n τ / / k n (τ)r X (τ)e iπfτ K n (f u)r X (u)du where k n (τ) τ for n + τ n and K n n(f) n τ n+ k n(τ)e iπfτ. ] { R V [ˆRx (f) X (f) for 0 < f < / R X (f) for f 0, ±/ The distribution of the periodogram estimate is ˆR x (f) R X (f) χ () for 0 < f < / Modified periodogram ˆR w (f) n x(t)w(t)e iπft n t0 / X (ν)w(f ν)dν n Lag-windowing Averaging of spectrum ˆR lw (f) / τ / / ˆR av (f) K k Ln (τ)ˆr x (τ)e iπfτ K Ln (f ν)ˆr x (ν)dν K ˆR x,j (f) where K different spectrum estimates, ˆR x,j (f), j... K, are used. The distribution is ˆR av (f) R X (f) χ (K) for 0 < f < / K j
8 Stationary Stochastic Processes, 06 Fourier transforms g(τ) (α > 0) G(f) e iπfτ g(τ) dτ e α τ α α +(πf) α +τ π α e πα f τ e α τ (α (πf) ) (α +(πf) ) τ k e α τ k! {(α + iπf) k+ + (α iπf) k+ } (α +(πf) ) k+ e ατ π/α exp( (πf) 4α ) e α τ cos(πf 0 τ) e α τ sin(πf 0 τ) { α if τ 0 sin(πατ) πτ if τ 0 { α τ if τ α 0 if τ > α g(τ)h(τ) g(τ) h(τ) g(t)h(τ t)dt g (τ) α α α +(πf 0 + πf) α +(πf 0 +πf) πf 0 πf α +(πf 0 + πf 0+πf πf) α +(πf 0 +πf) { / if f α 0 if f > α { α ( ( if f 0 α (πf) cos πf )) α if f 0 G(f) H(f) G(ν)H(f ν)dν G(f)H(f) iπf G(f) g(ατ) α G( f α ) α g( τ α ) G(αf) g(τ τ 0 ) G(f)e iπfτ 0 g(τ)e iπf 0τ G(f f 0 )
Stationary Stochastic Processes, 06 9 Gaussian distribution table F(x) Φ(x) x 0.00 0.0 0.0 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.0 0.50000 0.50399 0.50798 0.597 0.5595 0.5994 0.539 0.5790 0.5388 0.53586 0. 0.53983 0.54380 0.54776 0.557 0.55567 0.5596 0.56356 0.56749 0.574 0.57535 0. 0.5796 0.5837 0.58706 0.59095 0.59483 0.5987 0.6057 0.6064 0.606 0.6409 0.3 0.679 0.67 0.655 0.6930 0.63307 0.63683 0.64058 0.6443 0.64803 0.6573 0.4 0.6554 0.6590 0.6676 0.66640 0.67003 0.67364 0.6774 0.6808 0.68439 0.68793 0.5 0.6946 0.69497 0.69847 0.7094 0.70540 0.70884 0.76 0.7566 0.7904 0.740 0.6 0.7575 0.7907 0.7337 0.73565 0.7389 0.745 0.74537 0.74857 0.7575 0.75490 0.7 0.75804 0.765 0.7644 0.76730 0.77035 0.77337 0.77637 0.77935 0.7830 0.7854 0.8 0.7884 0.7903 0.79389 0.79673 0.79955 0.8034 0.805 0.80785 0.8057 0.837 0.9 0.8594 0.8859 0.8 0.838 0.8639 0.8894 0.8347 0.83398 0.83646 0.8389.0 0.8434 0.84375 0.8464 0.84849 0.85083 0.8534 0.85543 0.85769 0.85993 0.864. 0.86433 0.86650 0.86864 0.87076 0.8786 0.87493 0.87698 0.87900 0.8800 0.8898. 0.88493 0.88686 0.88877 0.89065 0.895 0.89435 0.8967 0.89796 0.89973 0.9047.3 0.9030 0.90490 0.90658 0.9084 0.90988 0.949 0.9309 0.9466 0.96 0.9774.4 0.994 0.9073 0.90 0.9364 0.9507 0.9647 0.9785 0.99 0.93056 0.9389.5 0.9339 0.93448 0.93574 0.93699 0.938 0.93943 0.9406 0.9479 0.9495 0.94408.6 0.9450 0.94630 0.94738 0.94845 0.94950 0.95053 0.9554 0.9554 0.9535 0.95449.7 0.95543 0.95637 0.9578 0.9588 0.95907 0.95994 0.96080 0.9664 0.9646 0.9637.8 0.96407 0.96485 0.9656 0.96638 0.967 0.96784 0.96856 0.9696 0.96995 0.9706.9 0.978 0.9793 0.9757 0.9730 0.9738 0.9744 0.97500 0.97558 0.9765 0.97670.0 0.9775 0.97778 0.9783 0.9788 0.9793 0.9798 0.98030 0.98077 0.984 0.9869. 0.984 0.9857 0.98300 0.9834 0.9838 0.984 0.9846 0.98500 0.98537 0.98574. 0.9860 0.98645 0.98679 0.9873 0.98745 0.98778 0.98809 0.98840 0.98870 0.98899.3 0.9898 0.98956 0.98983 0.9900 0.99036 0.9906 0.99086 0.99 0.9934 0.9958.4 0.9980 0.990 0.994 0.9945 0.9966 0.9986 0.99305 0.9934 0.99343 0.9936.5 0.99379 0.99396 0.9943 0.99430 0.99446 0.9946 0.99477 0.9949 0.99506 0.9950.6 0.99534 0.99547 0.99560 0.99573 0.99585 0.99598 0.99609 0.996 0.9963 0.99643.7 0.99653 0.99664 0.99674 0.99683 0.99693 0.9970 0.997 0.9970 0.9978 0.99736.8 0.99744 0.9975 0.99760 0.99767 0.99774 0.9978 0.99788 0.99795 0.9980 0.99807.9 0.9983 0.9989 0.9985 0.9983 0.99836 0.9984 0.99846 0.9985 0.99856 0.9986 3.0 0.99865 0.99869 0.99874 0.99878 0.9988 0.99886 0.99889 0.99893 0.99896 0.99900 3. 0.99903 0.99906 0.9990 0.9993 0.9996 0.9998 0.999 0.9994 0.9996 0.9999 3. 0.9993 0.99934 0.99936 0.99938 0.99940 0.9994 0.99944 0.99946 0.99948 0.99950 3.3 0.9995 0.99953 0.99955 0.99957 0.99958 0.99960 0.9996 0.9996 0.99964 0.99965 3.4 0.99966 0.99968 0.99969 0.99970 0.9997 0.9997 0.99973 0.99974 0.99975 0.99976 3.5 0.99977 0.99978 0.99978 0.99979 0.99980 0.9998 0.9998 0.9998 0.99983 0.99983 3.6 0.99984 0.99985 0.99985 0.99986 0.99986 0.99987 0.99987 0.99988 0.99988 0.99989 3.7 0.99989 0.99990 0.99990 0.99990 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 3.8 0.99993 0.99993 0.99993 0.99994 0.99994 0.99994 0.99994 0.99995 0.99995 0.99995 3.9 0.99995 0.99995 0.99996 0.99996 0.99996 0.99996 0.99996 0.99996 0.99997 0.99997 4.0 0.99997 0.99997 0.99997 0.99997 0.99997 0.99997 0.99998 0.99998 0.99998 0.99998