Stationary Stochastic Processes, 07 Stationary Stochastic Processes Table of Formulas, 07 Basics of probability theory The following is valid for probabilities: P(Ω), where Ω is all possible outcomes 0 P(A), where A is some event P(A c ) P(A), where A c is the complement of A P(A B) P(A) + P(B), if the events A and B are mutually exclusive The addition law of probability: P(A B) P(A) + P(B) P(A B) The conditional probability: P(B A) P(A B) P(A) A and B are independent P(A B) P(A) P(B) Stochastic variables p X (k) k x Distribution functions: F X (x 0 ) P(X x 0 ) 0 x0 f X (x) dx k p X (k) k Expected value: E[X] m X x f X (x) dx (X discrete) (X continuous) (X discrete) (X continuous) (k m X ) p X (k) (X discrete) Variance: V[X] E[X ] m X k (x m X ) f X (x) dx (X continuous) Rules for expected value and variance (a and b constants): E[aX + b] ae[x] + b V[aX] a V[X] V[X + b] V[X] E[X + Y] E[X] + E[Y] V[X + Y] V[X] + V[Y] + C[X, Y]
Stationary Stochastic Processes, 07 Covariance: C[X, Y] E[(X m X )(Y m Y )] E[XY] m X m Y Correlation coefficient: ρ[x, Y] C[X, Y] V[X] V[Y] Taylor series expansions ("Gauss approximations"): E[g(X,..., X n )] g(e[x ],..., E[X n ]) V[g(X,..., X n )] c i V[X i ] + c i c j C[X i, X j ] i i<j where c i g (x,..., x n ) x i xk E[X k ], k The two-dimensional probability density function of a jointly Gaussian random variable, (X, Y), with E[X] E[Y] 0, V[X] V[Y] and C[X, Y] ρ is { } f X,Y (x, y) π ρ exp ( ρ ) (x ρxy + y ) Trigonometric relations cos(α + β) cos α cos β sin α sin β cos(α β) cos α cos β + sin α sin β sin(α + β) sin α cos β + cos α sin β sin(α β) sin α cos β cos α sin β Stationary stochastic processes Estimation of expected value: ˆm n n t X t V [ ˆm n ] n n V [ ˆm n ] n τ n+ τ (n τ )r X (τ) r X (τ) for large n
Stationary Stochastic Processes, 07 3 If ˆm n N(m, V[ ˆm n ]), the confidence interval for m is I m : { ˆm n λ α/ V[ ˆmn ], ˆm n + λ α/ V[ ˆmn ]} with confidence level α. For confidence level 0.95, α 0.05 and λ α/ λ 0.05.96. Estimation of covariance function: ˆr n (τ) n τ (X t m X )(X t+τ m X ) for τ 0 n t where m X is replaced by ˆm n if m X is unknown. The Poisson process and the Wiener process A simply increasing process {X(t), t 0} is a homogeneous Poisson process, if X(0) 0 and X(t) has stationary, independent increments. If the intensity is λ, E[X(t)] λt V[X(t)] λt r X (s, t) λ min(s, t) The interarrival times are independent and exponentially distributed with mean value /λ. A Gaussian process {X(t), t 0} is a Wiener process, if X(0) 0, and X(t) has independent increments, where X(t) X(t + h) N(0, σ h), E[X(t)] 0 V[X(t)] σ t r X (s, t) σ min(s, t) Spectral representations Relations between covariance function r X (τ) and spectral density R X (f): Continuous time Discrete time r X (τ) R X(f)e iπfτ df r X (τ) / / R X(f)e iπfτ df R X (f) r X(τ)e iπfτ dτ R X (f) τ r X(τ)e iπfτ
4 Stationary Stochastic Processes, 07 Folding (aliasing): Let {Z t, t 0, ±d, ±d,... } be the continuous time process Y(t) sampled with time interval d and sampling frequency f s /d: R Z (f) R Y (f + kf s ) for f s / < f f s / k Sum of harmonic components with random phase and amplitude: X(t) A 0 + A k cos(πf k t + φ k ) where φ k Rect(0, π), A k, k 0,..., n, are independent and E[A 0 ] 0. Covariance function: r X (τ) σ 0 + σ k cos πf k τ k where σ 0 E [A 0] and σ k E [A k ] /. Spectral density: R X (f) b k δ fk (f), k k n where b 0 σ 0 E [A 0], and b k σ k / E [A k ] /4. Linear filters - general theory Impulse response h(u): Y(t) h(u)x(t u) du u h(u)x(t u) (continuous time) (discrete time) Relation between covariance functions: h(u)h(v) r X(τ + u v) du dv r Y (τ) v h(u)h(v) r X(τ + u v) u (continuous time) (discrete time) Relation between spectral densities: R Y (f) H(f) R X (f) where H(f) is the frequency function corresponding to the impulse response h(n).
Stationary Stochastic Processes, 07 5 Differentiation: X (t) exists (in quadratic mean) if r X (t) exists. This is equivalent to (πf) R(f)df <. If X (t) exists, the following relations hold: r X (τ) r X(τ) R X (f) (πf) R X (f) V [X (t)] (πf) R X (f) df r X,X (τ) r X(τ) r X (j),x (k)(τ) ( )j r (j+k) X (τ) Integration: [ E ] g(s)x(s) ds g(s)e[x(s)] ds [ C g(s)x(s) ds, ] h(t)y(t) dt g(s)h(t) C[X(s), Y(t)] ds dt Cross-covariance and cross-spectrum: r X,Y (τ) C[X(t), Y(t + τ)] e iπfτ R X,Y (f) df R X,Y (f) H(f)R X (f) A X,Y (f)e iφ X,Y(f) where A X,Y (f) is the amplitude spectrum and Φ X,Y (f) the phase spectrum. The squared coherence spectrum is κ X,Y(f) A X,Y (f) R X (f)r Y (f) AR- MA- and ARMA-models White noise in discrete time: {e t, t 0, ±,...}, E[e t ] 0 and V[e t ] σ : R e (f) σ for / f / AR(p)-process: (a 0 ) X t + a X t + a X t +... + a p X t p e t
6 Stationary Stochastic Processes, 07 Yule-Walker equations for covariance function: r X (k) + a r X (k ) +... + a p r X (k p) { σ for k 0 0 for k,,... Spectral density: MA(q)-process: (b 0 ) R X (f) p k0 a ke iπfk σ Covariance function: X t e t + b e t + b e t +... + b q e t q r X (τ) { σ j kτ b jb k for τ q 0 for τ > q Spectral density: q R X (f) b k e iπfk σ k0 Matched filter and Wiener filter Matched filter: with white noise: with colored noise: s(t u) c h(u) c s(t u) SNR N 0 s(t u) du SNR c h(v)r N (u v) dv h(u)h(v)r N (u v) du dv Wiener filter: H(f) SNR R S (f) R S (f) + R N (f) RS (f) df RS (f)r N (f) R S (f)+r N (f) df
Stationary Stochastic Processes, 07 7 Spectral estimation Periodogram of the sequence {x(t), t 0,,,... n }, where X (f) n t0 x(t)e iπft. ] E [ˆRx (f) ˆR x (f) X (f) n τ / / k n (τ)r X (τ)e iπfτ K n (f u)r X (u)du where k n (τ) τ for n + τ n and K n n(f) n τ n+ k n(τ)e iπfτ. ] { R V [ˆRx (f) X (f) for 0 < f < / R X (f) for f 0, ±/ The distribution of the periodogram estimate is ˆR x (f) R X (f) χ () for 0 < f < / Modified periodogram ˆR w (f) n x(t)w(t)e iπft n t0 / X (ν)w(f ν)dν n Lag-windowing Averaging of spectrum ˆR lw (f) / τ / / ˆR av (f) K k Ln (τ)ˆr x (τ)e iπfτ K Ln (f ν)ˆr x (ν)dν K ˆR x,j (f) where K different spectrum estimates, ˆR x,j (f), j... K, are used. The distribution is ˆR av (f) R X (f) χ (K) for 0 < f < / K j
8 Stationary Stochastic Processes, 07 Fourier transforms g(τ) (α > 0) G(f) e iπfτ g(τ) dτ e α τ α α +(πf) α +τ π α e πα f τ e α τ (α (πf) ) (α +(πf) ) τ k e α τ k! {(α + iπf) k+ + (α iπf) k+ } (α +(πf) ) k+ e ατ π/α exp( (πf) 4α ) e α τ cos(πf 0 τ) e α τ sin(πf 0 τ) { α if τ 0 sin(πατ) πτ if τ 0 { α τ if τ α 0 if τ > α g(τ)h(τ) g(τ) h(τ) g(t)h(τ t)dt g (τ) α α α +(πf 0 + πf) α +(πf 0 +πf) πf 0 πf α +(πf 0 + πf 0+πf πf) α +(πf 0 +πf) { / if f α 0 if f > α { α ( ( if f 0 α (πf) cos πf )) α if f 0 G(f) H(f) G(ν)H(f ν)dν G(f)H(f) iπf G(f) g(ατ) α G( f α ) α g( τ α ) G(αf) g(τ τ 0 ) G(f)e iπfτ 0 g(τ)e iπf 0τ G(f f 0 ) Parseval s theorem: n x(t) t0 / / X(f) df, where X(f) n t0 x(t)e iπft.
Stationary Stochastic Processes, 07 9 Gaussian distribution table F(x) Φ(x) x 0.00 0.0 0.0 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.0 0.50000 0.50399 0.50798 0.597 0.5595 0.5994 0.539 0.5790 0.5388 0.53586 0. 0.53983 0.54380 0.54776 0.557 0.55567 0.5596 0.56356 0.56749 0.574 0.57535 0. 0.5796 0.5837 0.58706 0.59095 0.59483 0.5987 0.6057 0.6064 0.606 0.6409 0.3 0.679 0.67 0.655 0.6930 0.63307 0.63683 0.64058 0.6443 0.64803 0.6573 0.4 0.6554 0.6590 0.6676 0.66640 0.67003 0.67364 0.6774 0.6808 0.68439 0.68793 0.5 0.6946 0.69497 0.69847 0.7094 0.70540 0.70884 0.76 0.7566 0.7904 0.740 0.6 0.7575 0.7907 0.7337 0.73565 0.7389 0.745 0.74537 0.74857 0.7575 0.75490 0.7 0.75804 0.765 0.7644 0.76730 0.77035 0.77337 0.77637 0.77935 0.7830 0.7854 0.8 0.7884 0.7903 0.79389 0.79673 0.79955 0.8034 0.805 0.80785 0.8057 0.837 0.9 0.8594 0.8859 0.8 0.838 0.8639 0.8894 0.8347 0.83398 0.83646 0.8389.0 0.8434 0.84375 0.8464 0.84849 0.85083 0.8534 0.85543 0.85769 0.85993 0.864. 0.86433 0.86650 0.86864 0.87076 0.8786 0.87493 0.87698 0.87900 0.8800 0.8898. 0.88493 0.88686 0.88877 0.89065 0.895 0.89435 0.8967 0.89796 0.89973 0.9047.3 0.9030 0.90490 0.90658 0.9084 0.90988 0.949 0.9309 0.9466 0.96 0.9774.4 0.994 0.9073 0.90 0.9364 0.9507 0.9647 0.9785 0.99 0.93056 0.9389.5 0.9339 0.93448 0.93574 0.93699 0.938 0.93943 0.9406 0.9479 0.9495 0.94408.6 0.9450 0.94630 0.94738 0.94845 0.94950 0.95053 0.9554 0.9554 0.9535 0.95449.7 0.95543 0.95637 0.9578 0.9588 0.95907 0.95994 0.96080 0.9664 0.9646 0.9637.8 0.96407 0.96485 0.9656 0.96638 0.967 0.96784 0.96856 0.9696 0.96995 0.9706.9 0.978 0.9793 0.9757 0.9730 0.9738 0.9744 0.97500 0.97558 0.9765 0.97670.0 0.9775 0.97778 0.9783 0.9788 0.9793 0.9798 0.98030 0.98077 0.984 0.9869. 0.984 0.9857 0.98300 0.9834 0.9838 0.984 0.9846 0.98500 0.98537 0.98574. 0.9860 0.98645 0.98679 0.9873 0.98745 0.98778 0.98809 0.98840 0.98870 0.98899.3 0.9898 0.98956 0.98983 0.9900 0.99036 0.9906 0.99086 0.99 0.9934 0.9958.4 0.9980 0.990 0.994 0.9945 0.9966 0.9986 0.99305 0.9934 0.99343 0.9936.5 0.99379 0.99396 0.9943 0.99430 0.99446 0.9946 0.99477 0.9949 0.99506 0.9950.6 0.99534 0.99547 0.99560 0.99573 0.99585 0.99598 0.99609 0.996 0.9963 0.99643.7 0.99653 0.99664 0.99674 0.99683 0.99693 0.9970 0.997 0.9970 0.9978 0.99736.8 0.99744 0.9975 0.99760 0.99767 0.99774 0.9978 0.99788 0.99795 0.9980 0.99807.9 0.9983 0.9989 0.9985 0.9983 0.99836 0.9984 0.99846 0.9985 0.99856 0.9986 3.0 0.99865 0.99869 0.99874 0.99878 0.9988 0.99886 0.99889 0.99893 0.99896 0.99900 3. 0.99903 0.99906 0.9990 0.9993 0.9996 0.9998 0.999 0.9994 0.9996 0.9999 3. 0.9993 0.99934 0.99936 0.99938 0.99940 0.9994 0.99944 0.99946 0.99948 0.99950 3.3 0.9995 0.99953 0.99955 0.99957 0.99958 0.99960 0.9996 0.9996 0.99964 0.99965 3.4 0.99966 0.99968 0.99969 0.99970 0.9997 0.9997 0.99973 0.99974 0.99975 0.99976 3.5 0.99977 0.99978 0.99978 0.99979 0.99980 0.9998 0.9998 0.9998 0.99983 0.99983 3.6 0.99984 0.99985 0.99985 0.99986 0.99986 0.99987 0.99987 0.99988 0.99988 0.99989 3.7 0.99989 0.99990 0.99990 0.99990 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 3.8 0.99993 0.99993 0.99993 0.99994 0.99994 0.99994 0.99994 0.99995 0.99995 0.99995 3.9 0.99995 0.99995 0.99996 0.99996 0.99996 0.99996 0.99996 0.99996 0.99997 0.99997 4.0 0.99997 0.99997 0.99997 0.99997 0.99997 0.99997 0.99998 0.99998 0.99998 0.99998