1999 by CRC Press LLC

Transcript

1 Poularias A. D. Probability ad Stochastic Processes The Hadboo of Formulas ad Tables for Sigal Processig. Ed. Aleader D. Poularias Boca Rato: CRC Press LLC, by CRC Press LLC

2 34 Probability ad Stochastic Processes 34. Aioms of Probability 34.. Aioms of Probability 34. Aioms of Probability 34. Coditioal Probabilities Idepedet Evets 34.3 Compoud (Combied)Eperimets 34.4 Radom Variable 34.5 Fuctios of Oe Radom Variable (r.v.) 34.6 Two Radom Variables 34.7 Fuctios of Two Radom Variables 34.8 Two Fuctios of Two Radom Variables 34.9 Epected Value, Momets, ad Characteristic Fuctio of Two Radom Variables 34. Mea Square Estimatio of R.V.'s 34. Normal Radom Variables 34. Characteristic Fuctios of Two Normal Radom Variables 34.3 Price Theorem for Two R.V.'s 34.4 Sequeces of Radom Variables 34.5 Geeral Cocepts of Stochastic Processes 34.6 Statioary Processes 34.7 Stochastic Processes ad Liear Determiistic Systems 34.8 Correlatio ad Power Spectrum of Statioary Processes 34.9 Liear Mea-Square Estimatio 34. The Filterig Problem for Statioary Processes 34. Harmoic Aalysis 34. Maroff Sequeces ad Processes Refereces I. PA ( ),II. PS ( ), III. If AB the PA ( + B) PA ( ) + PB ( )[. S a set of elemets of outcomes {ζ ι } of a eperimet (certai evet), empty set (impossible evet). {ζ ι } elemetary evet if {ζ ι } cosists of a sigle elemet. A + B uio of evets, AB itersectio of evets, evet a subset of S, P(A) probability of evet A. 999 by CRC Press LLC

3 34.. Corollaries of Probability Eample P( ), P( A) P( A),( A complemet set of A) PA ( + B) PA ( ) + PB ( ), PA ( + B) PA ( ) + PB ( ) PAB ( ) PA ( ) + PB ( ) S {hh,ht,th,tt} (tossig a coi twice), A {heads at first tossig} {hh,ht}, B {oly oe head came up} {ht,th}, G {heads came up at least oce} {hh,ht,th}, D {tails at secod tossig} {ht, tt} 34. Coditioal Probabilities Idepedet Evets 34.. Coditioal Probabilities PAM ( ) probability of evet AM PAM ( ) coditioal probaqbility of Agive M. PM ( ) probabilty of evet M PAM ( ) if AM PA PAM ( ) ( ) PM P ( A ) if ( ) AM A ( A M ) PAM PM ( ) ( ) if M A PM ( ) P( A+ BM) P( AM) + P( BM) if AB Eample P( f ) 6 /, i, L 6. M { odd} { f, f, f }, A { f }, AM { f }, P( M) 36 /, P( AM) 6 /,the i P( f eve) P( AM)/ P( M) 3 / 34.. Total Probability PB ( ) PBA ( ) PA ( ) + L + PBA ( ) PA ( ) arbitrary evet, AA i j,, L, A + L+ A S certai evet Baye's Theorem 3 5 PBA ( i) PA ( i) PA ( i B) PBA ( ) PA ( ) + + PBA ( ) PA ( ) L i j AA i j, i j,, L, A + A + LA S certai evet, Barbitrary Idepedet Evets PAB ( ) PAPB ( ) ( ) implies A ad B are idepedet evets Properties. PAB ( ) PA ( ) 999 by CRC Press LLC

4 . 3. PBA ( ) PB ( ) PAA ( LA) PA ( ) LPA ( ), Ai idepedet evets 4. PA ( + B) PA ( ) + PB ( ) PAPB ( ) ( ) 5. AB ( A+ B), P( A+ B) P( A+ B), P ( AB) P( A) P( B) If A ad B are idepedet. Overbar meas complemet set. 6. If Ai A A3 are idepedet ad A is idepedet of AA 3 the PAAA ( 3) PA ( ) PA ( ) PA ( 3) PA ( ) PAA ( 3). Also PA [ ( A + A3)] PAA ( ) + PAA ( 3) PAAA ( 3) PA ( ) [ PA ( ) + PA ( ) PA ( ) PA ( )] PA ( ) PA ( + A) PA ( + B+ C) PA ( ) + PB ( ) + PC ( ) PAB ( ) PAC ( ) PBC ( ) + PABC ( ) 34.3 Compoud (Combied, Eperimets SS S Cartesia product Eample S {,,3}, S {heads, tails}, S S S {( heads),( tails),( heads),( tails),(3 heads),(3 tails)} If A S, A S the A A ( A S )( A S ) (see Figure 34.) S A S A A A A S A S FIGURE Probability i Compoud Eperimets PA ( ) PA ( S) where ζ A ad ζ A Idepedet Compoud Eperimets PA ( A) P( A) P( A) Eample P(heads) p, P(tails) q, p+ q, E eperimet tossig the coi twice E E (E eperimet of first tossig), E eperimet of secod tossig), S {h,t} P {h}p P {t}q, E E eperimet of the secod tossig, S S S [ hh, ht, th, tt}, P{ hh} P{} h P{} h p assume idepedece, P{ ht} pq, P{ th} qp, P[ t, t} q. For heads at the first tossig, H { hh, ht} or PH ( ) Phh { } + P{ ht} p + pq p 999 by CRC Press LLC

5 Sum of more Spaces S S + S, S outcomes of eperimet E ad S outcomes of eperimet E. S space of the eperimet E E + E; A A + A where A ad A are evets of E ad E : A S, A S ; PA ( ) PA ( ) + PA ( ) Beroulli Trials PA ( ) probability of evet A, E E E... E perform eperimet times combied eperimet. p probability that evets occurs times i ay order pq ( ) PA ( ) ppa, ( ) qp, + q Eample A fair die was rolled 5 times. 5! 5 ( ) ( 5 )!! 6 6 probability that "four" will come up twice. Eample Two fair dice are tossed times. What is the probability that the dice total seve poits eactly four times? Solutio Evet probability of B occurig four times ad B si times is p 5 B P B 5 {( 6, ),( 5, ),( 34, ),( 43, ),( 5, ),( 6, )}, ( ) 6 p, P( 8) p P { } probability of success of A (evet) will lie betwee ad P { } p( ) pq. The. Approimate value: pq p pq e pq ( ) /, >> πpq DeMoivre-Laplace Theorem p pq pq e ( p)/ pq pq ( ), >> π Poisso Theorem! pq!( )! ( p) a a e e p p a!!,,, p 999 by CRC Press LLC

6 34.3. Radom Poits i Time t t T t a T ( ) λt ( λ a ) a P { i ta} e e, t t ta << T, radom poits i (,T), λ / T. If!!, T, / T λ the approimatio becomes equality. ta. P{oe i ta} e λ λta λt P. lim { oe i t } a λ ta t 3. P ( ( t, t)} e 34.4 Radom Variables Radom Variable a i t λ() t dt t To every outcome ζ of ay eperimet we assig a umber X( ζ ). The fuctio X, whose domai i the space S of all outcomes ad its rage is a set of umbers, is called a radom variable (r.v.) Distributio Fuctio F ( ) P { X } defied o ay umber < <. { X } is a evet for ay real umber Properties of Distributio Fuctio. F( ), F( + ). F ( ) F ( ) for < 3. F ( + ) F ( ) cotiuous from the right Desity Fuctio (or Frequecy Fuctio) a t t λ() tdt!, p α() t dt, α() t λ() t df( ) P { X + } f( ) ; f ( ) lim ; PX { } for cotiuous distributio fuctio; d o f( ) p δ i ( i) desity of discrete type, p F F i ( i) ( i ). i Eample Poisso distributio: PX { } λ λ e. The.!,,, L, λ > f( ) λ λ e δ! ( ) Eample If X is ormally distributed f e m ( ) / σ ( ) with m ad σ5, the the probability σ π m that X is betwee 9 ad,5 is P { X } f9ydy ) ( ) fydy ( ) ( ) + erf σ m erf erf + erf. 89 where error fuctio of erf y dy σ ep( / ) π t t 999 by CRC Press LLC

7 Tables of Distributio Fuctios (see Table 34.) TABLE 34. Distributio ad Related Quatities Defiitios. Distributio fuctio (or cumulative distributio fuctio [c.d.f.]): F ( ) probability that the variate taes values less tha or equal to P{ X } f( u) du. Probability desity fuctio (p.d.f.): f( ); P{ < X } f( ) d; f( ) l 3. Probability fuctio (discrete variates) f( ) probability that the variate taes the value P{ X } 4. Probability geeratig fuctio (discrete variates): u u l df d ( ) 5. Momet geeratig fuctio (m.f.g): Pt () Pt () t f( ), f( ) ( /!),,,,, X L > t t t µ rt Mt () t f( d ). Mt () + µ t+ + + µ L + L,! r! th r Mt () µ r r momet about the origi f( ) d r t M () t M () t M () t X+ Y X Y r r t 6. Laplace trasform of p.d.f.: 7. Characteristic fuctio : L s f () s e f( ) d, X Φ() t e jt f ( ) d, Φ () t Φ () t Φ () t X+ Y X Y 8. Cumulat fuctio: Kt () log Φ(), t KX+ Y() t KX() t + KY() t 9. r th r cumulat: the coefficiet of ( jt) / r! i the epasio of K(t). r th momet about the origi: r r Mt () µ r f( ) d r t r r Φ() t ( j) r t t t. Mea: µ first momet about the origi f ( ) d µ. r th momet about the mea: 3. Variace: σ secod momet about the mea ( µ ) f( ) d µ r µ r ( µ ) f ( ) d 4. Stadard deviatio: σ σ 5. Mea derivatio: µ f( ) d 6. Mode: A fractile (value of r.v.) for which the p.d.f is a local maimum 7. Media: m the fractile which is eceeded with probability /. 999 by CRC Press LLC

8 8. Stadardized r th momet about the mea: µ µ r ηr f( ) d r σ σ 3 9. Coefficiet of sewess: η µ / σ Coefficit of urtois: η 4 µ 4 / σ. Coefficiet of variatio: (stadard deviatio) / mea. Iformatio cotet: I f( )log ( f( )) d 3. r th factorial momet about the origi (discrete case): r σ / µ r () µ ( r) Pt f( ) ( ) L( r + ), X, µ ( r) r t 4. r th factorial momet momet about the mea (discrete case): t 5. Relatioships betwee momets: µ ( r) f( µ )( µ )( µ ) L( µ r+ ), X µ r µ µ µ r µ µ i r r i( ); r r i( ), µ µ, µ i i i i 6. log is the atural logarithm Distributios r r v w v w. Beta: p.d.f f( ) ( ) / Bvw (, ), Bvw (, ) beta fuctio u ( u) du ;r th momet about the r origi ( v+ i)( v+ w+ i) ; mea v/( v+ w) ; variace vw/( v+ w) ( v+ w+ ) ; mode ( v ) /( v+ w+ ), v >, i / / w>; coefficiet of sewess: [ ( w v)( v+ w+ ) ]/[( v+ w+ )( vw) ] ; coefficiet of urtois: ([ 3( v+ w)( v+ w+ ) ( v+ )( w v)]/{ vw( v+ w+ )( v+ w+ 3 )]) + [ v( v w)]/( v+ w) ; coefficiet of variatio: [ w/[ v( v+ w+)]] / ; p.d.f. v w f( ) [( v+ w )! ( ) ]/[( v )!( w )!], v ad w itegers; Bvw (, ) Γ() vγ( w)/ Γ( v+ w) Bwv (, ), Γ() c ( c ) Γ( c ) v w4 v4 w p.d.f. f() vw v w. Biomial:, p is the umber of successes i idepedet Bemoulli trials where the probability of success at each trial is p ad the probability of failure is q p, positive iteger < p <. c.d.f i i F ( ) pq, i i 999 by CRC Press LLC

9 iteger; p.d.f. iteger; momet geeratig fuctio: ; probability geeratig f pq ( ), [ pep( t) + q] fuctio: ( pt + q) ; characteristic fuctio : Φ( t) [ p ep( jt) + q]. momets about the origi: meap, secod p(p + q), third p[( )( ) p + 3p( ) + ] ; momet about the mea: variace pq, third pq(q - p), fourth pq[ + 3pq( )] stadard deviatio : ( pq) / ; mode: p ( + ) p ( + ) ; coefficiet of sewess: ( q p) /( pq) / ; coefficiet of urtois: 3-(6/)+(/pq); factorial momets about the mea: secod pq, third pq( + q) ; coefficiet of variatio ( q/ p) / 3. Cauchy: p.d.f f( ) /[ πb[( a) / b] + ]], α shift parameter, b,scale parameter, < < ; mode a media a ( v )/ v/ 4. Chi-Squared: p.d.f. f( ) [ ep( / )]/[ Γ( v/ )], v (shape parameter) degrees of freedom, < ; v / v / momet geeratig fuctio : ( t), t > / ; characteristic fuctio: Φ( t) ( jt) ; cumulat fuctio: ; r th r cumulat; ; r th r ( v/ )log( jt) v[( r )!] momet about the origi: [ i+ ( v/ )] ; mea v; variace: v; stadard deviatio ( v ) / ; Laplace trasform of the p.d.f: ( + s) v / r i f(). F() v 4 5. Discrete uiform: a a+ b, iteger, a lower limit of the rage, b scale parameter; c.d.f F() ( a+)/ b ; p.d.f. f( ) / b ;probability geeratig fuctio: ( t a a b t )/( t) ;characteristic fuctio: ep[j(a-)t] sih( jtb / )sih( jt / ) / b ; mea: a+ ( b )/ ; variace: ( b )/ ; coefficiet of sewess ; iformatio cotet: log b. 6. Epoetial: <, b scale parameter mea, λ /b alterative parameter; c.d.f F( ) ep( / b) ; p.d.f f( ) ( / b)ep( / b) ; momet geeratig fuctio: /( bt), t > ( / b) ; Laplace trasform of the p.d.f: /( + bs) ; characteristic fuctio: /( jbt) ; cumulat fuctio: log( jbt ) ; r th cumulat: ( r )! b r ; r th momet about the origi: rb! r ; mea: b : variace: b ; stadard deviatio: b; mea deviatio: b/e (e base ad atural log); mode: ; media: b log ; coefficiet of sewess: ; coefficiet of urtosis 9; coefficiet of variatio: ; iformatio cotet: log ( eb ). 7. F-distributio: <, v ad w positive itegers degrees of freedom: p.d.f f( ) [ Γ[ ( v w)]( v/ w) v / + ; r th ( v )/ ( v+ w)/ ]/[ Γ( v) Γ ( w)( + v/ w) ] momet about the origi: [( w/ v) r Γ ( v r) Γ ( w r) /[ Γ ( v) Γ + ( w)], w > r ; mea: w/( w ), w > ; variace: [ w ( v+ w )]/[ v( w ) ( w 4)], w > 4 ; mode [ wv ( )]/[ vw ( + )], / / v > ;coefficiet of sewess: [( v+ w )[ 8( w 4)] ]/[( w 6)( v+ w ) ], w > 6;coefficiet of variatio: [[ ( v + / w ) /[ v9w 4)]], w > by CRC Press LLC

10 a8 f() f() v4 w Gamma: <, b scal e parameter > (or λ /b ), c> shaper parameter; p.d.f f( ) ( / b) c c c [ep( / b)]/[ bγ( c)], Γ( c) ep( u) u du ; momet geeratig fuctio: ( bt), t > / b ; Laplace frasform of the c c p.d.f.: ( + bs) ; characteristic fuctio: ( jbt) ; cumulat fuctio: c log 9 jbt) ; r th cumulat: ( r )! cb r ; r th r momet about the origi: b ( c+ i) ; mea: bc; variace: bc ; stadard deviatio: b c ; mode: b ( c ), c ; coefficiet of sewess: r i c / ; coefficiet of urtosis: 3 + 6/c: coefficiet of variatio: c / f() f() c / c / c c Logoormal: <, m scale parameter media >, µ mea of log X >, m ep µ, µ log m, σ shape parameter stadard deviatio of log X, w ep( σ / ) ; p.d.f f( ) [ / σ( π) ]ep[ [log( / m)] / σ ] r th momet about the origi: m r ep( r σ / ) ; mea: m ep( σ / ) ; variace: mww ( ) : stadard deviatio: mw ( w) / ; mode m/w; media; m; coefficiet of sewess: ( w+ )( w ) / 4 3 ; coefficiet of urtosis: w + w + 3w 3 ; coefficiet of variatio: ( w ) /..8.4 f() m σ m σ.5 F() m σ.5 m σ 3 3. Negative bioomial: y umber of failures (iteger), umber of failures before success i a sequece of y + i i Beroulli trials; p probability of success at each trial, q p, y <, < p < ; c.d.f. Fy ( ) pq; i + y p.d.f. f y pq y ( ) ; momet geeratig fuctio: p ( qep t) ; probability geeratig fuctio: y p ( qt) ;; characteristic fuctio: p [ qep( jt)] ; cumulat fuctio: log( p) log( qep t); Cumulats: first 3 4 q / p, secod q / p, third q( + q)/ p, fourth q( 6q + p )/ p ;mea: q/p; Momets about the mea: variace 3 q / p, third q( + q)/ p, fourth ( q / p 4 )( q q p / ); stadard deviatio: ( q) / p; coefficiet of sewess: th i 999 by CRC Press LLC

11 / 6 p ( + q)( q) ; coefficiet of urtosis: factorial momet geeratig fuctio: r th factorial q ; ( t / ) q p momet about the origi: ( q/ p) r r / ( + r ) ; coefficiet of variato: ( q). f(y). p.5 f(y). 5 p y y. Normal: 8 < <, µ mea locatio parameter, σ stadard deviatio scale parameter, σ > ; p.d.f. / f( ) [ / σ( π ) ]ep[ ( µ ) / σ ]; momet geeratig fuctio: ep( µ t σ t ) ; characteristic fuctio: ep( jµ t σ t ); cumulat fuctio: jµ t σ t r th ; cumulat: K σ, K, r > ; mea: µ r th momet about r r/ the mea: µ r for r odd, µ ( σ r!) /[ [( r/ )!]] for r eve; variace: σ ; stadard deviatio: σ ; mea deviatio: σ( / π) / r / ; mode: µ; media: µ; coefficiet of sewess: ; coefficiet of urtosis: 3; iformatio cotet: log [ σ( πe) ] r + f() µ 3.5 F() 3 3. Pareto: <, c shape parameter; c.d.f. F ( ) c ; p.d.f. f c c ( ) ; r th momet about the origi: c/( c r), c > r; mea : c/( c ), c > ; variace: [ c/( c )] [ c/( c )], c > ; coefficiet of variatio: ( c )/[ c( c )] /, c >. f() c F() c 4 3. Pascal: umber of teals,, the Beroulli success parameter the umber of trials up to ad icludig the th success, p probability of success at each trial, < p <, q p; p.d.f. f momet geeratig pq ( ) ; fuctio: p ep( t) /( qep t) probability geeratig fuctio: ( pt) /( qt) ; characteristic fuctio: p ep( jt) /( q jt / ep( ) ; mea: /p; variace: q / p ; stadard deviatio: ( q) / p; coefficiet of variatio: ( q/ ) /. i 4. Poisso: <, λ mea (a parameter); c.d.f. F ( ) λ ep( λ ) / i!; p.d.f. momet geeratig fuctio: ep[ λ[ep( t) ]]; probability geeratig fuctio: ep[ λ( t)]; characteristic fuctio: i f( ) λ ep( λ) /!; 999 by CRC Press LLC

12 ep[ λ[ep( jt) ]] ;cumulat fuctio: λ[ep( t) ] t i / i!; r th cumulat: λ;momet about the origi: meaλ,secod i r λ + λ ; third λ[( λ + ) 3 r + λ],fourth λλ ( + 6λ + 7λ+ ); r th momet about the mea, µ i : λ µ i, r >, µ. i Momets about the mea: variace λ, thirdλ, fourth λ( + 3λ), fifth λ( + λ), sith λ( + 5λ + 5λ ); stadard deviatio λ / ;coefficiet of sewess: λ / ;coefficiet of urtosis: 3+ / λ ;factorial momets about the mea: secod λ, third -λ, fourth 3λ(λ+); coefficiet of variatio: λ /. i f().6 λ / f().3 λ f(). λ Rectagular: a a+ b, rage, a lower limit,bscale parameter; c.d.f F ( ) ( a)/ b; p.d.f. f( ) / b; momet geeratig fuctio: ep( at)[ep( bt) ]/ bt; Laplace trasform of the p.d.f: ep( as)[ ep( bs)]/ bs; characteristic fuctio: ep( jat)[ep( jbt) ]/ jbt; mea: a+ b/ ; r th momet about the mea: µ r for r odd, µ r r ( b/ ) /( r+ ) for r eve;variace: b / ;stadard deviatio: b / ;mea deviatio b /4;media a+ b/ ;stadardized r th r / momet about the mea: µ r for r odd, µ r 3 /( r + ) for r eve; coefficiet of sewess: ; coefficiet of / urtosis: 95; coefficiet of variatio: b/[ 3 ( a+ b)] ; iformatio cotet: log b. f() F() b a a+b a a+b 6. Studet s: < <, v shape parameter (degrees of freedom), v positive iteger; p.d.f. f( ) [ Γ[( v+ )/ ] ( v+ )/ / [ + ( / v)] ]/[( πv) Γ ( v/ )] ; mea: ; r th momet about the mea: µ r for r odd, µ r / r [ 35 L( r ) v ]/ [( v )( v 4) L( v r)] for r eve, r<v: variace: v/( v ), v > ; mea deviatio: v / Γ( ( v )/ π / Γ( v); mode: ; coefficiet of sewess ad urtosis: f().5 v F() Weibull: <, b > scale parameter, c shape aprameter c>; c.d.f. F ( ) ep[ ( / b); p.d.f. f( ) ( c c / b c )ep[ ( / b) c ]; r th momet about the origi: b r Γ[( c+ r) / c]; mea: bγ[( c+ ) / c]. 999 by CRC Press LLC

13 f() c 3 b F() c 3 c c 3 3 TABLE 34. Normal Distributio Tables. / γ / f( ) distributio desity ( / π ) e, F( ) cumulative distributio fuctio ( / π ) τ, e d f ( ) f( ), f ( ) ( ) f( ), F( ) F ( ), P{ < X< } F ( ) F( ) f( ) f ( ) f ( ) F( ) f( ) f ( ) f ( ) by CRC Press LLC

14 TABLE 34. Normal Distributio Tables. (cotiued) / γ / f( ) distributio desity ( / π ) e, F( ) cumulative distributio fuctio ( / π ) e dτ, f ( ) f( ), f ( ) ( ) f( ), F( ) F ( ), P{ < X< } F ( ) F( ) f( ) f ( ) f ( ) F( ) f( ) f ( ) f ( ) by CRC Press LLC

15 TABLE 34. Normal Distributio Tables. (cotiued) / γ / f( ) distributio desity ( / π ) e, F( ) cumulative distributio fuctio ( / π ) e dτ, f ( ) f( ), f ( ) ( ) f( ), F( ) F ( ), P{ < X< } F ( ) F( ) f( ) f ( ) f ( ) F( ) f( ) f ( ) f ( ) by CRC Press LLC

16 TABLE 34. Normal Distributio Tables. (cotiued) / γ / f( ) distributio desity ( / π ) e, F( ) cumulative distributio fuctio ( / π ) e dτ, f ( ) f( ), f ( ) ( ) f( ), F( ) F ( ), P{ < X< } F ( ) F( ) f( ) f ( ) f ( ) F( ) f( ) f ( ) f ( ) by CRC Press LLC

17 TABLE 34.3 Studet t-distributio Table f( ) + Γ y + / ) π Γ9 ( + )/ dy umber of degrees of freedom, umbers give of distributio, e.g., for 6 ad F.975,.447, F(-)-F() \F Coditioal Distributio F ( M ) PX M P { X X M } {, }, PM { } { X, M} evet of all outcomes ζ such that X( ζ ) ad ζ M.. F( M), F( M) F M FM P X M P { < X ( ) ( ) { }, M } < PM { } 999 by CRC Press LLC

18 TABLE 34.4 The Chi-Squared Distributio y F ( ) ( )/ / y e dy / F ( / ) umber of degrees of freedom \F.5, , by CRC Press LLC

19 TABLE 34.5 The F-Distributio f r r Γ( r + r)/ ]( r / r) F( f) p{ F f} Γ( r / ) Γ( r / )[ + ( r / r )] PF { f} 95. / ( / ) ( r+ r)/ d r \r , by CRC Press LLC

20 f r/ ( r/ ) Γ[( r + r) / ]( r / r) F( f) p{ F f} d F distributio ( r+ r)/ Γ( r / ) Γ( r / )[ + ( r / r )] PF { f}. 975 r \r by CRC Press LLC

21 TABLE 34.5 The F-Distributio f r r Γ( r + r)/ ]( r / r) F( f) p{ F f} Γ( r / ) Γ( r / )[ + ( r / r )] PF { f} 95. / ( / ) ( r+ r)/ d r \r F( f) p{ F f} PF { f} 99. f r / ( r / ) ( r+ r)/ Γ[( r + r) / ]( r / r) Γ( r / ) Γ( r / )[ + ( r / r )] d r \r by CRC Press LLC

22 f r/ ( r/ ) Γ[( r + r) / ]( r / r) F( f) p{ F f} d ( r+ r)/ Γ( r / ) Γ( r / )[ + ( r / r )] PF { f}. 995 r \r by CRC Press LLC

23 TABLE 34.5 The F-Distributio (cotiued) f r r Γ( r + r)/ ]( r / r) F( f) p{ F f} Γ( r / ) Γ( r / )[ + ( r / r )] PF { f}. 995 / ( / ) ( r+ r)/ d r \r , by CRC Press LLC

24 by CRC Press LLC

25 TABLE 34.6 The Poisso fuctio e f( )! λ λ 999 by CRC Press LLC

26 TABLE 34.7 The Poisso Distributio e F ( ) λ λ! 999 by CRC Press LLC

27 Coditioal Desity f M df ( M ) P X M ( ) lim { + } d f ( M ) d F ( M ) Eample X( fi ) i, i, L6 where f i face of a die. M { f, f4, f6 ] eve evet. For 6, { X, M} {,, } { f, f, f }, f( M) F f f f ; for 4 <6, { X, M} { f, f4}, F( M) P{ f, f4}/ PM { } PM { } ( / 6)/( 3/ 6) / 3; for < 4,{ X, M} { f}, F( M) P{ f}/ P{ M} (/6)(3/6) /3; for <, { X, M} ad FM ( ) Total Probability F ( ) FA ( ) PA ( ) + FA ( ) PA ( ) + + FA ( ) PA ( ) PA ( ), L their sum is equal to the certai evet S. A i s are mutually eclusive ad 34.5 Fuctio of Oe Radom Variable (r.v.) Radom Variable (Defiitio) To every eperimetal outcome ζ we assig a umber rage is the set I X of the real umbers X( ζ ).. X( ζ ). The domai of X is the space S, ad its Fuctio of r.v. Y g( X) g[ X( ζ )] Distributio Fuctio of Y (see 34.5.) F ( y) P{ Y y} P{ g( X) y} P{ X I } y y Note: To fid F ( y ) for a give y we must fid that set I y y ad the probability that X is i I y.refer to Figure 34.:If y the g ( ) yfor ay. Hece { Y y} certai evet ad F ( y ) P { Y y } y.if y y, the g ( ) y for ad,hece, Fy( y) P{ Y y} P{ X } F( ) ( depeds o y ), If y y, the g ( ) y has three solutios,, : g ( ) g ( ) g ( ) y ad from Figure 34. g ( ) y if or ad hece, F ( y ) P { X } P { } Y + FX( ) + FX( ) FX( ). If y < l o value of produces g ( ) y ad the evet { Y y} has zero probability: Fy( y). Eample Y / X. If y >, there are two solutios: y, / y. g( ) y if or ad thus F ( y) P{ Y y} P{ X / y} + P{ X / y} F ( / y) + F ( / y). y if y <, o will produce g ( ) y ad, hece, F ( y y ). 999 by CRC Press LLC

28 FIGURE Desity Fuctio of Yg(X) i Terms of f X () of X ) Solve yg() for i terms of y. If,, L, are all its real roots, the y g ( g ) L ( ) L, fx ( ) fx( ) the fy ( y) + L+ + L, g ( ) dg( )/ d. If yg() has o real roots the fy y. g ( ) g ( ) ( ) Eample g( ) ax + b ad ( y b)/ a for every y. g ( ) a ad hece f ( y) Y a f y b X a Eample gx ( ) ax with the r.v. y a, a>. If y < roots are imagiary ad f ( y ) Y. If y > the y / a ad y/ a.sice g ( ) a ay ad g ( ) a ay,the f Y (y) fx ay Eample 3 y f a + X y uy uy a ( ), ( ) uit step fuctio. Y asi( X + θ), a>. If y < a the y asi( + θ) has ifiitely may solutios L,,,, L. dg( ) / d acos( + θ) a y ad from f ( y) / a y f ( ), y < a. For y > there eist o solutios, ad f ( Y y ).; y si a θ, Eample 4 ax Y be u( X), a>, b>. If y < or y > b the the equatio y bep( a) u( ) has o solutio, ad hece f y If the ad Y y b, ( / a) l( y / b). g ( ) abe ay f Y y f ( ( / a) l( y/ b)) / ay, y b.. X Y ( ) X 999 by CRC Press LLC

29 Coditioal Desity of Yg() f Y ( ym) fx ( M) fx( M) + L+ + L g ( ) g ( ) Eample fx ( ) Y ax, a>, X, f ( X ) u ( ) (see Eample ), ad hece fy yx F ( ) ( ) fx ( y/ a) [ /( ay)] u ( ). F ( ) X X f( X t) f( )/{ F( t)] f( ) d, t t Epected Value E{ X} f ( ) d cotiuous r.v. EX { } PX { } p discrete r.v Epected Value of a Fuctio g(x) E{ Y g( X)} yf ( y) dy g( ) f ( ) d Y cotiuous r.v. EgX {( )} g ( ) PX { } discrete type of r.v Coditioal Epected Value EXM { } f( Md ) cotiuous r.v. EXM { } PX { M} discrete r.v Variace σ E{( X µ ) ( µ ) f( ) d cotiuous r.v. σ ( µ ) P{ X } discrete r.v. 999 by CRC Press LLC

30 σ EX { } E{ X} Eample PX { ) e Poisso distributio.!, L,, but or λ λ ad hece, EX { } λ Momets About the Origi Cetral Momets Absolute Momets Geeralized Momets λ λ λ λ λ λ EX { ) e e e.!!! d d e λ d λ λ e λ dλ! λ! λ λ λ e! r µ EX { } f( d ) µ µ r, µ µ EX { }, µ r r r r r r r µ EX µ f d E µ X µ µ r r r { } ( ) ( ) ( ) r ( ) 3 µ µ, µ µ µ, µ µ µµ + µ µ µ, µ µ 3µµ + 3µ µ µ µ 3 3µµ + µ M E{ X } f( ) d a E X a am E X a µ {( ) }, { } λ 3 3 r by CRC Press LLC

31 Eample a a a EX { } d, E { } a σ a + 3 for X uiformly distributed i (-a,a). Eample b+ a EX { } Γ( b + ) b a e b+ a Γ( b+ + ) d b+ + a Γ( b+ ) for a gamma desity b b a f( ) { a + / Γ( b+ )] e u ( ), u ( ) uit step fuctio Tchebycheff Iequality σ P{ X µ σ }, µ EX { }. Regardless of the shape of f( ), P{ µ ε < X < µ + ε} ε Geeralizatios:. If f ( y y ) the PY EY {} { α }, α > α. E{ X α } P{ X α ε } ε Characteristic Fuctio Φ( ), Φ( ω) Eample jω Φ( ω) Ee { } f( d ) for cotiuous r.v. jω Φ( ω) e P{ X } for discrete type r.v. jωy jω( ax+ b) jωb jωax Φ( ω) Ee { } Ee { } e Ee { }, Eample if Y ax + b PX { } λ λ e Poisso distributio!, L,, λ jωλ λ jω Φ( ω) e e e ( e )! Secod Characteristic Fuctio Ψ( ω) l Φ( ω) 999 by CRC Press LLC

32 Iverse of the Characteristic Fuctio jω f( ) Φ( ω) e dω π Momet Theorem ad Characteristic Fuctio d Φ( ) j µ E X, { } dω Covolutio ad Characteristic Fuctio Φ( ω) Φ ( ω) Φ ( ω), where Φ ( ω) ad Φ ( ω) are the characteristic fuctios of the desity fuctios f ad. ω ( + ) ω Ee j X X } ad ( ) f f ( ) ( ) where idicates covolutio Characteristic Fuctio of Normal r.v. Φ( ω ) ep( jµω σ ω ) 34.6 Two Radom Variables Joit Distributio Fuctio F ( y) P{ X, Y y}, F (, ) F ( ), F (, y) F ( y), y y y y F (, ), F (, y), F (, ) y y y Joit Desity Fuctio Fy (, ) f(, y), f ( ) f (, y ) dy, fy ( y ) f (, y ) d y Coditioal Distributio Fuctio PY { ym, } PX { Y, y} Fy(, y) Fy ( ym) P{ Y ym}, FyX { } PM { } PX { } F ( ) PX { ay, by, y} y b Fy ( yx a, Y b) PX { ay, b} Fy( a, y)/ Fy( a, b) y< b 999 by CRC Press LLC

33 Coditioal Desity Fuctio F y y f y d f y d y(, )/ y( ξ, ) ξ y(, ) fy( yx ), fy( y < X F ( ) f (, y) d dy ( ) F( ), ξ ξ Γ fy(, y) fy( yx ) f ( ) Baye's Theorem y f ( yx ) y f ( Y y) f ( y) f ( ) y Joit Coditioal Distributio F (, ya< X b) y Fy( b, y) Fy( a, y) F( b) F( a) PX { Y, ya, < Y b} Fy(, y) Fy( a, y) Pa { < X b} F( b) F( a) > b a< b a Coditioal Epected Value gyf ( ) y( ydy, ) EgY {( ) X } gy () fy( yx dy ), EEYX { { }} EY { } f (, y) dy Idepedet r.v Joitly Normal r.v. EX { } µ, EY { } µ, σ σ, σ y σ. If r, f(, y) f( ) fy( y) idepedet. r <, r correlatio coeffifiet. F (, y) F ( ) F ( y); f (, y) f( ) f( y); f ( y) f ( y); f ( y) f ( ) y y y y y ( µ ) r( )( y ) ( y ) µ µ µ f(, y) ep + πσ σ r ( r ) σ σσ σ y 999 by CRC Press LLC

34 Coditioal Desities rσ fy( yx ) ep y µ ( ) µ σ ( r ) ( r ) π σ σ rσ EYX { } µ + ( µ ), σ σ r y σ If r µ µ the EY { X } ( r σ σ ) + σ 34.7 Fuctios of Two Radom Variables Defiitios Z g( X, Y) g[ X( ζ), Y( ζ)], Fz( z) P{ Z z}, Dz regio of y-plae such that gy (, ) z, {Z z} {( XY, ) } D z Distributio Fuctio fz() z dz P{ z < Z z + dz} fy(, y) ddy D z Desity Fuctio fz() z dz P{ z < Z z + dz} fy(, y) ddy Eample If the r.v. are idepedet the Eample D z z y Z X Y y z F z f y ddy df z () +, +, z z( ) y fz z fy z yydy (, ), () dz (, ). f () z f () z y f (, y) f ( ) f ( y) ad hece f () z f ( z y) f () y dy f () f ( z ) d y y covolutio of desities. Z X + Y, if z > so the + y z circle with radius z, F ( z) f (, y) ddy, if z <, r / σ z/ σ Fz( z). fy(, y) ( / πσ )ep[ ( + y )/ σ the Fz () z πre dr e, πσ z / σ z > ad fz () z e, z σ z y y z z + y z 999 by CRC Press LLC

35 Eample 3 r / σ z / σ fy(, y) ( / πσ )ep[ ( + y )/ σ ], Z + X + Y, Fz ( z) πre dr e, πσ z >, fz ( z) ( z/ σ )ep( z / σ, z > Rayleigh distributed, EZ {} σ π/, EZ { } σ, σz ( ( π/ )) σ Eample 4 z If f (, y) f (, y) the F () z f (, y) ddy, f () z yf ( zy, y) dy. The for y y z yz y z ry y ( /[ πσσ r ])ep + ( r ) σ σσ σ y the fz () z of Z X/ Y is fy (, y ) y z rz fz ( z ) [ /( r )] y ep πσσ + dy. ( r ) σ σσ σ But w yep[ y / a ] dy a e dw a ad hece If µ µ the fz () z is Cauchy desity Two Fuctios of Two Radom Variables Defiitios f ( z) [( r σσ / π]/[ σ ( z rσ / σ ) + σ ( r )]. z Z gxy (, ), W hxy (, ), D zw regio of the y plae such that gy (, ) z ad hy (, ) w, { Z z, W w} {( X, Y) Dy}, Fzw( z, w) fy(, y) ddy D zw Desity Fuctio f zw (z,w) f zw fy(, y) fy(, y) (, z w) + L+ + L, z g( i, yi), w h( i, yi) where ( i, yi) J (, y) J (, y) there are o real solutios for certai values of (z,w) the f (, z w). Jacobia of trasformatio zw are solutios. if Jy (, ) gy (, ) gy (, ) y hy (, ) hy (, ) y 999 by CRC Press LLC

36 Eample If z a + by, w c + dy the az+ by, y cz+ dw, where a, b, c ad d are fuctios of a,b,c, ad d. Eample a b Jy (, ) ad bc, fzw ( z, w) /[ ad bc ] fy( az + bw, cz + dw) c d z + + y, w / y. If z > the the system has two solutios: zw/ + w, y z/ + w ad, y y for ay w. Jy (, ) / + y y/ + y / y / y ( + w )/( z) ad from zw z zw z fzw( z, w) [ z/( + w )] fy, fy,. + w + w + + w + w If z <, f ( z, w). zw Auiliary Variable If z g(, y) we ca itroduce a auiliary fuctio w or w y. f () z f (, z w) dw. Eample If z y set auiliary fuctio w. The system has solutios y w, y z/ w. J(, y) w ad, hece, z zw fzw(, z w) (/ w) fy( w, z/ w) ad fz( z) ( / w) fy( w, z/ w) dw Fuctios of Idepedet r.v.'s If X ad Y are idepedet the Z g( X) ad W h( Y) are idepedet ad sice f zw f f y y (, z w) ( ) ( ) g ( ) h ( y ) 999 by CRC Press LLC

37 34.9 Epected Value, Momets, ad Characteristic Fuctio of Two Radom Variables Epected Value g ( ) Jy (, ) g ( ) h ( ) h ( ) E{( g X, Y)} g(, y) f (, y) ddy; E{ z} zf ( z) dz if z g(, y); E{ g( X, Y)} g(, y ) p, z, PX {, Y y} p Coditioal Epected Values discrete case r.v. E{( g X, Y M)} g(, y) f (, y M) ddy; Momets EgXYX { (, } gyf (, ) ( ydy, ) / f( ) gyf (, ) ( yx dy ) r r µ E{ X Y } y f (, y) ddy, µ R E{ XY} r y µ E{( X µ ) ( Y µ ) } ( µ ) ( y µ ) f (, y) ddy r y r y r y µ σ, µ σ, µ µ Covariace µ E {( X µ )( Y µ y)} E { XY } µ E { Y } µ y E { X } + µ µ y Correlatio Coefficiet y y y r E{( X µ )( Y µ )}/ E{( X µ ) E{( Y µ ) } µ / σ σ 999 by CRC Press LLC

38 Ucorrelated r.v.'s EXY { } EXEY { } { } Orthogoal r.v.'s EXY { } µ µ µ, µ µ µ, r µ / µ µ Idepedet r.v.'s f(, y) f ( ) f ( y) Note: y. If X ad Y are idepedet, g(x) ad h(y) are idepedet or EgXhY { ( ) ( )} EgX { ( )} EhY { ( )}. If X ad Y are ucorrelated, the a. b. E{( X µ )( Y µ )}, r y + y + y σ σ σ c. E{( X + Y) } E{ X } + E{ Y } d. EgXhY { ( ) ( )} EgX { ( )} EhY { ( )} i geeral Joit Characteristic Fuctio Φ ( ω, ω ) E{ e f (, y) e ddy, Ψ ( ω, ω ) l Φ ( ω, ω ) y j ( ω + ω y ) j ( ω + ω y ) y y y j( ω+ ωy) fy(, y) e y( ) d d ( ) Ψ ωω ω ω π Eample Φ ( ω) Ee { } Φ ( ω, ), Φ ( ω) Φ (, ω) jωx y y y Φ ( ω) Ee { } Ee { } Φ ( aω, bω) if Z ax+ by. z jωz j( aωx+ bωy) y if X ad Y are idepedet. Φ ( ω, ω ) Φ ( ω ) Φ ( ω ) y y 999 by CRC Press LLC

39 34.9. Momet Theorem r Φ(, ) ( + r) j µ r ω ω r Series Epasio of Φ( ω, ω ) Φ( ω, ω ) + je{ X} ω + je{ Y} ω { X } ω 4 EY { } ω EXY { } ωω + L+ { } ωω, 4! EX Y + L Ψ( ω, ω ) l Φ( ω, ω ) jµ ω + jµ ω y 34. Mea Square Estimatio of R.V.'s 34.. Mea Square Estimatio of r.v.'s a. a miimizes E{ X a) } if a E{ X} µ b. The fuctio gx ( ) EYX { } regressio curve miimizes y y σ ω rσ σ ω ω σ ω +L E{[ Y g( X)] } [ y g( )] f (, y) ddy r c. y a σ ad b E{} Y ae{} X miimize the m.s. error σ e E{[ Y ( ax + b) ]} [ y ( a b) ] f (, y) ddy e miimum error σ ( r ), r correlatio coefficiet of X ad Y. m 3 y d. If EX { } EY { } the costat a that miimizes the m.s. error e E{( y a) } is such that E{( Y ax) X} (orthogoality priciple) ad the miimum m.s. error is: em E {( Y ax ) Y } a E{ XY}/ E{ X E XY } ad hece em E { Y } { } also EX { }, e E Y E ax m { } {( ) } e E{[ Y E{ Y X}] } m 34. Normal Radom Variables 34.. Joitly Normal If ex { } EY { } the ormal joit desity is: 999 by CRC Press LLC

40 ry y f(, y) ep +, EX { } σ, EY { } σ πσ σ r ( r ) σ σσ σ 34.. Coditioal Desity rσ f( y) ep y σ ( r ) ( r ) π σ σ, rσ r EYX { } X, E{ Y X} ( r σ σ ) + σ σ Mea Value Liear Trasformatios EXY { } rσσ, EXY { ] σσ + r σσ E{( X µ )( Y µ )} rσ σ If X ad Y are joitly ormal with zero mea the y Z ax + by, W cx + dy. z y y y σ E{ Z } E{( ax + by) } a σ + b σ + abr σ σ w y y y σ E{ W } c σ + d σ + cdr σ σ, zw z w y y y r σσ E{ ZW} acσ + bdσ + ( ad + bc) r σσ 34. Characteristic Fuctios of Two Normal Radom Variables 34.. Characteristic Fuctio Φ( ω, ω) E{ep[ j( ωx + ωy]} ep[ ( σω + rσσωω + σω )] for E{ X} E{ Y}, ad X ad Y joitly ormal Characteristic Fuctio with Meas Φ( ω, ω ) ep[ ( ω µ + ω µ )]ep{ µ ω + µ ω ω + µ ω ], µ meas. j y ij joit momets about the 999 by CRC Press LLC

41 34.3 Price Theorem for Two R.V s Price Theorem If X ad Y are joitly ormal with µ E {( X µ )( Y µ y)} E { XY } E { X } E { Y }, the, a. If µ (r.v. s idepedet) EXY { r } EX { } EY { r } b. c Sequeces of Radom Variables Defiitios Defiitios real r.v. X, X, L, X; F(,, L, ) P{ X, L, X } distributio fuctio; f(, L, ) F/, L, desity fuctio Margial Desities F (, 3) F (,, 3, ) margial distributio for a sequece of four r.v. ; f(, 3 ) f(,,, ) d d margial desity Fuctios of r.v.'s E{( g X, Y)} g(, y) f (, y) ddy r µ r r EXY { } r EX { Y } dµ + EX { } EY { } µ EXY { } 4 EXYd { } µ + EX { } EY { } 4 ( µ + EXEY { } { }) dµ + EX { } EY { } µ + 4µ EXEY { } { } + EX { } EY { } Y g ( X, L, X ), L, Y g ( X, L, X ), µ fy, Ly ( y, L, y) f(,, L, )/ J(, L, ), J(, L, ) g M g L L g g Coditioal Desities f(, L,, L, ) f(, L,, L, )/ f(, L, ). + + Eample f(, 3) f(,, 3)/ f(, 3), F(, 3) f( ξ,, 3) dξ/ f(, 3) 999 by CRC Press LLC

42 Chai Rule f(, L, ) f(, L, ) Lf( ) f( ) Removal Rule f( ) f(, ) d, 3 3 f( ) f(,, ) f(, ) d d, f( ) f(, ) f( ) d Idepedet r.v. F(, L, ) F( ) LF( ); f(, L, ) f( ) Lf( ) f(, L,,, L, ) f(, L, ) f(, L, ) + + if X, L, X are idepedet of X+, L, X Mea, Momets, Characteristic Fuctio Epected Value E{ g( X, L, X )} L g(, L, ) f (, L, ) d Ld Coditioal Epected Values E{ X, L, } f(, L, ) d f(, L, ) d / f(, L, ) a. EEX { { X, L, X}} EX { } b. EXX { X} EEXX { { X, X} EXEX { { X, X} X} c. EX {, L, } EX { } if X is idepedet from the remaiig r.v.'s Ucorrelated r.v.'s X, L, X are ucorrelated if the covariace of ay two ofthem is zero, EXX { i j} EX { i} EX { j} for i j Orthogoal r.v.'s EXX { } for ay i j i j Variace of Ucorrelated r.v.'s σ + L+ σ + L + σ, σz E{ Z E{ Z} } if Z X + jy comple r.v., EZZ { i j} EZ { i} EZ { j} ucorrelated r.v.'s i j, E{ Z Z } orthogoal, are idepedet, i j f(, y,, y ) f(, y ) f(, y ) if Z X + jy ad Z X + jy 999 by CRC Press LLC