Limit Trms W ft av t d fr apprximatis w gts vry larg. Fr xampl, smtims fidig t prbability distributis f radm variabls lad t utractabl matmatical prblms. W ca s tat t distributi fr crtai fuctis f a radm variabl ca asily b apprximatd fr larg valus f, v tug t xact distributi s difficult t btai. Cvrgc i Prbability Supps a ci as prbability f cmig up ads a sigl tss, ad if w tss it tims w wdr abut t fracti f ads tat sws up. Ituitivly, w wuld xpct tat t largr is, t cls t p t fracti bcms. Tis is t always s. Tis lads t t qusti "wat apps t t radm diffrc btw a stimat ad its targt valu?" Lt X b t umbr f ads bsrvd i tsss. T, E[X] = p ad Var[X] = p(-p). O way t masur t clsss f x/ t p is t xami x p ad t s if it will b lss ta a pr-assigd umbr,. L NM O QP P Tis P x x p p suld b cls t uity fr larg if ur ituiti is crrct. Dfiiti: T squc f radm variabls, x, x, x3,..x, is said t "Cvrg i Prbability" t t cstat c if fr vry psitiv umbr, lim P[ x c ] = Trm Lt X, X,. X b idpdt ad idtically distributd radm variabl wit E[X I ] = µ ad Var[X i ] = s < i. i i= Furtr, lt X = X lim P[ X µ ] = 0 lim P[ X µ < ] =. T fr ay psitiv ral umbr Tat is, X cvrgs i prbability t µ.. Tis is calld t "Wak law f larg umbrs" Atr trm usig t abv ccpts is Trm Supps X cvrgs i prbability tµ ad Y cvrgs t µ. T: () X ad Y cvrg i prbability t µ ad µ. () X Y cvrgs i prbability t µ µ (3) X /Y cvrgs i prbability t µ /µ ; µ 0 (4) X cvrgs i prbability t µ prvidd tat P[X 0]=
Exampl; Exampl: Supps X, X,. X ar idpdt ad idtically distributd radm variabls wit E[X i ] = µ, E[X i ] = µ ', E[X 3 i ] = µ ' 3, E[X 4 i ] = µ ' 4, wit all fiit Lt S ' b t sampl variac, c, ' S = ( Xi X ) i= Sw S ' cvrgs i prbability t Var(X i ) Sluti: ' S = ( Xi X ) i= X = ( Xi X Xi + X ) = Xi Xi + X Wr i= i= i= i= = Xi i= X = i= X X i ' Lkig at trms i S, X i is t avrag f idpdt ad idtically i= distributd radm variabls f t frm X wit E[ X ] = µ ', ad Var[ X ] = µ ' ( µ ' ) X. i i i X i i= Sic Var( X i ) is fiit, t 4 cvrgs i prbability t µ (Try tlls us tat X cvrgs i prbability t µ). Sic X i ad X cvrg i i= prbability t ' ' ' ' µ, ad µ it fllws tat S = Xi X cvrgs i prbability t µ µ = Var( Xi ) i= Tis sws tat fr larg sampls. T sampl variac suld b cls t ppulati variac wit ig prbability. If w ar ccrd wit wat apps t t prbability distributi f crtai typs f radm variabls as w d t fllwig dfiiti. Dfiiti: If w lt Y b a radm variabl wit a distributi fucti, F (x) ad w lt Y b a radm variabl wit distributi fucti F(y), t if lim F ( y) F( y) at vry pit y fr wic F(y) is ctiuus t Y is said t "cvrg i distributi t Y. F(y) is calld t limitig distributi fucti f y.
Lt X, X, X b idpdt radm variabls, uifrmly distributd vr t itrval (0, θ ) fr a psitiv θ. Als. Lt y = max(x,x, X ) Fid t limitig distributi f Y. Sluti: F ( y) = P( X y) = X i R S T 0 y θ Frm rdrd statistics w kw y 0 0 < y < θ y θ G( y) = P[ Y y] = F ( y), wr F X (y) is t distributi fucti fr ac X. R T 0 y T lim G( y) = Slim( ) θ X y 0 0 < y > θ y θ Tus, Y cvrgs i distributi t a radm variabl tat as a prbability f at t pit θ ad zr lswr. It is ft asir t fid limitig distributis wrkig wit mmt gratig fuctis. Trm: Lt Y ad Y b radm variabls wit mmt gratig fucti M (t) ad M(t), rspctivly. If lim M = M fr a ll ral t, t Y cvrgs i distributi t Y. Exampl; ` A xprimtr wats t cut t umbr f bactria/small vlum f watr. T sampl siz is t vlum f watr i wr cut is mad. Fr purpss f apprximati, t prbability distributi f cuts w tik f t vlum, ad c t avrag cut pr vlum, as quatitis gttig larg. Lt X dt t bactria cut pr cc f watr, ad assum tat X as a Piss prbability dsity fucti, wit ma valu f. W wat t apprximat t prbability distributi f X fr larg valus f.. W X d tis by swig ;Y = cvrgs i distributi t a stadard rmal radm variabl as. Spcifically, if t allwabl plluti i a watr supply is a cut f 0/cc apprximat t prbability tat X will b at mst 0, assumig = 00. Sluti: W tak t limit f t MGF f Y as ad us ur trm. t ( ) M = ad c, t MGF fr Y is M X X t t t t / = M X( ) = xp[ ( )]
3 / t t T trm ( ) =... 3/ + + 6 + Ad up addig xpts, 3 t t t M Y = xp[ t + ( + + +...] / 3 6 3 t t = xp[ + +...] / 6 W s tat t first trm is fr f ad t trs av raisd t sm pwr i t dmiatr.. Trfr, lim M Y = t Tis is t MGF fr a stadard rmal fucti. W wat F HG P( X ) = P X 0 0. X W av sw tat Y = is apprximatly a stadard rmal radm variabl fr larg valus f. S, fr. = 00 F 0 00I P Y P Y HG K J = ( ) =. 843 0 T rmal apprximati t Piss prbability wrks rasably wll fr gratr ta 5 I K J Piss Prcss Additial Tpics i Prbability Lt us csidr t applicati f Prbability try t mdlig radm pm tat cag wit tim. Tis is t s-calld STOCHASTIC PROCESS. Tus, a tim paramtr is itrducd (wr "tim" is usd i a vry gral ss) ad t radm variabl, Y(t), is rgardd as a fucti f tim. Exampls ar dlss, but Y(t) culd rprst t siz f a bilgical ppulati at tim t; t cst f pratig a cmplx idustrial systm fr t tim uits; t umbr f custmrs waitig t b srvd at a cck-ut cutr t tim uits aftr pig.
A stcastic prcss, Y(t), is said t av "idpdt" icrmts if, fr ay st f tim pits, t 0 <t < <t, t radm variabl [Y(t i )-Y( ti- )] ad [Y(t j )-Y(t j- )] ar idpdt fr i j. T prcss as "statiary" idpdt icrmts if, i additi, t radm variabls [Y(t + )-Y(t + )] ad [Y(t )-Y(t )] av idtical distributis fr ay >0. I makig a matmatical mdl fr a ral wrld pm it is always cssary t mak crtai simplifyig assumptis s as t rdr t matmatics tractabl. O t tr ad, w cat mak t may simplifyig assumptis fr t ur cclusis wuld t b applicabl. O assumpti usually mad is t assum tat crtai radm variabls ar xptially distributd. T ras fr tis ids tat t xptial distributi is bt rlativly asy t wrk wit ad is ft gd apprximati t t actual distributi. T prprty f t xptial distributi tat maks it asy t aalyz is tat it ds t dtrirat wit tim. I.. if liftim f a itm is xptially distributd, t a itm wic as b usd fr t (ay umbr) urs is as gd as a w itm is, rgardlss t t amut f tim rmaiig util failur. Tis is als rfrrd t as t xptial distributi NOT avig a mmry, r mmrylss. T xptial distributi is t ly distributi tat psssss tis prprty. A ctiuus radm variabl is said t av a xptial distributi wit paramtr, >0, if x R x 0 f ( x) = S 0 x < 0 r T zx x F(x) = dx = R S T 0 x x 0 x < 0 T; tx E[ X ] = MGF = E[ ] = t < t E[ X ] = Var[ X ] = A radm variabl is mmrylss if P[X>s+t X>t] = P[X>s} s,t 0. Tis ca als b writt as P[X > s + t, X > t] = p[ X > s] P[ X > t] P[X > s + t] = P[X > s]p[x > t]
Ts quatis stat tat if t lctric cmpt (sic) is pratig at tim t, t t distributi f t rmaiig tim it survivs is t sam as rigial liftim distributi i.. T cmpt dis t rmmbr tat it as alrady b i us fr a tim t.\ Exampl: Supps tat t amut f tim spds i a bak is xptially distributd, wit a ma f t miuts, ( = /0). (a) Wat is t prbability tat a custmr will spd mr ta 5 miuts i t bak? (b) Wat is t prbability tat a custmr will spd mr ta 5 miuts giv tat still is i t bak aftr 0 miuts? Sluti: (a) Lt X = amut f tim i t bak. P[X>5]=-F(5) = -+ t/0. > = 5 / P[ X 5] 3 = = 0. 0 (b) Sic t prprty f mmrylss applis, giv tat spds 0 miuts i t bak, w d ly 5 miuts mr. > = 5 / P[ X 5] = =. 604 Furtr prprtis f t xptial distributi ar: () If X, X,. X ar idpdt ad idtically distributd radm variabls wit a ma valu f /, t t ( t) f X X X t.. ( ) = (Gamma distributi) ( )! () P( X < X ) = + A Stcastic prcss {N(t), t 0} is a cutig prcss if N(t) rprsts t ttal umbr f vts tat av ccurrd up t tim t. Fr xampl, a) Numbr f ppl trig a str prir t tim t. b) Ttal umbr f ppl br by tim t. c) Numbr f gals sccr playrs scr by tim t. S, t b a cutig prcss, i) N(t) 0 ii) N(t) is itgr valud iii) Fr s < t, t N(s) < N(t) iv) Fr s<t, N(t) - N(s) quals umbr f vts tat av ccurrd i t itrval (s,t)
A cutig prcss as idpdt icrmt if t umbr f vts tat ccur i disjit itrvals ar idpdt. I.. Evts ccurrd by tim 0 (tat is, N(0)) must b idpdt f vts ccurs btw 0 ad 5 [N(5) - N(0)]. A cutig prcss as statiary icrmts if t distributi f t umbr f vts wic ccur i ay itrval f tim dpds ly t lgt f t tim itrval, ad t t spcific tim. i.. [N(t +s) - N(t +s)] as t sam distributi as [N(t ) - N(t )], fr all t < t, s >0. O f t mst imprtat cutig Prcsss is t Piss Prcss. Dfiiti: T cutig prcss {N(t),t 0} is said t b a Piss prcss avig t rat, >0, if ) N(0) = 0 ) Prcss as idpdt icrmts 3) T umbr f vts i ay itrval, t, is Piss distributd, wit ma t. Tat is, fr all s,t 0 - t ( t) P[N(t + s) - N(s) = } = = 0,,,3,4..! Nt: Frm 3), a Piss prcss as statiary icrmts ad E[N(t)]=t -- wic xplais wy is calld t rat f t prcss I rdr t dtrmi if a arbitrary cutig prcss is actually. a Piss prcss w must sw tat ), ) ad 3) ar satisfid. Cditi ) is simply tat t cutig bgis at t=0. Cditi ) ca b dirctly vrifid by kwldg f t prcss. Hwvr, 3) is t clar as t w w culd dtrmi t cditi as satisfid. S. T lp dtrmi tis, a quivalt dfiiti is usful. As a prlud, lt us dfi t ccpt f a fucti, f( ) big (). Dfiiti: T fucti is said t b () if f lim ( ) = 0 Fr xampl, f() () f = x is () sic lim = lim = 0 0 0 () f = x is t () sic lim = lim = 0 0 0 (3) If f( ) is () ad g( ) is () t f( ) + g( ) is (), sic f() + g() f() g() lim = lim + lim = 0+ 0 = 0 0 0
cf() (4) If f( ) is () t s is g( ) = cf( ), sic lim = c lim f() = c 0 = 0 0 Frm 3) ad 4) it fllws tat ay fiit liar cmbiati f fuctis, ac f wic is ()m is als (). I rdr fr t fucti f( ) t b () it is cssary tat f()/ g t zr as gs t zr. But if gs t zr, t ly way fr f() t g t zr is fr f() t g t zr fastr ta ds. Tat is, fr small, f() must b small cmpard t. w ar w i a psiti t giv a altrativ dfiiti f a Piss prcss. Dfiiti: T cutig prcss {N(t),t 0} is said t b a Piss prcss avig rat, > 0, if i.) N(0) =0 ii.) Prcss as statiary ad idpdt icrmts iii.) P{N()=}= + () iv.) P{N() =() Examiig ts cditi, iii) stats t prbability f ccurrc i small tim itrval,, is prprtial t. Cditi iv) stats, prbability fmr ta ccurrc i gs t zr fastr ta itslf. Ts tw axims imply P{N(t+) - N(t)]=0}=--()= P[N()=0]= - - () T tw dfiitis ar ttally quivalt. Lt us dfi P ( t ) = P ( N ( t ) = ), ad w will driv a diffrtial quati fr P (t) as fllws: P ( t + ) = P{ N ( t + ) = 0} = P{ N = 0, N ( t + ) N = 0} = P{ N = 0} P{ N ( t + ) N = 0} = P [ + ( )] ( t + ) t) ( ) Hc, P P( = P + Lttig 0, i t limit w gt P' (t)=-p (t), r P ' =, wic, by itgrati, implis P l P = t + c r P = k Sic P ( 0) = P{ N ( 0) = 0} = (crtai vt frm i)) w arriv at P = t t
Similarly, fr >0 P ( t + ) = P[ N( t + ) = ] = P{ N =,[ N ( t + ) N = 0]} = P{ N =,[ N ( t + ) N = 0]} + P{ N =,[ ( t + ) N = ]} + ki= P{ N = k,[ N ( t + ) N = k]} Hwvr, frm iv) abv, t last trm is (); P ( t + ) = P P + P P ( ) + ( ) But, S, P ( ) = P{ N ( ) = 0} = + ( ) P ( ) = P{ N( ) = } = + ( ) P ( t + ) = ( ) P + P + ( ) Tus, P ( t + ) P = P + P + Lttig 0 yilds ' P = P + P r quivaltly, t ' t [ P + P ] = P t Usig P =, ( ) d t t t ( P ) = = dt r P ( t ) = ( t + c ) t Sic w av frm I), N(0)=0, t N=0 givs P =0, ad c=0 P ( t ) = t t Usig matmatical iducti, assumig t sluti fr =, t d t t ( P ) = P dt t t = ( )( t ) P = M t ( t) P =! Sic P ( 0) = 0 = t t t
Exampl: Supps tat i a crtai ara plats ar radmly disprsd wit a ma dsity f 0/squar yard. If a bilgist radmly lcats 00 -squar yard samplig quadrats t ara, w may f tm ca b xpctd t ctai plats? Sluti: Assumig t plat cuts/uit ara t av a Piss distributi wit a ma f 0/sq. yd., t prbability f plats i a squar yard ara is: =0 ad P[N(t=)=0]= t ( t) = = = 0! 0 0 40 If t 00 quadrats ar radmly lcatd, t xpctd umbr f quadras ctaiig zr plats is 40 00P[ Y( ) = 0] = 00 = 4. 5x0-6 Exampl: Custmrs arriv at a bak at a Piss rat. Supps tw custmrs arrivd durig t first ur. Wat is t prbability tat (a) bt arrivd durig t first 0 miuts (b) at last arrivd durig t first 0 miuts. Sluti: Lt s= fracti f r. (i.. s=/ mas 30 miuts) Giv = /r -,a rat f ccurrc Piss wit =, t=s Piss wit =, t=-s (a) P[ arrivalsi itrval (0, s) arrivalsi itrval (0,)] = P[ i (0, s),0 i (s,) P[ i (0,) Piss wit =, t= L = ( ) N M = s = 9! OL QP NM s ( s) ( s) ( [ s])! 0! ad w s = 3 0 O QP (0 miuts)
Exampl: (b) At last i first 0 miuts = at last i first s= /3 P[At last i first /3 ] = - P[Bt i last s=/3 (40 miuts)] Sic mmry, P[arrivals i last 40 miuts] is t sam as P[arrivals i first 40 miuts]. Hc, P[at last arrival i first 0 miuts]= -s = (-(/3) ) = 5/9 Cars crss a crtai pit i t igway i accrdac wit a Piss prcss wit rat = 3 pr miut. If Al blidly rus acrss t igway, t wat is t prbability tat will b ijurd if t amut f tim tat it taks im t crss t rad is s scds (assum tat if is t igway w t car passs, t will b ijurd)? D it fr s=,5,0,0. Sluti; = 3, ad s= scds t igway. t P t! ( ) = s = 0! 3 0 3 60 Hc, s/ 0 s/ 0 s/ 0 s/ 0 = s/ 0 ( / ) (fr s = ) =.90 (fr s = 5) =.779 (fr s = 0) =.607 (fr s = 0) =.368 Summary f Piss Prcsss Ctral t t dvlpmt is t ccpt f (). Rcall tat a fucti is said t b () if f ( ) / 0. Tat is, f is )) if, fr small valus f, f() is small v i rlati lim 0 = t. Supps w tat "vts" ar ccurrig at radm tim pits ad lt N)t) dt t umbr f vts tat ccur i t tim itrval [0,t]. T stcastic prcss {N(t),t 0} is a=said t b a Piss prcss avig t=rat, >0 if (i) N(0)=0.
(ii) T umbr f vts tat ccur i disjit tim itrvals ar idpdt (iii) T distributi f t umbr f vts tat ccur i a giv itrval dpds ly t lgt f tat itrval ad t its lcati. (iv) P { N( ) = } = t = ( ) (v) P { N( ) } = ( ). Tus, cditi (I) stats tat t prcss bgis at tim 0. Cditi (ii), t idpdt icrmt assumpti, stats fr istac, tat t umbr f vts by tim t ]tat is, N(T(] is idpdt f r umbr f vys tat ccur btw t ad t+s [tat is, N(t+s) - N(t)]. Cditi (iii), t statiary icrmt assumpti, stats tat, t prbabiltiy distributi f N(t+s) - N(t) is t sam fr all valus f t. W av dvlpd t Piss distributi by a limitig vrsi f t bimial distributi. W w ca btai t=is sam rsult by a diffrt mtd. Lmma. Fr a Piss prcss wit rat t P{ N( t) = 0}= Prf: Lt P = P{ N( t) = 0}. W driv a diffrtial quati fr P (t) i t fllwig mar. P ( t + ) R S T = P{ N ( t + ) = 0} = P{ N = 0, N ( t + ) N = 0} = P{ N = 0} P{ N ( t + ) N = 0} = P [ + ( ) Wr t fial tw quatis fllw frm Assumpti (ii) plus t fact tat Assumptis (iii) ad (iv) imply tat P{N()=0}=-+(). Hc P ( t + ) P ( ) = P + ' ' PO 9t0 Nw, lttig 0, w btai P = P, r quivaltly =, wic P implis by itgrati, tat lg P (t) = -t +c, r P (t)=k -t. Sic P (0)=P{N(0)=0}=, w arriv at P{ N( t) = 0}= Fr a Piss prcss, lt us dt by T t tim f t first vt. Furtr, fr >, lt T dt t lapsd tim btw t (-)st ad t t vt. T squc f itrarrival tims. Fr istac, if T = 5 ad t
T =0, t t first vt f t Piss prcss wuld av ccurrd at tim 5 ad t scd at tim 5. W w dtrmi t distributi f t T. T d s, w first t tat t vbt{t>t} taks plac if ad ly if vts f t Piss P{ T t P N t t > } = { ( ) = 0} prcss ccur i t itrval [0,t]. ad tus, Hc T as a xptial distributi wit a ma f. Nw Hwvr P{ T > t} = P{ T > t T } P{ T > t T = s} = P{ vts i (s,s + t) T = s} 0 = P{ 0 vts i(s,s + t)} = t wr t last tw quatis fllwd frm idpdt ad statiary icrmts. Trfr, frm t abv w cclud tat T is als a xptial radm variabl wit ma /, ad furtrmr, tat T is idpdt f T. Usig tis argumt rpatdly, w ca cclud: " T, T, ar idpdt xptial radm variabls ac wit ma /" Atr quatity f itrst is S, t arrival tim f t t vt, als calld t waitig tim util t t vt. It is asily s tat S = T ad c frm prpsiti statd abv, ad t rsults btaid arlir, it fllws tat S as a gamma distributi wit paramtrs ad. Tat is, t prbability dsity f S is giv by x f ( x S x ) ( ) = ( )! W ar w rady t prv tat N(t) is a Piss radm variabl wit ma t. Trm: Fr a Piss prcss wit rat, i= i P{ N = } =! t Prf: Nt tat t t vt f t Piss prcss will ccur bfr r at tim t if, ad ly if t umbr f vts tat ccur by tim t is a last. tat is
N S t ad s P{ N = } = P{ N } P{ N + } = P{ S t} P{ S t} z t x t t ( ) t ( x) = dx dx 0 ( )! 0 ( )! z z But itrgrati by parts frmula udv = uv vdu yilds, wit u x =, dv = [( γ x) / ( )!] dx, z + zt t 0 t t ( x) t ( t) dx = + 0 ( )!! Wic cmplts t prf. z ( x) d ( )!