CODING & MODULATION Prof. Ig. Ato Čžmár, PhD. also from Dgtal Commucatos 4th Ed., J. G. Proaks, McGraw-Hll It. Ed. 00 CONTENT. PROBABILITY. STOCHASTIC PROCESSES
Probablty ad Stochastc Processes The theory of probablty ad stochastc processes s a essetal mathematcal tool the desg of dgtal commucato systems. Ths subect s mportat the statstcal modelg of sources that geerate the formato, the dgtzato of the source output, the characterzato of the chael through whch the dgtal formato s trasmtted, the desg of the recever that processes the formato-bearg sgal from the chael, ad the evaluato of the performace of the commucato system.. PROBABILITY The sample space S a de of the epermet cossts of the set of all possble outcomes. I the case of the de S {,, 3, 4, 5, 6 }.- where the tegers,, 6 represet the umber of dots o the s faces of the de. These s possble outcomes are the sample pots of the epermet. A evet s a subset of S ad may cosst of ay umber of sample pots. For eample, the evet A defed as A {,4}.- cossts of the outcomes ad 4. The complemet of the evet A, deoted by A _, cossts of all the sample pots S that are ot A ad, hece _ A {,3,5,6}.-3 Two evets are sad to be mutually eclusve f they have o sample pots commo that s, f the occurrece of oe evet ecludes the occurrece of the other. For eample, f A s defed as Eq..- ad the evet B s defed as the A ad B are mutually eclusve evets. I ths case we ca wrte B {,3,6}.-4 PA + PB /6 +3/6 5/6.
Jot evets ad ot probabltes Istead of dealg wth a sgle epermet, let us perform two epermets ad cosder ther outcomes. For eample, the two epermets may be two separate tosses of a sgle de or a sgle toss of two dce. I ether case, the sample space S cossts of the 36 twotuples, where,,,,6. If the dce are far, each pot the sample space s assged the probablty /36. I geeral, f oe epermet has the possble outcomes A,,,,, ad the secod epermet has the possble outcomes B,,,,m, the the combed epermet has the possble outcomes A,B,,,,,,,,m. Assocated wth each ot outcome A,B s the ot probablty PA,B whch satsfes the codto 0 P A, B Assumg that the outcomes B are mutually eclusve, t follows that m P A, B P A.-8 Smlarly, f outcomes A, are mutually eclusve the P A, B P B.-9 Furthermore, f all the outcomes of the two epermets are mutually eclusve, the m P A, B.-0
Codtoal probabltes Cosder a combed epermet whch a ot evet occurs wth probablty PA,B. Suppose that the evet B has occurred ad we wsh to determe the probablty of occurrece of the evet A. The codtoal probablty of the evet A gve the occurrece of the evet B s defed as P A, B P A B.- P B PrA B B A I a smlar maer P A, B P B A.- P A The relatos Eq..- ad.- may also be epressed as PA,B PA BPB PB APA.-3 A etremely useful relatoshp for codtoal probabltes s Bayes theorem. P A, B P A B P B P B A P A P B A P A.-4 We use ths formula to derve the structure of the optmum recever for a dgtal commucato system whch: the evets A,,,, represet the possble trasmtted messages a gve tme terval; PA represet ther a pror probabltes; B represets the receved sgal, whch cossts of the trasmtted message oe of the A corrupted by ose; ad PA B s the a posteror probablty of A codtoed o havg observed the receved sgal B.
Statstcal depedece The statstcal depedece of two or more evets s aother cocept probablty theory. It usually arses whe we cosder two or more epermets or repeated trals of a sgle epermet. To epla ths cocept, we cosder two evets A ad B ad ther codtoal probablty PA B, whch s the probablty of occurrece of A gve that B has occurred. Suppose that the occurrece of A does ot deped o the occurrece of B. That s PA B PA.-5 After substtuto P A, B P A P B.-6 Whe the evets A ad B satsfy the relato Eq...-6, they are sad to be statstcally depedet.
.. Radom Varables, Probablty Dstrbutos, ad probablty Destes Radom varable s fucto Xs whose doma s S ad elemets s S. Flp a co, possble outcomes are head H ad tal T, so S cotas pots labeled H ad T. X s ±. Dscrete radom varables. Cotuous radom varables -X ose voltage geerated by a electroc amplfer has a cotuous ampltude. Probablty dstrbuto fucto of the radom varable X. It s also called cumulatve dstrbuto fucto CDF. F P X < <.-9 Sce F s a probablty, ts rage s lmted to the terval 0-. I fact, F 0, F. Fgure -- Eamples of the comulatve dstrbuto fuctos of two dscrete radom varables. The radom varable Xt wll be wrtte smply as X Probablty desty fucto PDF of the radom varable X df p d < <.-0 or, equvaletly F p u du < <.- The dscrete part of p may be epressed as p P X δ.- Ofte we face wth the problem of determg the probablty that a radom varable X falls a terval,, where >. P < X F F p d.-3
I other words, the probablty of the evet { <X< } s smply the area uder the PDF the rage < X. Statstcally depedet radom varables The multdmesoal radom varables are statstcally depedet f ad oly f p,,, p p p.-38 Fgure -- A Eamples of the comulatve dstrbuto fuctos of cotuous radom varable Fgure -- A Eamples of the comulatve dstrbuto fuctos of a med type
..3 Statstcal Averages of Radom Varables Averages play a mportat role the characterzato of the outcomes of epermets ad the radom varables defed o the sample space of the epermets. Of partcular terest are: the frst ad secod momets of a sgle radom varable ad the ot momets, such as the correlato ad covarace, betwee ay par of radom varables a multdmesoal set of radom varables. The mea or epected value of X sgle radom varable the frst momet - s E X m p d.-6 where E deotes epectato statstcal averagg. I geeral, the th momet s defed as E X p d.-6 Now, suppose that we defe a radom varable YgX, where gx s some arbtrary fucto of the radom varable X. The epected value of Y s E Y E[ g X ] g p d.-63 I partcular, f Y X m, the ] m p E Y E[ X m d.-64 Ths epected value s called th cetral momet of the radom varable X.
Whe, the cetral momet s called varace m p σ d.-65 Ths parameter provdes a measure of the dsperso of the radom varable X. σ E.-66 X [ E X ] E X m The ot momet of X ad X s k k X p, d E X d.-67 The ot cetral momet s E[ X k m X k m m ] m p, d d.-68 Of partcular mportace to us are the ot momet ad ot cetral momet correspodg to k. These ot momets are called the correlato ad the covarace of the radom varables X ad X, respectvely. I cosderg multdmesoal radom varables, we ca defe ot momets of ay order. However, the momets that are most useful practcal applcatos are the correlatos ad covaraces betwee pars of radom varables. To elaborate suppose that X,,,,, are radom varables wth ot PDF p,,. Let p, be the ot PDF of the radom varables X ad X. The the correlato betwee X ad X s gve by the ot momet Ad the covarace of X ad X s p, dd E X X.-69 µ E[ X m m X m m ] p, d d E X X m m.-70
The matr wth elemets µ s called the covarace matr of the radom varables X,,,,. Two radom varables are sad to be ucorrelated f E X X m m. I that case, the covarace µ 0. We ote that whe X ad X are statstcally depedet, they are also ucorrelated. However, f X ad X are ucorrelated, they are ot ecessarly statstcally depedet. Two radom varables are sad to be orthogoal f E X X 0. We ote that ths codto holds whe X ad X are ucorrelated ad ether oe or both of the radom varables have zero mea.
..4 Some Useful Probablty Dstrbutos Several types of radom varables. Bomal dstrbuto Let X be a dscrete radom varable, that has two possble values, say X or X0, wth probabltes p ad -p. Now, suppose that Y X Fg..-6 The probablty dstrbuto fucto of X What s the PDF of Y? To aswer ths questo, we observe that rage of Y s the set of tegers from 0 to. The probablty that Y0 s smply the probablty that all the X 0. Sce X are statstcally depedet, P Y 0 p The probablty that Y s smply the probablty that oe X ad the rest of the X 0. Sce ths evet ca occur dfferet ways, P Y p p To geeralze, the probablty that Yk s the probablty that k of the X are equal to oe ad -k are equal to zero. Sce there are k!.-84 k! k! dfferet combatos that result the evet {Yk}, t follows that P k k Y k p p.-85 k Cosequetly, the PDF of Y may be epressed as p y k 0 P Y k k k δ y k p p δ y k.-86 k k 0
Uform dstrbuto Fg..-7 The pdf ad cdf of a uformly dstrbuted radom varable The frst two momets of X are 3 b a ab b a X E b a X E + + + σ.-90 Gaussa ormal dstrbuto The PDF s / σ πσ m e p.-9 where s the mea ad s the varace of the radom varable. m σ The CDF s / / σ π πσ σ σ t m m u m erf dt e du e du u p F +.-93 where erf deotes the error fucto, defed as dt e erf t 0 π.-94
Fg..-8 [ote p tet f fgure] The fucto that s frequetly used for the area uder the tal of the Gaussa PDF s deoted by Q ad defed as Q π e t / dt.-97 Q erfc.-98
. STOCHASTIC PROCESSES The sgal at the output of a source that geerates formato s characterzed as a radom sgal that vares wth tme a audo sgal, vdeo sgal,. All these are eamples of stochastc radom processes. I our study of dgtal commucatos, we ecouter stochastc processes all blocs of the dgtal commucato system. We shall deote such a process by Xt. I geeral, the parameter t s cotuous, whereas X may be ether cotuous or dscrete, depedg o the characterstcs of the source that geerates the stochastc process. The ose voltage geerated by a sgle resstor or a sgle formato source represets a sgle realzato of the stochastc process. Hece, t s called a sample fucto of the stochastc process. The set of all possble sample fuctos, e.g. the set of all ose voltage waveforms geerated by resstors, costtutes a esemble of sample fuctos or, equvaletly, the stochastc process Xt. We may cosder the values of the process at ay set of tme stats t >t >t 3 > >t Where s ay postve teger. I geeral, the radom varables X t are characterzed statstcally by ther ot PDF. Statoary stochastc processes p + t,,...,,,..., t t p t + t t + t t t.- The stochastc process s sad to be statoary the strct sese. That s, the statstcs of a statoary stochastc process are varat to ay traslato of the tme as. O the other had, whe the ot PDFs are dfferet, the stochastc process s ostatoary... Statstcal Averages We may smlarly defe statstcal averages for a stochastc process esemble averages. The th momet of the radom varable X t s defed as X t E p d.- The ot momet correlato of two radom varables t t t E X t X t, t t p t t d t d t.-3 Autocorrelato fucto of the stochastc process E X t X t φ t, t φ t t φ.-4 τ The autocorrelato fucto of Xt does ot deped o the specfc tme stats t
ad t but, stead, t depeds o the tme dfferece t t. We ote that φ 0 E deotes the average power the process Xt! X t Autocovarace fucto of the stochastc process s µ t, t E{[ X m t ][ X m t ]} φ t, t m t m.-5 t t t Whe the process s statoary µ t, t µ t t µ τ φ τ m.-6 Averages for a Gausso process Suppose that Xt s a Gaussa radom process wth mea values autocovaraces. m t, ad µ t, t E[ X m t X m t ].-7 t t The Gaussa process s completely specfed by the mea ad autocovarace fuctos... Power Desty Spectrum The frequecy cotet of a sgal s a very basc characterstc that dstgushes oe sgal from aother. I geeral, a sgal ca be classfed as havg ether a fte ozero average power fte eergy or fte eergy. The frequecy cotet of a fte eergy sgal s obtaed as the Fourer trasform of the correspodg tme fucto. If the sgal s perodc, ts eergy s fte ad, cosequetly, ts Fourer trasform does ot est. The mechasm for dealg wth perodc sgals s to represet them a Fourer seres. Wth such a represetato, the Fourer coeffcets determe the dstrbuto of power at the varous dscrete frequecy compoets. A statoary stochastc process s a fte eergy sgal, ad, hece, ts Fourer trasform does ot est. The spectral characterstc of a stochastc sgal s obtaed by computg the Fourer trasform of the autocorrelato fucto. That s, the dstrbuto of power wth frequecy s gve by the fucto τ f φ τ e π f Φ dτ.-6 The verse Fourer trasform
We observe that πfτ φ τ Φ f e df.-7 φ 0 Φ f df E 0.-8 X t Sce φ 0 represets the average power of the stochastc sgal, whch the area uder Φ f, Φ f s the dstrbuto of power as a fucto of frequecy. Therefore, Φ f s called the power desty spectrum of the stochastc process...3 Respose of a Lear Tme-Ivarat System Chael to a Radom Iput Sgal Its mpulse respose s ht or equvaletly, ts frequecy respose s Hf. Let t be the put sgal. The output of the system s y t h τ t τ dτ.-4 Now, suppose that t s a sample fucto of a statoary stochastc process Xt. The, the output yt s a sample fucto of a stochastc process Yt. We wsh to determe the mea ad autocorrelato fucto of the output. The mea value of Yt s m y m E[ Y t] h τ dτ m H 0 h τ E[ X t τ ] dτ.-5 where H0 s the frequecy respose of the lear system at f0. Hece, the mea value of the output process s a costat. The power desty spectrum of the output process s Φ yy f Φ f H f.-7
Thus, we have the mportat result that the power desty spectrum of the output sgal s the product of the power desty spectrum of the put multpled by the magtude squared of the frequecy respose of the system! The autocorrelato fucto s πfτ πfτ φyy τ Φ yy f e df Φ f H f e df.-8 We observe that the average power the output sgal s Eample.- φ yy 0 Φ f H f df.-9 Suppose that the low-pass flter llustrated Fg..- s ected by a stochastc process t havg a power desty spectrum Φ f N0 for all f
..4 Samplg Theorem for Bad-Lmted Stochastc Processes The bad-lmted sgal sampled at the Nyqust rate ca be recostructed from ts samples by use of the terpolato formula s t s[πw t ] W πw t W s W.-35 Fg..-4 Sgal recostructo based o deal terpolato A statoary stochastc process Xt s sad to be bad-lmted f ts power desty spectrum Φ f 0 for f > W. Sce Φ f s the Fourer trasform of the autocorrelato fucto φ τ, t follows that s[πw τ ] φ τ φ W.-36 W πw τ W Now, f Xt s a bad-lmted statoary stochastc process, the Xt ca be represeted as X t s[πw t ] W πw t W X W.-37 Ths s the samplg represetato for a statoary stochastc process.
..5 Dscrete-Tme Stochastc Sgals ad Systems A dscrete-tme stochastc proces X cossts of a esemble of sample sequeces {}. The statstcal propertes of X are smlar to the characterzato of Xt wth the restrcto that s ow a teger tme varable. As the case of cotuous-tme stochastc processes, a dscrete-tme statoary process has fte eergy but a fte average power, whch s gve as E φ0.-43 X The power desty spectrum for the dscrete-tme process s obtaed by computg the Fourer trasform of φ. The Fourer trasform s defed as πf Φ f φ e.-44 Fally, let us cosder the respose of a dscrete-tme, lear tme-varat system to a statoary stochastc put sgal. The system s characterzed the tme doma by ts ut sample respoe h ad the frequecy doma by the frequecy respose Hf, where H f πf h e.-46 The respose of the system to the statoary stochastc put sgal X s gve by the covoluto sum k y h k k.-47 The mea value of the output of the system s m y m E[ y ] k k h k m H 0 h k E[ k].-48 where H0 s the zero frequecy [drect curret DC] ga of the system. The autocorrelato sequece for the output process s
φ k yy * E[ y y + k] * * h h E[ + k * h h φ k + ].-49 Ths s the geeral form for the autocorrelato sequece of the system output terms of the autocorrelato of the system put ad the ut sample respose of the system. By takg the Fourer trasform of φyyk ad substtutg the relato Eq..-49, we obta the correspodg frequecy doma relatoshp Φ yy f Φ f H f.-50 Whch s detcal to Eq..-7 ecept that Eq..-50 the power desty spectra Φ ad Φ f ad the frequecy respose Hf are perodc fuctos of frequecy yy f wth perod fp...6 Cyclostatoary Processes I dealg wth sgals that carry dgtal formato we ecouter stochastc processes that have statstcal averages that are perodc. To be specfc, let us cosder a stochastc process of the form X t a g t T.-5 where {a } s dscrete-tme sequece of radom varables wth mea m E a all ad autocorrelato sequece φ aa k E aa+ k. The sgal gt s determstc. The stochastc process Xt represets the sgal for several dfferet types of lear modulato techques troduced later. The sequece {a } represets the dgtal formato sequece of symbol that s trasmtted over the commucato chael ad /T represets the rate of trasmsso of the formato symbols. Let us determe the mea of Xt s tme-varyg, perodc wth perod T a for
E[ X t] m a g t T E a g t T.-5 The autocorrelato fucto of Xt s also perodc wth perod T φ t + τ, t φaa m E[ X t + τ X m g t] t T g t + τ mt.-53 Such a stochastc process s called cyclostatoary or perodcally statoary. Sce the autocorrelato fucto depeds o both the varables t ad τ, ts frequecy doma represetato requres the use of a two-dmeso Fourer trasform. Sce t s hghly desrable to characterze such sgals by ther power desty spectrum, a alteratve approach s to compute the tme-average autocorrelato fucto over a sgle perod, defed as T / φ τ φ t + τ, t dτ.-55 T T / Thus, we elmate the tme depedecy by dealg wth the average autocorrelato fucto. Now, the Fourer trasform of φ τ yelds the average power desty spectrum of the cyclostatoary stochastc process. Ths approach allows us to smply characterze cyclostatoary processes the frequecy doma terms of the power spectrum. That s, the power desty spectrum s Φ τ φ τ π f f e dτ.-56
SELECTED PROBLEMS:. Oe epermet has four mutually eclusve outcomes A,,,3,4, ad a secod epermet has three mutually eclusve outcomes B,,,3. The ot probabltes PA,B are PA,B 0,0 PA,B 0,08 PA,B 3 0,3 PA,B 0,05 PA,B 0,03 PA,B 3 0,09 PA 3,B 0,05 PA 3,B 0, PA 3,B 3 0,4 PA 4,B 0, PA 4,B 0,04 PA 4,B 3 0,06 Determe the probabltes PA,,,3,4, ad PB,,,3. Sol:. The autocorrelato fucto of a stochastc proces Xt s φ τ N0δ τ Such a process s called whte ose. Suppose t s the put to a deal bad-pass flter badwdth B [Hz] havg the frequecy respose characterstc Hf as show Fg. Determe the total ose power at the output of the flter.
Sol:.9 Determe the mea, the autocorrelato sequece, ad the power desty spetrum of the output of a system wth ut sample respose h 0 0 othrewse Whe the put s a whte-ose process wth power desty spectrum σ. Sol:
.0 k The autocorrelato sequece of a dscrete-tme stochastc proces s φ k. Determe ts power desty spectrum. Sol:. Cosder a bad-lmted zero-mea statoary stochastc Xt wth power desty spectrum f W Φ f 0 f > W Xt s sampled at a rate f s /T to yeld a dscrete-tme process X X T. a determe the epresso for the autocorrelato sequece of X. b determe the mmum value of T that results a whte spectrally flat sequece c repeat b f the power desty spectrum of Xt s Φ f f / W 0 f f W > W
Sol: