58 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY Xin Li, Tan F. Wong, and John M. Shea

Size: px
Start display at page:

Download "58 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY Xin Li, Tan F. Wong, and John M. Shea"

Transcription

1 58 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY 2008 Performance Analyss for Collaboratve Decodng wth Least-Relable-Bts Exchange on AWGN Channels Xn L, Tan F. Wong, and John M. Shea Abstract Collaboratve decodng s an approach that can acheve dversty and combnng gan by exchangng decodng nformaton among a cluster of physcally separated recevers. On AWGN channels, the least-relable-bts LRB exchange scheme can acheve performance close to equal-gan combnng EGC of all receved symbols from all recevers, whle reducng the amount of nformaton that must be exchanged. In ths paper, we analyze the error performance of collaboratve decodng wth the LRB exchange scheme when nonrecursve convolutonal codes are used. The analyss s based on the observaton that the extrnsc nformaton generated n the collaboratve decodng of these convolutonal codes can be approxmated by Gaussan random varables. A densty-evoluton model based on a sngle maxmum a posteror decoder s used to obtan the statstcal characterstcs of the extrnsc nformaton. Wth the statstcal nowledge of the extrnsc nformaton, we develop an approxmate upper bound for the error performance of the collaboratve decodng process. Numercal results show that our analyss gves very good predctons of the bt error rate for collaboratve decodng wth LRB exchange. At hgh sgnal-to-nose ratos collaboratve decodng wth properly chosen parameters can acheve the same error performance as EGC of all receved symbols from all recevng nodes. Index Terms Collaboratve decodng, dstrbuted array, densty evoluton, teratve decodng analyss. I. INTRODUCTION COLLABORATIVE decodng wth a dstrbuted array, orgnally proposed n [1] [3], allows a group of physcally separated recevers called nodes to collaborate n ther decodng process to obtan dversty and combnng gan. In the collaboratve decodng process, these nodes exchange a small porton of ther decodng nformaton wth each other, and the nformaton from other nodes s used as a pror nformaton n future decodng. The exchange and decodng process s then repeated n an teratve fashon. An nformaton exchange scheme named least-relable-bts LRB exchange was also proposed for collaboratng decodng n [3]. The LRB exchange scheme allows each node n the dstrbuted array to request extrnsc nformaton from other nodes for Paper approved by A. H. Banhashem, the Edtor for Codng and Communcaton Theory of the IEEE Communcatons Socety. Manuscrpt receved October 6, 2004; revsed September 26, Ths wor was supported by the Natonal Scence Foundaton under Grants ANI and CNS and by the Offce of Naval Research under Grant N The authors are wth the Wreless Informaton Networng Group, Department of Electrcal & Computer Engneerng, Unversty of Florda, Ganesvlle, Florda , USA e-mal: lx@ecel.ufl.edu, twong, jshea}@ece.ufl.edu. Dgtal Object Identfer /TCOMM /08$25.00 c 2008 IEEE ts unrelable decoded nformaton bts n each teraton. It was demonstrated n [3] that on AWGN channels, collaboratve decodng wth the LRB exchange scheme can provde sgnfcant savngs on the cost of nformaton exchange whle stll achevng performance close to that of EGC 1. Further, the collaboratve decodng technque was nvestgated n [4] for spectral-effcent transmssons wth hgh-order modulatons. Due to the exchange of extrnsc nformaton n the process, nowledge of the statstcal characterstcs of extrnsc nformaton from maxmum a posteror MAP decoders n collaboratve decodng s mportant to ts performance analyss. From smulaton, we observe that the extrnsc nformaton generated n the decodng process can be well approxmated by Gaussan random varables when nonrecursve convolutonal codes are employed. By vewng collaboratve decodng as an teratve decodng system, we use a typcal analyss technque for turbo-le codes, nown as densty evoluton, to analyze the performance of collaboratve decodng. As n most of the lterature e.g., [5] and [6] on analyss of turbo codes, we use smulaton to obtan the statstcal characterstcs of the extrnsc nformaton, whch s approxmated by a Gaussan dstrbuton. To smplfy the problem, we model the collaboratve decodng process as a densty evoluton system wth only one MAP decoder. Then we can generate the aprornformaton of the denstyevoluton model accordng to the LRB exchange scheme. By smulatng the densty-evoluton model wth only one MAP decoder, we obtan the statstcal characterstcs of the actual extrnsc nformaton wth a modest smulaton load n comparson to that of the actual collaboratve decodng system. Wth the nowledge of the extrnsc nformaton probablty dstrbuton at each teraton, we derve an approxmate bterror rate BER upper bound for collaboratve decodng wth the LRB exchange scheme. II. SYSTEM DESCRIPTION The model of collaboratve decodng wth a dstrbuted array proposed n [1] [3] s shown n Fg. 1. A remote source transmts a message through a sngle-nput/multple-output forward channel to the destnaton that contans M M 2 physcally separated recevng nodes, denoted by a node set M = 1, 2,,M}. The source encodes and transmts the message wth a non-recursve convolutonal code and 1 Here and n all that follows, EGC s used to refer to equal-gan combnng of all of the receved symbols from all of the recevng nodes.

2 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 59 Source Fg. 1. Forward Channel Dstrbuted array system model. Dstrbuted Array BPSK modulaton. Intally, all the nodes perform coherent demodulaton and decode the receved message ndvdually. Each node n M use the max-log-map decodng algorthm [7] to process the receved symbols. The channels from the source to the recevng nodes are ndependent dentcally dstrbuted..d. transmsson channels. Meanwhle, the recevng nodes form a local networ such that they can communcate wth one another on a broadcast channel, whch we model as an error-free channel. Ths assumpton s reasonable when the SNR of the broadcast channel among the recevng nodes s sgnfcantly hgher than that of the forward channel from the source to the recevng nodes. The performance of the forward channel s the man concern of ths paper. The forward channels are nonfadng addtve whte Gaussan nose AWGN channels. Let y denote the receved sgnal at the th recevng nodes correspondng to the transmtted BPSK sgnal x.e., x +1, 1} at tme nstant. Then y = x + n, where n for Mand all are..d. zero-mean Gaussan random varables wth varance E[ n 2 ]=σn. 2 The dea of the LRB exchange scheme s that each node requests extrnsc nformaton from other nodes for ts leastrelable decoded nformaton bts [3]. Ths approach s smlar also to ndependently developed technques on relabltybased schedulng for decodng on graphs [8] n whch the relablty of nformaton s used to determne whether t should be propagated n the graph durng a partcular decodng teraton. We note, however, that the approach descrbed n [8] s more smlar to the most-relable bts MRB approach descrbed n [3] than the LRB approach analyzed n ths paper. All the nformaton collected from other nodes, called addtonal nformaton, s used as a pror nformaton n the next decodng teraton. For AWGN channels, EGC s used on the extrnsc nformaton receved from other nodes, so the addtonal nformaton for each bt s the sum of the extrnsc nformaton receved for that bt. To weaen the correlaton among the addtonal nformaton at dfferent teratons, once nformaton about a bt has been exchanged, no further nformaton about that bt wll be exchanged n later teratons. In each teraton, the bts for whch nformaton has not been prevously exchanged are called canddate bts, and the remanng bts are non-canddate bts. We denote the total number of exchanges by I, and the fracton of canddate bts to exchange n the jth 0 j I 1 teraton by p j 0 p j 1, respectvely. The analyss s based on the followng procedures of the LRB exchange scheme: 1 Set all nformaton bts to be canddate bts. 2 Decode the receved sgnals at each node. 3 If I +1 decodng teratons.e., I exchanges have been performed, then stop the decodng procedure and go to step 1 to process a new pacet. 4 Otherwse, each node rans the canddate bts accordng to ther soft output magntude absolute value of the soft output and requests soft nformaton for the bottom p j fracton of the canddate bts the least relable canddate bts from other nodes. 5 Each node broadcasts the soft output for those bts that are requested by other nodes. 6 Those bts nvolved receved and broadcast n the current exchange are set to be non-canddate bts for later teratons. 7 Each node adds the nformaton from other nodes to ts aprornformaton and returns to step 2. Steps 4 and 5 requre nformaton exchange among the recevers n the cluster. These exchanges may be performed followng a round-robn schedule based on an orderng decded pror to the collaboratve process. The requests transmtted n step 4 wll generally consst of source-encoded bt ndces for the least relable bts. As shown n prevous wor on relablty-based hybrd ARQ, the postons of the unrelable bts are hghly correlated at the output of a MAP decoder for convolutonally coded data, and a varety of compresson technques cf. [9], [10] can be used to reduce the sze of the nformaton request pacets. The soft nformaton transmtted n step 5 wll typcally be quantzed for transmsson wth dgtal modulaton. To provde performance close to no quantzaton, fve-bt quantzaton has been shown to be suffcent for teratve decodng applcatons [11], [12]. The fracton of canddate bts to be exchanged n each teraton, p j } I 1 j=0, are the desgn parameters, whch are set n advance. In ths paper, the focus s on evaluatng the error performance for collaboratve decodng gven p j } I 1 j=0 and the number of nodes, M. We use the analyss to compare the performance of several dfferent choces of p j } I 1 j=0.afull study of the desgn of these parameters s an nterestng topc but s outsde the scope of ths paper. Note that the soft output for non-systematc codes conssts only of extrnsc nformaton and aprornformaton; f a canddate bt has not obtaned addtonal nformaton prevously, then ranng and exchangng the soft output for canddate bts s equvalent to ranng and exchangng the extrnsc nformaton for those bts. Also, the sets of canddate bts and non-canddate bts for a pacet n each teraton are exactly the same for all nodes. These facts are mportant to understand the analyss n the followng sectons. III. GAUSSIAN-APPROXIMATED DENSITY EVOLUTION Because of the symmetry among the nodes n our system model, the statstcal characterstcs of the extrnsc nformaton at each node are the same n a partcular teraton. Thus the behavor of the LRB exchange process can be determned by nowng the statstcal characterstcs of the output from the MAP decoder at a sngle node. The collaboratve decodng

3 60 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY 2008 Fg. 2. Probablty densty Addtonal Info Intrnsc Info Info Exchange Addtonal Info Generaton MAP Decoder System model for collaboratve decodng process. 1st teraton 2nd teraton Extrnsc Info 3rd teraton 4th teraton Hstogram Ideal Gaussan pdf Extrnsc nformaton Fg. 3. Emprcal pdfs of extrnsc nformaton generated by the MAP decoders n successve teratons n collaboratve decodng wth the LRB exchange for M =6and E b /N 0 =3dB on AWGN channels, where the maxmum free-dstance 4-state nonrecursve convolutonal code s used. process can be modeled by the jont operaton of an nformaton exchange unt and a MAP decoder unt as shown n Fg. 2. The output of the nformaton exchange s fed bac to the MAP decoder as aprornformaton for use n the next decodng teraton. The followng analyss s based on ths system model. Assumng that the all-zero codeword s transmtted, t s well nown that the extrnsc nformaton generated by a MAP decoder, n the log-lelhood rato LLR form, s well approxmated by Gaussan random varables when the nputs to the decoder are..d. Gaussan [5]. For the collaboratve decodng process descrbed n Secton II, the addtonal nformaton obtaned from the nformaton exchangng process has a non-gaussan dstrbuton. Nevertheless, we observe that the probablty dstrbuton of the extrnsc nformaton from the MAP decoder n each teraton s stll well approxmated as Gaussan when nonrecursve convolutonal codes are employed. Fg. 3 shows the typcal hstograms of the extrnsc nformaton generated by MAP decoders at successve teratons n the collaboratve decodng process for nonrecursve convolutonal codes. The hstograms are very close to Gaussan dstrbutons. Thus we apply the Gaussanapproxmated densty-evoluton technque n [5],[6] to predct the behavor of the MAP decoders n collaboratve decodng. As n [5], we assume that at each node the extrnsc nformaton generated by the MAP decoder for all the nformaton bts at that node are..d. Gaussan random varables n each teraton. We further assume that extrnsc nformaton generated by dfferent nodes are ndependent. Thus, the statstcal behavor of the extrnsc nformaton s suffcently specfed by ts mean and varance. Unfortunately, obtanng an analytc dstrbuton for the extrnsc nformaton generated by MAP decoders s an ntractable problem, especally for non-gaussan nputs. Hence, we use smulaton based on the model n Fg. 2 to quantfy the evoluton of the probablty dstrbuton. By nputtng actual addtonal nformaton to the MAP decoder, the mean and varance of the extrnsc nformaton can be obtaned wth modest smulaton complexty n comparson to the actual collaboratve decodng process. Ths nowledge of extrnsc nformaton s used to evaluate the error performance n Secton IV. We frst descrbe the generaton of the addtonal nformaton. For the jth decodng teraton, let ξ j denote the extrnsc nformaton generated by the MAP decoder for the th nformaton bt at node, and let B j denote the event that bt s a canddate bt. Under the Gaussan assumpton, ξ j N µ j,σj 2, and ξj } are..d. for all and, where N µ, σ2 means Gaussan dstrbuted wth mean µ and varance σ 2. When the bloc sze s large enough, the nformaton request crteron for ξ j to ran n the bottom p j fracton among the canddate bts n the th node s approxmately equvalent to ξ j T j, where T j 0 s a threshold related to the dstrbuton of ξ j and p j by P ξ j j B = p j. 1 T j Let λ j denote the addtonal nformaton for the th bt at the th node generated by the LRB exchange process n the jth teraton. Ths addtonal nformaton wll be added to the aprornformaton n the j +1th teraton by node. Below, we consder M 3; we dscuss M =2later. In the LRB scheme, f bt s a non-canddate bt n the jth teraton, then λ j =0. Otherwse, there are three possbltes for a canddate bt: No node requests nformaton for the th bt,.e., t M ξj t, >T j, n whch case λ j =0; The th node does not request nformaton for bt, but there s one other node requestng nformaton for that bt. We denote ths event by Ṙj,.e., Ṙ j = r M,r ξ j r, T j, t M,t r ξ j t, >T j }. 2 Then the th node obtans nformaton from other nodes except the requester,.e., λ j = ξ j j t, λ ; 3 t M t r,t,r The th node or more than one node n M requests j nformaton for bt. We denote ths event by R, as shown n 4. In ths case, the th node wll obtan

4 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 61 R j = ξj T j} ξ j >T j} r M,r ξ j r, T j, t M,t r,t ξ j t, >T j }}. 4 nformaton from all other nodes, and λ j λ j = t M,t ξ j t, s j λ. 5 Under the Gaussan assumpton, we see that wthout the j constrant of canddate bts, λ N M 2µ j, M 2σj 2 j and λ N M 1µ j, M 1σj 2. Clearly, Ṙ j j and R are dsjont. In the LRB scheme, only under case wll bt be a canddate bt agan n the next teraton. Hence, P B j+1 B j = P ξ j B >Tj j =1 p j M. 6 M From ths recursve relaton, we mmedately obtan and P B j = P B j, B,, B 0 = P B l+1 B l = 1 p l M, 7 B j = M ξ l >T l}. 8 Unle case, n cases and bt wll be a non-canddate bt n the next teraton. Thus, P B j+1 B j Ṙj j = P R B j = P Ṙ j B j + P Rj B j. 9 Wth the above arguments, we can smulate the addtonal nformaton generated n the actual LRB process for the densty-evoluton model n Fg. 3. Wthout loss of generalty, we assume that the MAP decoder n Fg. 3 s n the Mth node. Also, we assume that the bloc length of the code s long enough to ensure valdty of the Gaussan and threshold approxmatons. In the jth teraton, the MAP decoder generates ξ j M, for the th bt. Then the values of µ j and σj 2 of the extrnsc nformaton are estmated. To fnd T j usng 1, we frst use a nonparametrc estmaton method to estmate the cumulatve dstrbuton functon F j x of the extrnsc nformaton for the canddate bts,.e., F j x = P ξ j <x j B. Then, accordng to 1 we have p j = F j T j F j T j.forp j < 1, the approxmate value of T j s found by solvng ths equaton numercally. For p j =1, we set T j =. In the nformaton exchange module, we use M 1..d. random varables followng dstrbuton F j x to smulate the extrnsc nformaton ξ j for canddate bt at node, for =1, 2,,M 1, respectvely. Then, wth ξ j }, for all Mwe chec f Ṙj M, or Rj M, occurs, set λ j j j M, to λ M, or λ M, accordngly, and flag ths bt as a non-canddate bt for the next teraton. For all other cases, we set λ j M, =0. Then based on the LRB scheme we construct the aprornformaton, denoted by η j+1 M,,forthej +1th teraton, as j η j+1 M, = λ l M, = λ l M, f R l M,, 0 l j 0 otherwse, 10 where R l l M, Ṙl M, R M,. By nputtng ths aprornformaton to the MAP decoder and teratng the above procedure, we can obtan the statstcal characterstcs of the Gaussanapproxmated extrnsc nformaton for the whole collaboratve decodng process. Fg. 4 shows the mean and varance of the extrnsc nformaton and the threshold T j estmated n our densty-evoluton model along wth smulatons results for the actual collaboratve decodng process for M =6. In the fgure, the maxmum free-dstance, 4-state non-recursve convoluton code s used, and p j } = 0.1, 0.15, 0.25}. The results show that our densty-evoluton model well approxmates the actual collaboratve decodng process wth only 1/M th of the smulaton load. In the next secton, we show that to evaluate the error performance for I teratons, we only need statstcal nowledge of the extrnsc nformaton n the frst I 1 teratons. IV. ERROR PERFORMANCE ANALYSIS Wth nowledge of the statstcal characterstcs of the extrnsc nformaton, we evaluate the error performance of collaboratve decodng wth LRB exchange. We agan consder the decodng process and performance at the Mth node. Let M = 1, 2,,M 1} denote the set of the other M 1 nodes. Snce the average BER s consdered, we drop the bt ndex,.e., the subscrpt, n the notaton of varables and events for the bt of nterest. For convenence, we also drop the subscrpt M for the Mth node n the followng dervaton. From the defnton 3, we now that λj M, s a Gaussan random varable for M 3 but equals zero for M =2. Thus we treat M =2as a specal case and consder M 3 frst below. A. BER Upper Bound for M 3 For M 3, the BER of the MAP decoders n the jth j >0 teraton s the probablty that the soft output of a bt s smaller than zero gven that the all-zero sequence s transmtted,.e., P j b = P ξ j + η j < 0, 11 where ξ j s the extrnsc nformaton, and η j s the apror nformaton n the jth teraton gven n 10 at the M th node, respectvely. Here, we evaluate the error performance by fndng an upper bound for 11. Accordng to 10 and 9, we rewrte 11 as P j b = P ξ j + η j < 0, B j +Pξ j < 0, B j = P ξ j + λ l < 0, R l, B l +Pξ j < 0, B j. 12

5 62 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY Densty evoluton model Actual collaboratve decodng varance 4th ter. 4th ter. 1st ter. mean 1st ter E /N db b Densty evoluton model Actual collaboratve decodng a Mean and varance 1st ter. 3rd ter. 2nd ter E /N db b 0 b Threshold Fg. 4. Mean and varance of the extrnsc nformaton and the threshold estmated from the densty-evoluton model. They are compared wth those from the actual collaboratve decodng process. We frst consder the summand n 12. Usng 9, 3 and 5, we have P ξ j + λ l < 0, R l, B l = P ξ j + λ l < 0, Ṙl, B l + P ξ j + λ l < 0, R l, B l P ξ j + λ l < 0, Ṙl, B l + P ξ j + λ l < Wth 2 and 8, when p t < 1.e., T t < for0 t l,we upper bound the frst term n 13 as shown n 14 and 15, where a s obtaned by droppng all the events n Ṙl, B l } assocated wth ξ t for all t and M, r, and b s due to the probabltes n the sum n 14 beng equal for 1 r M 1 and to ξ j and λ l beng ndependent of ξ r t for all t. To evaluate P ξ l 1 T l, ξt 1 >T t n 15, we use 8 to rewrte P B j as M P B j = P = = M P =1 =1 [ P } ξ l >T l ξ l >T l ξ l >T l] M, M. 16 By comparng 16 wth 7, for all Mwe have P ξ l >T l = 1 p l. 17 In a smlar manner, t s easy to see that for all M P ξ l T l, ξ t >T t = p l 1 p t. 18 Thus, wth 18 and tang nto account the fact P ξ j + λ l < 0, Ṙl, B l P ξ j + λ l < 0, we refne the upper bound 15 as shown n 19. Ths bound s for the case that all p t are not equal to 1. If there exsts a 0 t l such that p t =1, then P ξ j + λ l < 0, Ṙl, B l =0because P Ṙl, B l =0. To nclude ths case, we rewrte the upper bound 19 as P ξ j + λ l < 0, Ṙl, B l a l P ξ j + λ l < 0, 20 where the value of a l s gven n 21. In the same way, we consder the probablty P ξ j < 0, B j n 12. Wth 7 and 17, ths probablty can be easly expanded and upper bounded as shown n 22, where b j = 1 p l M By nsertng 20, 22 and 13 nto 12, we obtan followng upper bound P j b [ a l P ξ j + λ l < 0 + P ξ j + λ ] l < 0 + b j [ P ξ < T +P ξ j < 0,ξ >T ], 24 where a l and b j are gven by 21 and 23, respectvely. Below, we employ a unon bound for the max-log-map decoder to further upper bound the probabltes n 24. B. Unon Bound for Max-log-MAP Decodng Let u and c denote an nformaton bt sequence and the correspondng codeword generated by a nonrecursve convolutonal code C: u c, where u = u 0,u 1,,u,, c = c 0,c 1,,c,, and u and c 0, 1} are the nformaton bt and coded bt, respectvely. Correspondngly, y s the receved BPSK sgnal at the decoder. Under the assumpton that the all-zero sequence s transmtted, the extrnsc

6 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 63 P ξ j + λ l < 0, Ṙl, B l =P ξ j + λ l < 0, = a M 1 r=1 M 1 r=1 P ξ j + λ l < 0, ξ r l T l, r M t M,t r P ξ j + λ l < 0, ξ r l T l, b = M 1P ξ j + λ l < 0P ξ l r T l, ξ l t >T l, ξ t r >T t ξ l 1 T l, ξ t 1 >T t t M,t r ξ l M } t >T l, B l >T t} ξ t P ξ j + λ } l < 0, Ṙl, B l mn 1, M 1p l 1 p t P ξ j + λ l < 0 19 l 0 f a l = 1 p } mn 1, M 1p l 1 p t otherwse 21 P ξ j < 0, B j = P ξ j < 0, M = P ξ j < 0, ξ l >T l ξ l >T l P M 1 =1 ξ l >T l P ξ j < 0, ξ >T 1 p l M 1 b j [ P ξ < T +P ξ j < 0,ξ >T ] 22 nformaton generated by the max-log-map decoder n the LLRformsgvenby[7] ξ j = max Γ j u,c} + mn Γ j u,c}, 25 u,c C + u,c C where C + and C are the sets of all codeword pars u, c that gves the decson of u 0 =0and u 0 =1, respectvely, and Γ j u,c s the error event metrc for u, c n the jth teraton, defned as Γ j u,c = + L c y. 26 :u =1} η j :c =1} In 26, : u =1} and : c =1} mean tang the ndces of the non-zero bts n u and c; η j s the a pror nformaton of the th nformaton bt; and L c =2/σn 2 s the channel relablty measure. Note that snce the all-zero codeword 0, 0 C + and Γ j 0,0 =0,wehave max Γ j u,c} = max 0, Γ j u,c} u,c C + u,c C + Wth 27 and 25, we can obtan the unon bound n 28, where K c s the number of nput bts per trells state transton. Let d=wc and w= wu denote the Hammng weghts of the codeword c and the correspondng nformaton bt sequence u, respectvely. Snce the error event metrc Γ j u,c n 26 does not depend on the codeword pattern and, but only on w and d, we can wrte the metrc as w 1 Γ j = η j =1 d 1 + L c y, 29 where we have consdered the bt u 0 n u wthout loss of generalty. Thus, by usng 29 and droppng the subscrpt, the unon bound n 28 can be wrtten as P ξ j <x 1 K c d d mn w 1 =0 wa P Γ j <x, 30 where d mn s the mnmum Hammng dstance of the code C, and A s the number of error events wth Hammng weght d and nput weght of w. Eq. 30 s a generalzed unon bound for max-log-map decodng. The well nown unon bound for maxmum lelhood decodng s a specal case of 30 wth x =0and the aprornformaton equal to 0. C. Applyng Max-log-MAP Decodng Unon Bound to Collaboratve Decodng To apply the generalzed unon bound n 30 to collaboratve decodng, the crucal step s the evaluaton of the probablty P Γ j < x. Thus we study the error event

7 64 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY 2008 P ξ j <x = P max Γ j u,c} + mn u,c C + } P mn Γ j u,c C u,c <x = 1 P K c u,c C Γ j } u,c <x u,c <x u,c C Γ j 1 P Γ j u,c <x 28 K c u,c C metrc n 29. From 10, we now that n the jth decodng teraton, not all the w 1 non-zero nformaton bts n u the frst non-zero bt u 0 tself s excluded necessarly obtan aprornformaton. Among those bts that obtan apror λ l λ l nformaton, some obtan whle others obtan for l<j. For convenence, we use A l to denote the bt set n the w 1 nonzero bts of u that obtan addtonal nformaton λ l n the lth teraton for l<j. Ths means that n teraton l, the event R l occurs only for A l for convenence, we do not dstngush between a bt and ts bt ndex. Furthermore, Ȧ l and Äl are subsets that partton A l such that the events Ṙ l l and R occur for Ȧl and Äl, respectvely. Thus, bts n Ȧl and Äl l l obtan addtonal nformaton λ and λ, respectvely. Note that no nformaton can be exchanged for a non-canddate bt, so A l A = for l. Then the above event can be expressed as V j = Ȧl Ṙ l, Äl R l, B l }, } B j, 31 A l B j where B j = A l s the bt set for whch no nformaton exchange occurs n the prevous j 1 teratons. The set B j contans all the non-zero canddate bts left for the jth decodng teraton. From 29, the error event metrc assocated wth event V j can be defned as Γ j = λ l A l + Y d = Ȧl λ l + Äl λ l + Y d, 32 d 1 where Y d L c =0 y NdL c, 2dL c. Snce n teraton l the extrnsc nformaton ξ l } are..d. for all, the statstcal characterstcs of Γ j n 32 and the probablty of V j only depends on the sze of Ȧ l and Ä l for l<jgven the statstcal nowledge of the extrnsc nformaton, but not on the partcular choces of the bt sets. Let A l = m l, and A l = n l wth 0 n l m l.dueto the fact that the events Ṙl l and R are dsjont, we now that Ȧ l Äl =. Snce A l = Ṙl l R,wehave Äl = m l n l. Thus, to determne the statstcal characterstcs of Γ j and V j, t s suffcent to specfy a 2j-tuple V j as V j = A l = m l, A } l = n l. For convenence, we use Γ j V j to denote the error event metrc wth a partcular V j. Then we have Γ j V j N µv j,σ 2 V j, where µv j =dl c + φ l µ l, and σ 2 V j =2dL c + φ l σl 2, 33 n whch φ l = m l M 1 n l. 34 Recall that n the LRB exchange scheme, no nformaton can be exchanged for a non-canddate bt;.e., A l A = for l. Thus 0 m l w l, where w l = w m s the number of non-zero canddate bts left n u gven the event A t = m t } occurs. We note that w 0 = w 1. Based on the above arguments, the probablty P Γ j <x can be calculated as P Γ j 2j <x= P Γ j V j <x, V j = V j, 35 V j 2j V j where means the 2j-fold summaton 2j over all possble values of V j,.e., V j = w0 w w1 m0 m m1 m 0=0 m 1=0 m n 0=0 n 1=0 n, and we use V j = V j } to denote the occurrence of } all possble sets AV j = A l, A l, Äl satsfyng Al = m l, A } l = n l. Snce the 2j-tuple V j only constrans the sze of A l and Ȧl, for all l, A l can be an arbtrary bt set n the w l nonzero bts of u and the subset Ȧ l can be arbtrary subset n A l. Hence, for a gven V j, wl there are ml m l n l possble choces of AVj. For all these choces, the probabltes of the event V j = AV j }, are the same. Thus, we can upper bound 35 by upper boundng P Γ j V j <x,v j = AV j for each partcular choce of AV j. In a manner smlar to 14 and 22, λ l λ l we drop all the events assocated wth or n V j and use 17 and 18 to obtan 36, where c V j s calculated as shown n 37. On the other hand, we now that P Γ j V j j <x, V j = V j P Γ V j <x. Then the upper bound n 36 can be refned as P Γ j V j <x, V j = V j cvj P Γ j V j <x, 38 where cv j = mn1,c V j }, and P Γ j V j <x µvj x = Q 39 σv j wth Q beng the Gaussan Q-functon. Then by nsertng 38 nto 35 we have P Γ j <x 2j cv j P Γ j V j <x. 40 V j Combnng 39, 40 and 30, we obtan followng upper bound

8 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 65 P Γ j V j <x, V j = V j = wl ml m l n l wl ml P m l P n l Ȧl Äl P Γ j V j <x,v j = AV j P Γ j V j <x r M l ξ r, <T l, ξ t r, <T } l } t, ξ t M, >T t ξ t M, >T t P B j B j c V j P Γ j V j <x 36 c V j = wl ml m l n l } [M 1p l ] n l 1 p l Mwj+n l 1 p t m l+n l 37 P ξ j <x 1 K c d d mn w 1 wa V j 2j µvj x cv jq. σv j 41 Ths closed-form bound can also be appled to the probabltes P ξ j + λ l < 0, P ξ j + λ l < 0, and P ξ < T n 24 wthout any dffculty. Now, we only have P ξ j < 0,ξ > T left n 24 to evaluate. The dffculty here s the correlaton between ξ j and ξ. To unvel ths correlaton, we consder the extrnsc nformaton expresson gven n 25. Let u, c + opt = j arg max u,c C + Γ u,c} denote the optmal decodng sequence found by the decoder n C + and u, c opt denote the optmal decodng sequence n C. In max-log-map decodng, the fnal survval sequence u, c opt s generated between u, c + opt and u, c opt. If u, c + opt s not selected as the survvor sequence, t becomes the competng sequence. We assume the code s good enough that, when the SNR s not too low, the decoder can at least fnd the correct codeword as the competng sequence f t s not selected to be the survvor sequence. Ths assumpton s the same as that used n [13]. Thus, under the assumpton that the all-zero sequence 0, 0 s transmtted, we have u, c + opt =0, 0 snce 0, 0 C +. j } j That s, max u,c C + Γ u,c =Γ 0,0 =0. Wth the above arguments, we can rewrte 25 as follows by droppng the frst term,.e., ξ j j mn u,c C Γ u,c} when the SNR s hgh. Thus, for j>0 we have 42. Followng the dervaton from 29 through 41, we then obtan 43. To evaluate the probablty P Γ j V j < 0, Γ V >T n 43, we rewrte 32 as Γ j V j=γ V +Ψ, where Ψ λ Ȧ λ Ä Gven V j, we now that Γ V N µv,σ 2 V, Ψ Nφ µ,φ σ 2, and Γ and Ψ are ndependent of one another, where µv j and σv j are gven n 33 and φ j s gven n 34. Thus, we have 45, where the relatons µv j=µv +φ µ, and σ 2 V j=σ 2 V +φ σ 2 are used, and 46 s the bvarate Gaussan Q-functon, for whch [14] gves a smplfed expresson for numercal evaluaton. To ths pont, we have upper bounded the BER P j b for M 3. D. BER Upper Bound for M =2 For M =2, we note that the addtonal nformaton defned n 3 becomes 0 for all, and j, and the event defned n 4 reduces to λ j R j R j = ξj T j}. 47 Due to these dfferences, t s necessary to mae some modfcatons to the prevous analyss to obtan a tght bound for ths case. Agan, we consder the error performance at the Mth node. Followng the notaton n Secton IV-A, the nequalty n 13 becomes P ξ j + λ l < 0, R l, B l = P ξ j < 0, Ṙl, B l + P ξ j + λ l < 0, R l, B l P ξ j < 0, Ṙl, B l + P ξ j + λ l < Analogous to 14 and 22, we upper bound the frst term n 48 as shown n 49, where a l s gven by 21. Thus, by usng 48 for 13, the upper bound for P j b n 24 s shown n 50. Then, as for M 3, we apply the unon bound for max-log-map decodng to further upper bound the probabltes n 50. All the dervatons are the same as n Secton IV-C except for 36 and 37. Due to the change n 47, Rl l becomes ndependent of λ. Thus, for l M =2, we can eep R for Äl when we drop all the l events assocated wth λ n the dervaton of 36. Wth ths modfcaton, c V j n 37 becomes

9 66 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY 2008 P ξ j P ξ j < 0,ξ } >T P mn Γ j u,c} < 0, mn Γ u,c C u,c C u,c >T = 1 P u,c < 0, u,c >T K c < 0,ξ >T 1 K c 1 K c u,c C Γ j u,c C P Γ j u,c < 0, u,c C Γ Γ u,c u,c C >T 1 P Γ j u,c < 0, Γ u,c >T 42 K c u,c C d d mn w 1 [wa V j 2j cv j P Γ j V j < 0, Γ V >T ] 43 P Γ j V j < 0, Γ V >T = P Γ V +Ψ< 0, Γ V >T = P Γ V +Ψ< 0, Ψ < T P Γ V <T, Ψ < T = P Γ V +Ψ< 0, Ψ < T P Γ V <T P Ψ < T = Q µv j σv j, φ µ + T φ σ ; φ σ µv T Q σv j σv Q φ µ + T φ σ 45 Qx, y; ρ x y 1 2π 1 ρ exp 2 z2 1 + z2 2 2ρz 1 z 2 21 ρ 2 dz 1 dz 2 46 c w l m [ l V j= p ml l 1 p l 2w j +n l 1 p t m l+n l ]. m l n l Smulaton wth M=1 Smulaton wth M=2 Smulaton wth M=6 Proposed bound Unon bound for EGC V. NUMERICAL RESULTS The upper bound developed n Secton IV s used wth the addtonal-nformaton generaton procedure descrbed n Secton III to gve an approxmate upper bound for the BER. In ths secton, we gve numercal results to demonstrate the tghtness of ths approxmate upper bound. For convenence of dscusson, we refer to the numercal results from evaluatng ths approxmate upper bound as the analytcal results n all that follows. Frst, we set the number of teratons I to 3.e., 3 exchanges and 4 decodng teratons are performed n total, and set p j } to 0.1, 0.15, 0.25} n the collaboratve decodng process. In Fg. 5 the analytcal results are compared wth the smulaton results for the BER at each teraton for M = 2 and M = 6, respectvely. In the system, a non-recursve convolutonal code wth generatng polynomals [1+D 2, 1+D +D 2 ] s used. We denote t by CC5, 7. From the fgure, we see that n the low BER regon, the analytcal results are very close to the smulaton results n all teratons. For very low E b /N 0.e., hgh BER, the analytcal results dffer sgnfcantly from the smulaton results. Ths s due to the nature of the unon bound gven n 28. In the fgure, we also show the unon bounds for EGC of all symbols. We Bt Error Rate M=6 EGC 3 exch. 2 exch. 1 exch. Unon bound for sngle recever E b /N 0 db Fg. 5. Comparson of the approxmate upper bounds and smulaton results for M =2and 6 on AWGN channels, where CC5, 7 and p j = 0.1, 0.15, 0.25} are used. can see that, for M =2, the performance of collaboratve decodng wth the LRB exchange scheme s very close that of EGC, whle t s wthn about 2dB of EGC for M =6. Ths means that most of the combnng gan can be obtaned through collaboratve decodng. In Fg. 6, we show the results for another non-recursve convolutonal code wth generator polynomals [1 + D 2 + M=2 EGC

10 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 67 P ξ j < 0, Ṙl, B l = P ξ j < 0, ξ l 1 T l, ξ l >T l, B l P ξ j < 0, ξ l 1 T l, ξ l >T l, ξ t 1 >T t = P ξ j < 0, ξ l >T l P ξ l 1 T l, ξ t 1 >T t a l [ P ξ l < T l +P ξ j < 0,ξ l >T l ] 49 P j b [ a l P ξ l < T l +Pξ j < 0,ξ l >T l ] + P ξ j + λ } l < 0 + b j [ P ξ < T +P ξ j < 0,ξ >T ] 50 TABLE I DIFFERENT CHOICES OF p j } AND CORRESPONDING COOPERATIVE OVERHEADS, Θ WITHOUT EARLY STOPPING CRITERION AND ˆΘ WITH EARLY STOPPING CRITERION FOR A RATE 1/2 CC5, 7 CODE AND M =8. Θ AND ˆΘ ARE EXPRESSED AS A PROPORTION OF THE COOPERATIVE OVERHEAD FOR EGC, Θ EGC. Case Max#of Value of p j } I 1 Cooperatve overhead Exchanges j=0 wthout early stop, Θ Cooperatve overhead wth early stop, ˆΘ E b /N 0 =0dB E b /N 0 =2dB E b /N 0 =4dB 1 I =3 0.1, 0.2, 0.3} Θ EGC 0.449Θ EGC 0.287Θ EGC Θ EGC 2 I =3 0.2, 0.4, 0.6} Θ EGC Θ EGC Θ EGC 0.119Θ EGC 3 I =3 0.3, 0.6, 0.9} Θ EGC Θ EGC Θ EGC Θ EGC 4 I =3 0.1, 0.2, 1} Θ EGC Θ EGC 0.286Θ EGC Θ EGC 5 I =5 0.03, 0.06, 0.1, 0.2, 1} Θ EGC Θ EGC Θ EGC Θ EGC 6 I =1 1} 0.5Θ EGC 0.5Θ EGC 0.5Θ EGC 0.5Θ EGC Smulaton P roposed bound Unon bound for EGC case B t E rror R ate 10 6 ^ Θ / Θ EGC case 1 and 4 case 2 case EGC, M=8 M=8 M=6 M=4 M=3 M=2 M= case E b /N 0 db E /N db b 0 Fg. 6. Comparson of the approxmate upper bounds and smulaton results n the last teraton for M =2, 3, 4, 6 and 8 on AWGN channels, where CC15, 17 and p j = 0.1, 0.15, 0.25} are used. Fg. 7. Rato of the cooperatve overhead of LRB wth the early stoppng crteron ˆΘ to the cooperatve overhead of equal-gan combnng Θ EGC for the 6 dfferent choces of p j } n Table I. D 3, 1+D +D 2 +D 3 ], denoted by CC15, 17. The parameter set p j } s the same as for Fg. 5. We compare the analytcal results wth smulaton results for M = 2, 3, 4, 6, and 8, respectvely. For clarty, we only show the BER n the last teraton for each M. We note that the analytcal results are a lttle bt below the smulaton results when M =2.Ths s attrbutable to the ndependence assumpton and Gaussan approxmaton n Secton III not beng very accurate for CC15, 17 when M = 2. However, when M 3 the assumptons become much closer to the actual stuaton. From the fgure, we can see that the bounds are very tght. We can apply the analyss developed n ths paper to determne the effects of dfferent choces of p j } on the error performance n collaboratve decodng. Whle comparng dfferent collaboratve decodng processes, t s mportant to consder both the error performance and the amount of nfor-

11 68 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 1, JANUARY 2008 maton exchanged durng the process. Snce the nformaton beng exchanged conssts of soft-outputs n LLR form for a porton of the nformaton bts, we use the average total number of LLRs transmtted through the broadcast channel n the dstrbuted array as a smple measure of the amount of nformaton exchanged n the collaboratve decodng procedure, whch we refer to as the cooperatve overhead. Other protocol overhead s gnored here. Let Θ denote the cooperatve overhead for a full I teratons of the LRB exchange procedure, regardless of whether correct decodng s acheved n an earler teraton. Then by applyng the ndependence assumpton of Secton III, for a set of p j } I 1 j=0, I 1 [ Θ=MN 1 1 pj M 1] 1 p l M, 52 j=0 where N s the bloc sze n nformaton bts. Correspondngly, the cooperatve overhead of EGC s Θ EGC = MN/R c wth R c beng the code rate, whch wll be used for the purpose, when the SNR s not too low, correct decodng s often acheved durng an early teraton of the collaboratve decodng process, so that further nformaton exchange becomes unnecessary. Thus, f the collaboratve decodng process s termnated whenever correct decodng s acheved, the number of teratons I becomes a random varable. We denote the cooperatve overhead wth ths early stoppng crtera by ˆΘ. Below, we consder the rate 1/2 code CC5, 7 and compare the 6 cases lsted n Table I. The table also shows the cooperatve overhead wthout the early stoppng crtera, Θ, whch s calculated from 52, and the cooperatve overhead wth the early stoppng crteron, ˆΘ, whch s obtaned from Monte Carlo smulaton. In Fg. 7, we show the rato of the cooperatve overhead wth the early stoppng crteron ˆΘ to the cooperatve overhead of equal-gan combnng Θ EGC for the dfferent cases of p j } lsted n Table I. It can be seen that as E b /N 0 ncreases, the cooperatve overhead for LRB wth the early stoppng crteron becomes much less than that for EGC. Ths shows the advantage of collaboratve decodng n term of nformaton-exchange effcency. In Fg. 8a we show the BER bounds for each teraton for each of the sx cases lsted n Table I. From the fgure, we see that cases 4, 5 and 6 acheve the same performance n ther last teraton and outperform cases 2 and 3. Case 1 offers the worst performance. In Fg. 8b, we compare the analytcal results for all sx cases n ther last teraton wth that of EGC n the very low BER regon. Ths gves an ndcaton of the asymptotc performance of the systems. In the fgure, we see that n cases 4, 5 and 6, the recevers fnally acheve the same error performance as EGC, but wth a much smaller cooperatve overhead when the early stoppng crteron s used cf. Fg. 7. Ths ndcates that wth proper choces of p j }, collaboratve decodng wth LRB exchange can acheve the full combnng gan. of comparson. We note that for a gven p j } I 1 j=0 VI. CONCLUSION We developed an approxmate analyss based on densty evoluton to study the performance for collaboratve decodng wth LRB exchange. We consder a scenaro n whch the B t E rror R ate B t E rror R ate Last teratons for case 4, 5 and 6 Las t teratons for cas e 2 and 3 Last teratons for cas e 1 Bounds for case 1 Bounds for case 2 Bounds for case 3 Bounds for case 4 Bounds for case 5 Bounds for case 6 Unon bound for EGC Unon bound for s ngle recever Iteratons E /N db b 0 EGC Case 4, 5 and 6 Case 3 a Case 1 Case 2 Unon bound for s ngle recever E /N db b 0 b Fg. 8. Comparson of performance for M =8and CC5, 7 wth dfferent choces of p j } n Table I. a Comparson of approxmate upper bounds and smulaton results. b Performance comparson at hgh SNR. channels from the remote source to the collaboratng nodes are non-fadng AWGN channels wth equal SNRs at each of the collaboratng nodes. Ths scenaro may occur when the collaboratng nodes are clustered together and approxmately equdstant from the remote source. The analytcal results ndcate that, wth proper desgn of the nformaton exchange parameters, as the SNR ncreases, the performance of collaboratve decodng wth LRB exchange approaches that of equal-gan combnng of all receved symbols from all of the collaboratng nodes. At the same tme, the cooperatve overhead of collaboratve decodng wth LRB exchange s shown to become much smaller than that of equal-gan combnng as the SNR ncreases. The analyss s based on the observaton that the nodes have dentcal channel characterstcs and that the extrnsc nformaton generated n the collaboratve decodng process can be well-approxmated by Gaussan dstrbutons. The assumpton of dentcal channel characterstcs prevents us from currently

12 LI et al.: PERFORMANCE ANALYSIS FOR COLLABORATIVE DECODING WITH LEAST-RELIABLE-BITS EXCHANGE ON AWGN CHANNELS 69 generalzng these result to systems wth dfferent attenuatons on the channels from the remote source to the collaboratng nodes. The use of the Gaussan dstrbuton for the extrnsc nformaton lmts our results to systems that use non-recursve convolutonal codes and to nonfadng channels. The advantage of these assumptons s that they mae the calculatons n the analyss smple. If we can fnd proper probablty dstrbuton models, our analyss may be extended to the cases of recursve convolutonal codes and fadng channels. For the ndependent Raylegh case, we have recently found a model that wll allow us to apply the basc deas presented n ths paper, although the analyss becomes much more complcated [16]. REFERENCES [1] T. F. Wong, X. L, and J. M. Shea, Dstrbuted decodng of rectangular party-chec code, Electroncs Lett., vol. 38, no. 22, pp , Oct [2] T. F. Wong, X. L and J. M. Shea, Iteratve decodng n a twonode dstrbuted array, n Proc IEEE Mltary Commun. Conf. MILCOM, vol. 2, pp , Oct [3] A. Avudanayagam, J. M. Shea, T. F. Wong, and X. L, Relablty exchange schemes for teratve pacet combnng n dstrbuted arrays, n Proc IEEE Wreless Commun. and Networng Conf. WCNC, vol. 2, pp , Mar [4] X. L, T. F. Wong, and J. M. Shea, Bt-nterleaved rectangular partychec coded modulaton wth teratve demodulaton n a two-node dstrbuted array, n Proc IEEE Int. Conf. Commun. ICC, vol. 4, pp , May [5] H. E. Gamal and A. R. Hammons, Jr., Analyzng the turbo decoder usng the Gaussan approxmaton, IEEE Trans. Inf. Theory, vol. 47, no. 2, pp , Feb [6] D. Dvsalar, S. Dolnar, and F. Pollara, Iteratve turbo decoder analyss based on densty evoluton, IEEE J. Sel. Areas Commun., vol. 19, no. 5, pp , May [7] J. Hagenauer, E. Offer, and L. Pape, Iteratve decodng of bnary bloc and convolutonal codes, IEEE Trans. Inf. Theory, vol. 42, pp , Mar [8] A. Nouh and A. Banhashem, Relablty-based schedule for btflppng decodng of low-densty party-chec codes, IEEE Trans. Commun., vol. 52, pp , Dec [9] A. Roongta and J. M. Shea, Relablty-based hybrd ARQ and ratecompatble punctured convolutonal codes, n Proc IEEE Wreless Commun. and Networng Conf. WCNC, vol. 4, pp , Mar. 2004, [10] A. Avudanayagam, J. M. Shea, and A. Roongta, Improvng the effcency of relablty-based hybrd-arq wth convolutonal codes, n Proc IEEE Mltary Commun. Conf. MILCOM, vol. 1, pp , Oct [11] G. Montors and S. Benedetto, Desgn of fxed-pont teratve decoders for concatenated codes wth nterleavers, IEEE J. Sel. Areas Commun., vol. 19, pp , May [12] B. Blanchard, Quantzaton effects and mplementaton consderatons for turbo decoders, Master s thess, Unversty of Florda, [13] L. Reggan and G. Tartara, Probablty densty functon of soft nformaton, IEEE Commun. Lett., vol. 6, no. 2, pp , Feb [14] M. Smon and D. Dvsalar, Some new twsts to problem nvolvng the Gaussan probablty ntegral, IEEE Trans. Commun., vol. 46, no. 2, pp , Feb [15] G. McLachlan and D. Peel, Fnte Mxture Models. Wley, [16] X. L, T. F. Wong, and J. M. Shea, Performance analyss for collaboratve decodng wth most-relable-bt exchange. Avalable: twong/ Preprnts/mrb.pdf Xn L receved the B.S. degree from Northwestern Polytechncal Unversty, X an, Chna, n 1996, the M.S. degree from Shangha Jao Tong Unversty, Shangha, Chna, n 1999, and the Ph.D. degree from the Unversty of Florda n 2006, all n electrcal engneerng. From 1999 to 2000, he was a communcatons engneer at ZTE, Corp., Chna. He also served as a research engneer at Extemporal Wreless, Inc. n Currently, he s a DSP systems engneer at 2Wre, Inc. Hs research nterests nclude wreless communcaton theory, statstcal sgnal processng, speech sgnal processng, and wreless networs. Tan F. Wong receved the B.Sc. degree 1st class honors n electronc engneerng from the Chnese Unversty of Hong Kong n 1991, and the M.S.E.E. and Ph. D. degrees n electrcal engneerng from Purdue Unversty n 1992 and 1997, respectvely. He was a research engneer worng on the hgh speed wreless networs project n the Department of Electroncs at Macquare Unversty, Sydney, Australa. He also served as a post-doctoral research assocate n the School of Electrcal and Computer Engneerng at Purdue Unversty. Snce August 1998 he has been wth the Unversty of Florda, where he s currently an assocate professor of electrcal and computer engneerng. He was Edtor for Wdeband and Multple Access Wreless Systems for the IEEE Transactons on Communcatons and was the Edtor for the IEEE Transactons on Vehcular Technology from 2003 to John M. Shea S 92-M 99 receved the B.S. wth hghest honors n Computer Engneerng from Clemson Unversty n 1993 and the M.S. and Ph. D. degrees n electrcal engneerng from Clemson Unversty n 1995 and 1998, respectvely. Dr. Shea s currently an Assocate Professor of electrcal and computer engneerng at the Unversty of Florda. Pror to that, he was an Assstant Professor at the Unversty of Florda from July 1999 to August 2005 and a post-doctoral research fellow at Clemson Unversty from January 1999 to August He was a research assstant n the Wreless Communcatons Program at Clemson Unversty from 1993 to He s currently engaged n research on wreless communcatons wth emphass on error-control codng, crosslayer protocol desgn, cooperatve dversty technques, and hybrd ARQ. Dr. Shea was selected as a Fnalst for the 2004 Eta Kappa Nu Outstandng Young Electrcal Engneer Award. Dr. Shea was a Natonal Scence Foundaton Fellow from 1994 to He receved the Ellersc Award from the IEEE Communcatons Socety for the Best Paper n the Unclassfed Program of the IEEE Mltary Communcatons Conference n He was an Assocate Edtor for the IEEE Transactons on Vehcular Technology from 2002 to 2007.

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Error Probability for M Signals

Error Probability for M Signals Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal

More information

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

Iterative Multiuser Receiver Utilizing Soft Decoding Information

Iterative Multiuser Receiver Utilizing Soft Decoding Information teratve Multuser Recever Utlzng Soft Decodng nformaton Kmmo Kettunen and Tmo Laaso Helsn Unversty of Technology Laboratory of Telecommuncatons Technology emal: Kmmo.Kettunen@hut.f, Tmo.Laaso@hut.f Abstract

More information

Lecture 3: Shannon s Theorem

Lecture 3: Shannon s Theorem CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts

More information

Low Complexity Soft-Input Soft-Output Hamming Decoder

Low Complexity Soft-Input Soft-Output Hamming Decoder Low Complexty Soft-Input Soft-Output Hammng Der Benjamn Müller, Martn Holters, Udo Zölzer Helmut Schmdt Unversty Unversty of the Federal Armed Forces Department of Sgnal Processng and Communcatons Holstenhofweg

More information

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations Applcaton of Nonbnary LDPC Codes for Communcaton over Fadng Channels Usng Hgher Order Modulatons Rong-Hu Peng and Rong-Rong Chen Department of Electrcal and Computer Engneerng Unversty of Utah Ths work

More information

ECE559VV Project Report

ECE559VV Project Report ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate

More information

Pulse Coded Modulation

Pulse Coded Modulation Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal

More information

Lecture 4: November 17, Part 1 Single Buffer Management

Lecture 4: November 17, Part 1 Single Buffer Management Lecturer: Ad Rosén Algorthms for the anagement of Networs Fall 2003-2004 Lecture 4: November 7, 2003 Scrbe: Guy Grebla Part Sngle Buffer anagement In the prevous lecture we taled about the Combned Input

More information

Composite Hypotheses testing

Composite Hypotheses testing Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter

More information

VQ widely used in coding speech, image, and video

VQ widely used in coding speech, image, and video at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models

More information

x = , so that calculated

x = , so that calculated Stat 4, secton Sngle Factor ANOVA notes by Tm Plachowsk n chapter 8 we conducted hypothess tests n whch we compared a sngle sample s mean or proporton to some hypotheszed value Chapter 9 expanded ths to

More information

Chapter - 2. Distribution System Power Flow Analysis

Chapter - 2. Distribution System Power Flow Analysis Chapter - 2 Dstrbuton System Power Flow Analyss CHAPTER - 2 Radal Dstrbuton System Load Flow 2.1 Introducton Load flow s an mportant tool [66] for analyzng electrcal power system network performance. Load

More information

Simulated Power of the Discrete Cramér-von Mises Goodness-of-Fit Tests

Simulated Power of the Discrete Cramér-von Mises Goodness-of-Fit Tests Smulated of the Cramér-von Mses Goodness-of-Ft Tests Steele, M., Chaselng, J. and 3 Hurst, C. School of Mathematcal and Physcal Scences, James Cook Unversty, Australan School of Envronmental Studes, Grffth

More information

Lossy Compression. Compromise accuracy of reconstruction for increased compression.

Lossy Compression. Compromise accuracy of reconstruction for increased compression. Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

Problem Set 9 Solutions

Problem Set 9 Solutions Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem

More information

The optimal delay of the second test is therefore approximately 210 hours earlier than =2.

The optimal delay of the second test is therefore approximately 210 hours earlier than =2. THE IEC 61508 FORMULAS 223 The optmal delay of the second test s therefore approxmately 210 hours earler than =2. 8.4 The IEC 61508 Formulas IEC 61508-6 provdes approxmaton formulas for the PF for smple

More information

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder.

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder. PASSBAND DIGITAL MODULATION TECHNIQUES Consder the followng passband dgtal communcaton system model. cos( ω + φ ) c t message source m sgnal encoder s modulator s () t communcaton xt () channel t r a n

More information

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem. Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS Avalable onlne at http://sck.org J. Math. Comput. Sc. 3 (3), No., 6-3 ISSN: 97-537 COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

More information

The Synchronous 8th-Order Differential Attack on 12 Rounds of the Block Cipher HyRAL

The Synchronous 8th-Order Differential Attack on 12 Rounds of the Block Cipher HyRAL The Synchronous 8th-Order Dfferental Attack on 12 Rounds of the Block Cpher HyRAL Yasutaka Igarash, Sej Fukushma, and Tomohro Hachno Kagoshma Unversty, Kagoshma, Japan Emal: {garash, fukushma, hachno}@eee.kagoshma-u.ac.jp

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform

More information

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s

More information

ISSN: ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 1, July 2013

ISSN: ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 1, July 2013 ISSN: 2277-375 Constructon of Trend Free Run Orders for Orthogonal rrays Usng Codes bstract: Sometmes when the expermental runs are carred out n a tme order sequence, the response can depend on the run

More information

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons

More information

Foundations of Arithmetic

Foundations of Arithmetic Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an

More information

Notes on Frequency Estimation in Data Streams

Notes on Frequency Estimation in Data Streams Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to

More information

Power Allocation for Distributed BLUE Estimation with Full and Limited Feedback of CSI

Power Allocation for Distributed BLUE Estimation with Full and Limited Feedback of CSI Power Allocaton for Dstrbuted BLUE Estmaton wth Full and Lmted Feedback of CSI Mohammad Fanae, Matthew C. Valent, and Natala A. Schmd Lane Department of Computer Scence and Electrcal Engneerng West Vrgna

More information

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method Appled Mathematcal Scences, Vol. 7, 0, no. 47, 07-0 HIARI Ltd, www.m-hkar.com Comparson of the Populaton Varance Estmators of -Parameter Exponental Dstrbuton Based on Multple Crtera Decson Makng Method

More information

The Geometry of Logit and Probit

The Geometry of Logit and Probit The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.

More information

Inductance Calculation for Conductors of Arbitrary Shape

Inductance Calculation for Conductors of Arbitrary Shape CRYO/02/028 Aprl 5, 2002 Inductance Calculaton for Conductors of Arbtrary Shape L. Bottura Dstrbuton: Internal Summary In ths note we descrbe a method for the numercal calculaton of nductances among conductors

More information

Structure and Drive Paul A. Jensen Copyright July 20, 2003

Structure and Drive Paul A. Jensen Copyright July 20, 2003 Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.

More information

Finding Dense Subgraphs in G(n, 1/2)

Finding Dense Subgraphs in G(n, 1/2) Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng

More information

RELIABILITY ASSESSMENT

RELIABILITY ASSESSMENT CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department

More information

NUMERICAL DIFFERENTIATION

NUMERICAL DIFFERENTIATION NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the

More information

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number

More information

Pop-Click Noise Detection Using Inter-Frame Correlation for Improved Portable Auditory Sensing

Pop-Click Noise Detection Using Inter-Frame Correlation for Improved Portable Auditory Sensing Advanced Scence and Technology Letters, pp.164-168 http://dx.do.org/10.14257/astl.2013 Pop-Clc Nose Detecton Usng Inter-Frame Correlaton for Improved Portable Audtory Sensng Dong Yun Lee, Kwang Myung Jeon,

More information

For now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.

For now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results. Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

The Expectation-Maximization Algorithm

The Expectation-Maximization Algorithm The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.

More information

An (almost) unbiased estimator for the S-Gini index

An (almost) unbiased estimator for the S-Gini index An (almost unbased estmator for the S-Gn ndex Thomas Demuynck February 25, 2009 Abstract Ths note provdes an unbased estmator for the absolute S-Gn and an almost unbased estmator for the relatve S-Gn for

More information

MAXIMUM A POSTERIORI TRANSDUCTION

MAXIMUM A POSTERIORI TRANSDUCTION MAXIMUM A POSTERIORI TRANSDUCTION LI-WEI WANG, JU-FU FENG School of Mathematcal Scences, Peng Unversty, Bejng, 0087, Chna Center for Informaton Scences, Peng Unversty, Bejng, 0087, Chna E-MIAL: {wanglw,

More information

THE SUMMATION NOTATION Ʃ

THE SUMMATION NOTATION Ʃ Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the

More information

Kernel Methods and SVMs Extension

Kernel Methods and SVMs Extension Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general

More information

System in Weibull Distribution

System in Weibull Distribution Internatonal Matheatcal Foru 4 9 no. 9 94-95 Relablty Equvalence Factors of a Seres-Parallel Syste n Webull Dstrbuton M. A. El-Dacese Matheatcs Departent Faculty of Scence Tanta Unversty Tanta Egypt eldacese@yahoo.co

More information

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence

More information

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010 Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton

More information

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4) I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes

More information

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U) Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of

More information

Chapter 13: Multiple Regression

Chapter 13: Multiple Regression Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to

More information

DC-Free Turbo Coding Scheme Using MAP/SOVA Algorithms

DC-Free Turbo Coding Scheme Using MAP/SOVA Algorithms Proceedngs of the 5th WSEAS Internatonal Conference on Telecommuncatons and Informatcs, Istanbul, Turkey, May 27-29, 26 (pp192-197 DC-Free Turbo Codng Scheme Usng MAP/SOVA Algorthms Prof. Dr. M. Amr Mokhtar

More information

On the Interval Zoro Symmetric Single-step Procedure for Simultaneous Finding of Polynomial Zeros

On the Interval Zoro Symmetric Single-step Procedure for Simultaneous Finding of Polynomial Zeros Appled Mathematcal Scences, Vol. 5, 2011, no. 75, 3693-3706 On the Interval Zoro Symmetrc Sngle-step Procedure for Smultaneous Fndng of Polynomal Zeros S. F. M. Rusl, M. Mons, M. A. Hassan and W. J. Leong

More information

Markov Chain Monte Carlo Lecture 6

Markov Chain Monte Carlo Lecture 6 where (x 1,..., x N ) X N, N s called the populaton sze, f(x) f (x) for at least one {1, 2,..., N}, and those dfferent from f(x) are called the tral dstrbutons n terms of mportance samplng. Dfferent ways

More information

1 GSW Iterative Techniques for y = Ax

1 GSW Iterative Techniques for y = Ax 1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn

More information

Markov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement

Markov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs

More information

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011 Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected

More information

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or Sgnal Compresson Sgnal Compresson Entropy Codng Entropy codng s also known as zero-error codng, data compresson or lossless compresson. Entropy codng s wdely used n vrtually all popular nternatonal multmeda

More information

Comparison of Regression Lines

Comparison of Regression Lines STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence

More information

More metrics on cartesian products

More metrics on cartesian products More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of

More information

Statistical Evaluation of WATFLOOD

Statistical Evaluation of WATFLOOD tatstcal Evaluaton of WATFLD By: Angela MacLean, Dept. of Cvl & Envronmental Engneerng, Unversty of Waterloo, n. ctober, 005 The statstcs program assocated wth WATFLD uses spl.csv fle that s produced wth

More information

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran

More information

Estimation: Part 2. Chapter GREG estimation

Estimation: Part 2. Chapter GREG estimation Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the

More information

DUE: WEDS FEB 21ST 2018

DUE: WEDS FEB 21ST 2018 HOMEWORK # 1: FINITE DIFFERENCES IN ONE DIMENSION DUE: WEDS FEB 21ST 2018 1. Theory Beam bendng s a classcal engneerng analyss. The tradtonal soluton technque makes smplfyng assumptons such as a constant

More information

Expectation Maximization Mixture Models HMMs

Expectation Maximization Mixture Models HMMs -755 Machne Learnng for Sgnal Processng Mture Models HMMs Class 9. 2 Sep 200 Learnng Dstrbutons for Data Problem: Gven a collecton of eamples from some data, estmate ts dstrbuton Basc deas of Mamum Lelhood

More information

Boostrapaggregating (Bagging)

Boostrapaggregating (Bagging) Boostrapaggregatng (Baggng) An ensemble meta-algorthm desgned to mprove the stablty and accuracy of machne learnng algorthms Can be used n both regresson and classfcaton Reduces varance and helps to avod

More information

The Order Relation and Trace Inequalities for. Hermitian Operators

The Order Relation and Trace Inequalities for. Hermitian Operators Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence

More information

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,

More information

Errors for Linear Systems

Errors for Linear Systems Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch

More information

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003 Tornado and Luby Transform Codes Ashsh Khst 6.454 Presentaton October 22, 2003 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel

More information

Global Sensitivity. Tuesday 20 th February, 2018

Global Sensitivity. Tuesday 20 th February, 2018 Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values

More information

Resource Allocation with a Budget Constraint for Computing Independent Tasks in the Cloud

Resource Allocation with a Budget Constraint for Computing Independent Tasks in the Cloud Resource Allocaton wth a Budget Constrant for Computng Independent Tasks n the Cloud Wemng Sh and Bo Hong School of Electrcal and Computer Engneerng Georga Insttute of Technology, USA 2nd IEEE Internatonal

More information

Lecture 12: Classification

Lecture 12: Classification Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna

More information

Graph Reconstruction by Permutations

Graph Reconstruction by Permutations Graph Reconstructon by Permutatons Perre Ille and Wllam Kocay* Insttut de Mathémathques de Lumny CNRS UMR 6206 163 avenue de Lumny, Case 907 13288 Marselle Cedex 9, France e-mal: lle@ml.unv-mrs.fr Computer

More information

Real-Time Systems. Multiprocessor scheduling. Multiprocessor scheduling. Multiprocessor scheduling

Real-Time Systems. Multiprocessor scheduling. Multiprocessor scheduling. Multiprocessor scheduling Real-Tme Systems Multprocessor schedulng Specfcaton Implementaton Verfcaton Multprocessor schedulng -- -- Global schedulng How are tasks assgned to processors? Statc assgnment The processor(s) used for

More information

Negative Binomial Regression

Negative Binomial Regression STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...

More information

Lecture 3 Stat102, Spring 2007

Lecture 3 Stat102, Spring 2007 Lecture 3 Stat0, Sprng 007 Chapter 3. 3.: Introducton to regresson analyss Lnear regresson as a descrptve technque The least-squares equatons Chapter 3.3 Samplng dstrbuton of b 0, b. Contnued n net lecture

More information

CONTRAST ENHANCEMENT FOR MIMIMUM MEAN BRIGHTNESS ERROR FROM HISTOGRAM PARTITIONING INTRODUCTION

CONTRAST ENHANCEMENT FOR MIMIMUM MEAN BRIGHTNESS ERROR FROM HISTOGRAM PARTITIONING INTRODUCTION CONTRAST ENHANCEMENT FOR MIMIMUM MEAN BRIGHTNESS ERROR FROM HISTOGRAM PARTITIONING N. Phanthuna 1,2, F. Cheevasuvt 2 and S. Chtwong 2 1 Department of Electrcal Engneerng, Faculty of Engneerng Rajamangala

More information

DETERMINATION OF UNCERTAINTY ASSOCIATED WITH QUANTIZATION ERRORS USING THE BAYESIAN APPROACH

DETERMINATION OF UNCERTAINTY ASSOCIATED WITH QUANTIZATION ERRORS USING THE BAYESIAN APPROACH Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata TC XVII IMEKO World Congress Metrology n the 3rd Mllennum June 7, 3,

More information

EEE 241: Linear Systems

EEE 241: Linear Systems EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they

More information

FUZZY FINITE ELEMENT METHOD

FUZZY FINITE ELEMENT METHOD FUZZY FINITE ELEMENT METHOD RELIABILITY TRUCTURE ANALYI UING PROBABILITY 3.. Maxmum Normal tress Internal force s the shear force, V has a magntude equal to the load P and bendng moment, M. Bendng moments

More information

Chapter 8 Indicator Variables

Chapter 8 Indicator Variables Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n

More information

An Improved multiple fractal algorithm

An Improved multiple fractal algorithm Advanced Scence and Technology Letters Vol.31 (MulGraB 213), pp.184-188 http://dx.do.org/1.1427/astl.213.31.41 An Improved multple fractal algorthm Yun Ln, Xaochu Xu, Jnfeng Pang College of Informaton

More information

A Robust Method for Calculating the Correlation Coefficient

A Robust Method for Calculating the Correlation Coefficient A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal

More information

Transfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system

Transfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng

More information

Module 2. Random Processes. Version 2 ECE IIT, Kharagpur

Module 2. Random Processes. Version 2 ECE IIT, Kharagpur Module Random Processes Lesson 6 Functons of Random Varables After readng ths lesson, ou wll learn about cdf of functon of a random varable. Formula for determnng the pdf of a random varable. Let, X be

More information

Implicit Integration Henyey Method

Implicit Integration Henyey Method Implct Integraton Henyey Method In realstc stellar evoluton codes nstead of a drect ntegraton usng for example the Runge-Kutta method one employs an teratve mplct technque. Ths s because the structure

More information

CHAPTER 14 GENERAL PERTURBATION THEORY

CHAPTER 14 GENERAL PERTURBATION THEORY CHAPTER 4 GENERAL PERTURBATION THEORY 4 Introducton A partcle n orbt around a pont mass or a sphercally symmetrc mass dstrbuton s movng n a gravtatonal potental of the form GM / r In ths potental t moves

More information

A new construction of 3-separable matrices via an improved decoding of Macula s construction

A new construction of 3-separable matrices via an improved decoding of Macula s construction Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula

More information

Grover s Algorithm + Quantum Zeno Effect + Vaidman

Grover s Algorithm + Quantum Zeno Effect + Vaidman Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the

More information

Goodness of fit and Wilks theorem

Goodness of fit and Wilks theorem DRAFT 0.0 Glen Cowan 3 June, 2013 Goodness of ft and Wlks theorem Suppose we model data y wth a lkelhood L(µ) that depends on a set of N parameters µ = (µ 1,...,µ N ). Defne the statstc t µ ln L(µ) L(ˆµ),

More information

Chapter Newton s Method

Chapter Newton s Method Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve

More information

Asymptotic Quantization: A Method for Determining Zador s Constant

Asymptotic Quantization: A Method for Determining Zador s Constant Asymptotc Quantzaton: A Method for Determnng Zador s Constant Joyce Shh Because of the fnte capacty of modern communcaton systems better methods of encodng data are requred. Quantzaton refers to the methods

More information

Canonical transformations

Canonical transformations Canoncal transformatons November 23, 2014 Recall that we have defned a symplectc transformaton to be any lnear transformaton M A B leavng the symplectc form nvarant, Ω AB M A CM B DΩ CD Coordnate transformatons,

More information

Numerical Heat and Mass Transfer

Numerical Heat and Mass Transfer Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and

More information

Statistics Chapter 4

Statistics Chapter 4 Statstcs Chapter 4 "There are three knds of les: les, damned les, and statstcs." Benjamn Dsrael, 1895 (Brtsh statesman) Gaussan Dstrbuton, 4-1 If a measurement s repeated many tmes a statstcal treatment

More information

P R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /

P R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering / Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons

More information