The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X, Y, S Sets are deoted by calligraphic letters X, Y, Z Radom variables are deoted by capital letters X i, x i X j i, xj i X, x Sigle elemets of vectors are deoted by a subscript idex The iterval subvectors (i.e., X i, X i+,..., X j ) of a vector Complete vectors are deoted by uderlies.2 Optimal Decodig Rules Let X be a arbitrary alphabet ad C X be a legth- code. Assume a radom codeword X C is chose with probability p (X ) ad trasmitted through a DMC with trasitio probability W (y x). For sequeces, the coditioal probability of the observed sequece Y Y is give by P r (Y y X x ) W (y x ) W (y i x i ). Choosig the codeword x C which maximizes P r(x x Y y ) is ow as maximum a posteriori (MAP) decodig ad this miimizes the probability of bloc error. Usig Bayes s rule, we fid that P r (X x Y y ) Sice the deomiator is a costat for all x, we fid that i p (x ) W (y x ) x C p ( x ) W (y x ). D MAP (y ) arg max x C p (x ) W (y x ). The maximum lielihood (ML) decodig rule is defied as D ML (y ) arg max x C W (y x ). Notice that D MAP (y ) D MAP (y ) if p (x ) is a costat for all x C. Therefore, ML decodig is optimal for equiprobable trasmissio.
.3 Maximum Lielihood Decodig The ML decodig rule implicitly divides the received vectors ito decodig regios ow as Vorooi regios. The Vorooi regio (i.e., decisio regio) for the codeword x C is the subset of Y defied by V (x ) {y Y W (y x ) > W (y x ) x C, x x }. I this case, the average probability of bloc error is give by P B W (y x ). x C y V (x ) It is worth otig that this formula breas dow if ties may occur. This ca be rectified by directig the ML decoder to choose a codeword radomly i this case. I this case, the above expressio for P B oly gives a upper boud..4 Chael Symmetry ad Liear Codes I may cases, the chael satisfies a symmetry coditio that allows us to simplify thigs. For simplicity, we will assume that X forms a Abelia group uder + ad that the chael symmetry is defied by W (y x + z) W (π z (y) x) for a set of Y-permutatios π x idexed by x X. Each permutatio is a oe-to-oe mappig π x : Y Y that satisfies π x+z (y) π x (π z (y)) π z (π x (y)) ad therefore the set of permutatios forms a group which is isomorphic to X. This type of chael is ow as output symmetric. We exted this symmetry to legth- sequeces by defiig W (y x + z ) W ( π z (y ) x ) W (π zi (y i ) x i ) with π z (y ) (π z (y ), π z2 (y 2 ),..., π z (y )). It is worth otig that this symmetry coditio is sufficiet to imply that a uiform iput distributio achieves the capacity of this DMC. Example. Cosider the BSC where X {0, }, Y {0, }, ad { p if x y W (y x) p if x y. The, π 0 (y) y ad π (y) y defies the atural symmetry of the chael. Example. Cosider the biary-iput AWGN chael where X {0, }, Y R, ad Y N ( ( ) x, σ 2). Although this is ot a DMC, similar results hold whe sums are replaced by itegrals. I this case, π 0 (y) y ad π (y) y defies the atural symmetry of the chael. If the code is also a group code (i.e., sum of ay two codewords is a codeword), the the Vorooi regio of ay codeword ca be writte as a trasformatio of V (0) with V (0 + x ) {y Y W (y 0 + x ) > W (y x ) x C, x 0 + x } { y Y W ( π x (y ) 0 ) > W ( π x (y ) x x ) x C, x 0 + x } { y Y W ( π x (y ) 0 ) > W ( π x (y ) 0 ) z C, z 0 }. πx (V (0)). The last step follows from the fact that y V (x ) implies that π x (y ) V (0). We ca also use this to simplify the probability of bloc error to P B W (y x ) x C x C y V (x ) y π x (V (0)) y V (0) W (y 0). i W ( π x (y ) 0 ) 2
This shows that the probability of ML decodig error for a group code over a output-symmetric chael is idepedet of the trasmitted codeword..5 The Pairwise Error Probability (PEP).5. Discrete Memoryless Chaels Sice computig the exact probability of error requires extesive owledge of the code, it is ofte useful to have bouds that are easier to compute. The basis of may of these bouds is the pairwise error probability (PEP) betwee ay two codewords. The PEP, deoted P (x x ), is the probability that the ML decoder chooses x whe x was trasmitted. This probability ca be writte as P (x x ) W (y x ) I (W (y x ) W (y x )), y Y where I(E) is the idicator fuctio for the evet E (i.e., it equals if the argumet is true ad 0 otherwise). The idicator fuctio is upper bouded by ( W (y I (W (y x ) W (y x )) x ) s ) W (y x ), for ay s [0, ], because the LHS is zero if the RHS is less tha oe ad the LHS is oe whe the RHS is greater tha oe. I geeral, the best boud is foud by miimizig over s. For biary-iput symmetric-output chaels, the miimum occurs at s /2 ad the implied boud is P (x x ) ( W (y W (y x ) x ) /2 ) W (y y Y x ) W (yi x i ) W (y i x i ) i y i Y d H (x, x ) i W (y 0) W (y ), because the sum is oe if x i x i. This boud is ow as the Bhattacharyya boud ad is typically writte as y Y P (x x ) γ d H(x, x ), where γ y Y W (y 0)W (y ) is the Bhattacharyya costat of the chael. For the BSC chael, this gives γ BSC 2 p( p). For the biary-iput AWGN (BIAWGN) chael with eergy per symbol E s ad oise spectral desity N 0, we have ad γ BIAW GN W (y x) (πn 0 ) /2 e (y E s( ) x ) 2 /N 0 (πn 0 ) /2 [ e (y E s) 2 /N 0 e (y+ E s) 2 /N 0 ] /2 dy (πn 0 ) /2 e (y2 +E s)/n 0 dy e Es/N0. It is also ow that γ is the best possible costat for bouds of the form γ d H..5.2 The AWGN Chael If the chael cosists of a modulator M (x ) R ad zero-mea AWGN with variace σ 2 perdimesio, the the PEP ca be computed exactly. I this case, the memoryless chael (it is o loger discrete) is defied by the coditioal p.d.f. W (y x ) ( 2πσ 2) /2 e 2σ 2 y M(x ) 2. 3
Sice the p.d.f. depeds oly o the distace betwee the received ad trasmitted vectors, we fid that the ML decoder pics the codeword whose trasmitted vector is closest to the received vector. To aalyze this, we ca project the received vector y oto the differece vector w M ( x ) M (x ) to get the decisio variable i Z w iy i. i w2 i Oe ca verify that Z is a zero-mea Gaussia radom variable with variace σ 2. Furthermore, the decoder will mae a error if a oly if Z w /2 (i.e., the received vector is closer to x tha x ). This allows us to rewrite the PEP as P (x x ) 2πσ 2 dy dy 2 dy dy 2 f Z (z) I dy W (y x ) I (W (y x ) W (y x )) dy W (y x ) I ( y M (x ) y M ( x ) ) ) dz ( z 2 M ( x ) M (x ) M( x ) M(x ) /2 e z 2 /(2σ 2) dz. A chage of variables shows that this itegral is equal to ( ) P (x x ) Q 2σ M ( x ) M (x ), where Q(α) /2 2π dz is the tail probability of zero-mea uit-variace Gaussia. α e z2 Recall that the biary-iput AWGN chael with M(x) E s ( ) x has eergy per trasmitted symbol E s ad oise spectral desity N 0 2σ 2. Therefore, the Euclidea distace is M ( x ) M (x ) 2 4E s d H (x, x ). Substitutig these ito our expressio gives ) P (x x ) Q ( 2 d H (x, x ) E s/n 0. Applyig the stadard boud, Q(α) e α2 /2, to the Q-fuctio gives a alterate proof of the Bhattacharyya boud for AWGN..6 The Uio Boud Sice every decodig error is caused a by pairwise error, we fid that P B x C x C, x x P (x x ). This is oly a upper boud because the received vector may be closer to two other codewords tha it is to the trasmitted codeword, ad this causes overcoutig of the error probability. If we assume that the code is liear ad that the PEP is a fuctio f(h) of the Hammig distace h, the we get P B f (d H (x, x )) x C x C, x x f (d H (0, x )) x C x C, x 0 f (d H (0, x )) x C, x 0 A h f(h), h 4
where A h ( is the umber of codewords of weight h. For biary codes, the fuctio f is either chose 2h ) to be Q Es /N 0 (for the AWGN chael) or γ h (for a arbitrary DMC with γ equal to the Bhattacharyya costat). I this case, the weight eumerator (WE) is ofte give i the polyomial form, A(H) h 0 A hh h, ad we ca use the Bhattacharyya boud (i.e., f(h) γ h ) to write P B A(γ). Example. The [7,4,3] Hammig code has the WE A(H) + 7H 3 + 7H 4 + H 7. This implies that ML decodig of this code o the BIAWGN chael has a bloc error probability which satisfies.7 Bit Error Probability P B 7e 3Es/N0 + 7e 4Es/N0 + e 7Es/N0. I may cases, we are iterested ot oly i the probability of bloc error P B but also i the probability of bit error P b (or symbol error P s for o-biary codes). To compute P b we eed to compute the average umber of message bit (or symbol) errors that occur as the result of a bloc error. Let E : U X be a ecoder which maps ay legth iput sequece to a legth output sequece. Cosider ay two iput-output pairs, x E ( u) ad x E ( ũ), ad otice that the pairwise error x x implies the message error u ũ (ũ ad produces d H, u) symbol errors i the decoded message. Usig the uio boud, we ca boud P s with P s u U U ũ U, ũ u P (E(u ) E(ũ )) d (ũ H, u). While this quatity ca be computed or bouded for ay code, it ca be simplified for codes with liear ecoders. Now, we will assume that U has a field structure ad that E is liear so that, for α, β U, E ( αu + βũ ( ) (ũ ) ) α E u + β E. I this case, the liearity implies that x x E ( ũ u ) ad that the pairwise error x x produces w H (ũ u ) symbol errors i the decoded message. If we also assume that the PEP is a fuctio f(h) of the Hammig distace h, the we ca write P s u U U ũ U, ũ 0 f w h f ( ( w H E(u ) E(ũ ) )) w (ũ H u) ( ( wh E(ũ ) )) w (ũ ) H ũ U, ũ u A w,h f(h) w, where the iput-ouput weight-eumerator (IOWE), A w,h, is the umber of codewords with iput weight w ad output weight h. The IOWE of a liear code is ofte give i polyomial form as A(W, H) w h A w,hw w H h. For biary codes, we ca therefore use the Bhattacharyya boud (i.e., f(h) γ h ) to write P b [ ] d A(W, γ). dw W Example. Oe ecoder for the [7,4,3] Hammig code has the IOWE A(W, H) + (3W + 3W 2 + W 3 )H 3 + (W + 3W 2 + 3W 3 )H 4 + W 4 H 7. This implies that ML decodig of this code o the BIAWGN chael has a bloc error probability which satisfies P b 2 4 e 3Es/N0 + 6 4 e 4Es/N0 + 4 4 e 7Es/N0. 5