On the Bit Error Probability of Noisy Channel Networks With Intermediate Node Encoding I. INTRODUCTION

Transcription

1 5188 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 [8] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood estimation from inomplete data via the EM algorithm, J. Roy. Statist. So. B, vol. 39, pp. 1 38, [9] J. Fritz, An information-theoretial proof of limit theorems for reversible Markov proesses, in Trans. Sixth Prague Conf. Inf. Theory, Statist. De. Funtions, Random Proesses, Prague, [10] A. E. Gelfand and A. F. M. Smith, Sampling-based approahes to alulating marginal densities, J. Amer. Statist. Asso., vol. 85, pp , [11] S. Geman and D. Geman, Stohasti relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mahine Intell., vol. 6, pp , [12] P. Harremoës and K. K. Holst, Convergene of Markov hains in information divergene, J. Theoret. Prob., 2008, to be published. [13] J. Liu, W. H. Wong, and A. Kong, Correlation struture and onvergene rate of the Gibbs sampler for various sans, J. Roy. Statist. So. B, vol. 57, pp , [14] X. L. Meng and D. A. van Dyk, The EM algorithm An old folk song sung to a fast new tune, J. Roy. Statist. So. B, vol. 59, pp , [15] X. L. Meng and D. A. van Dyk, Seeking effiient data augmentation shemes via onditional and marginal augmentation, Biometrika, vol. 86, pp , [16] H. L. Royden, Real Analysis, 3rd ed. New York: Mamillan, [17] M. J. Shervish and B. P. Carlin, On the onvergene of suessive substitution sampling, J. Computat. Graph. Statist., vol. 1, pp , [18] M. A. Tanner and W. H. Wong, The alulation of posterior distributions by data augmentation, J. Amer. Statist. Asso., vol. 82, pp , [19] L. Tierney, Markov hains for exploring posterior distributions, Ann. Statist., vol. 22, pp , [20] D. A. van Dyk and X. L. Meng, The art of data augmentation, J. Computat. Graph. Statist., vol. 10, pp , [21] Y. Yu, Information Geometry and the Gibbs Sampler Dept. of Statistis, University of California, Irvine, CA, Teh. Rep., [22] Y. Yu and X. L. Meng, Espousing Classial Statistis With Modern Computation: Suffiieny, Anillarity and an Interweaving Generation of MCMC Dep. Statistis, University of California, Irvine, CA, Teh. Rep., On the Bit Error Probability of Noisy Channel Networks With Intermediate Node Enoding Ming Xiao, Member, IEEE, and Tor Aulin, Fellow, IEEE Abstrat We investigate the alulation approah of the sink bit error probability (BEP) for a network with intermediate node enoding. The network onsists of statistially independent noisy hannels. The main ontributions are, for binary network odes, an error marking algorithm is given Manusript reeived August 25, 2006; revised February 29, Current version published Otober 22, The material in this orrespondene was presented in part at the IEEE International Symposium on Information Theory, Seattle, WA, July M. Xiao was with the Department of Computer Siene and Engineering, Chalmers University of Tehnology, Sweden. He is now with ACCESS Linnaeus Center, Royal Institute of Tehnology, Stokholm 10044, Sweden ( Ming.Xiao@ee.kth.se). T. Aulin is with the Department of Computer Siene and Engineering, Chalmers University of Tehnology, Sweden ( aulin@halmers.se). Communiated by S. W. MLaughlin, Assoiate Editor for Coding Tehniques. Color versions of Figures 2, 3, and 5 in this orrespondene are available online at Digital Objet Identifier /TIT to ollet the error weight (the number of erroneous bits). Thus, we an alulate the exat sink BEP from the hannel BEPs. Then we generalize the approah to nonbinary odes. The oding sheme works on the Galois field 2, where m is a positive integer. To redue omputational omplexity, a subgraph deomposition approah is proposed. In general, it an signifiantly redue omputational omplexity, and the numerial result is also exat. For approximate results, we disuss the approah of only onsidering error events in a single hannel. The results well approximate the exat results in low BEP regions with muh lower omplexity. Index Terms Binary ode, bit error probability, network oding, noisy hannel, nonbinary ode, subgraph deomposition. I. INTRODUCTION The onept of network oding was first proposed in [1] to address the problem of network multiast apaity. After that, a lot of researh has been arried out on network oding. One example of network oding is shown in Fig. 1. Two information bits (b 1 and b 2 ) are multiast from the soure s to two sinks (y and z). The requirement for the multiast senario is that both sinks should reeive both bits orretly. The apaity of all hannels is one. Obviously, the traditional multiast approah with routing only annot meet the requirement [1]. However, oding in the intermediate node w makes it feasible. In the sinks, deoding operations are performed to reprodue the information bits. The result of [1] shows that the network apaity is determined by its minimal-ut. In [2], linear network oding is well addressed. In [3], an elegant algebrai framework is established for network oding. Though original work on network oding assumes that the hannels are error-free [1], network oding for erroneous hannels has attrated more and more researh interest. In [4], the onept of network error orretion is proposed and theoretial limits are derived. In [5], it is shown that hannel and network oding an be asymptotially separated for independent disrete memoryless hannels (DMCs). A similar result is also shown in [6]. In [7], [8], network oding with DMCs is onsidered from an information theoreti point of view. The papers hek by mutual information the rate benefit of intermediate node enoding and show that hannel and network oding usually annot be separated without loss of optimality. A similar result is also shown in [9] for the Aref network. In [10], a lower bound on the soure alphabet size of the network orretion ode is addressed. In [11], streaming multiast with network oding is onsidered for the wireless relay network. In [12], the basi algebrai properties of network error orretion odes are studied for various errors, e.g., random errors by hannel noise, paket erasure due to traffi jam in networks, pakets altered by maliious nodes. Readers are referred to [13] for more papers on network oding with noisy hannels. Here we will investigate the error probability of a noisy hannel network using network oding (oded network). Speifially, we will show how to alulate the sink bit error probability (BEP). The motivation for this work has both pratial and theoretial sides. For a pratial ommuniation network, espeially for a system with finite proessing apability or finite proessing delay, an error is inevitable due to hannel noise. Error-free transmission in a noisy hannel needs an infinite odeword length (and unlimited proessing power and delay) [14]. This annot be allowed in pratie. The optimality of separation of network and hannel odes is asymptotial [5], [6], i.e., it needs error-free hannel odes, whih use an infinite blok length and unlimited proessing apability (delay). In the intermediate node enoding and sink deoding proess, noise disturbed transmitted symbols from different paths are mixed. Transmission errors might propagate (or anel) by intermediate node enoding or sink deoding. For example, in Fig. 1, one bit error in hannel /$ IEEE

2 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER Fig. 1. A network oding example. b and b are the information bits transmitted in the network. P (i =1;...; 9) is the bit error probability of hannel e. 8 denotes the modulo-2 addition operation. e 3 will ause two bit errors after deoding of the sink y, and one bit error in hannel e 5 (or e 6) will ause two bit errors in the sinks y and z. In a oding sheme with a large alphabet size, a bit of the hannel error an propagate severely in the enoding or deoding proess. For example, for the oding sheme in GF(2 3 ), the ode operates on the group of 3 bits. Multiplied by 6, one bit error in the seond bit of the three-bit group will propagate to all three bits (2 2 6=7in GF(8)). Moreover, with the BEP onstraint, some new design approahes an be onsidered. It an be, for instane, an optimal oding sheme to minimize the BEP [15], or optimal transmitted energy alloation with BEP onstraints [16], et. Thus, the BEP is one of the essential performane measures for a oded network with noisy hannels. Next, theoretial aspets are onsidered. If we an alulate the exat sink error probability, it may help to investigate theoretial issues. For example, for the binary symmetry hannel (BSC), the hannel apaity is determined by its BEP [17]. For a oded network omposed of BSC hannels, the BEP may also help to understand the apaity from the soure to sinks. Moreover, for a network with asaded independent DMCs, optimization of the network apaity turns to minimizing the error probability [18]. A similar idea might apply to a oded network with the independent DMCs. The alulation of the soure-to-sink BEP is the first step for this purpose. The organization of the orrespondene is as follows. In Setion II, we desribe the network model. Then, we define the performane measure in Setion III. In Setion IV, we show the sink-bep omputation approah for binary oding shemes. In Setion V, we generalize the approah to odes working on the Galois field (GF)2 m, where m 1. Finally, we propose a redued-omplexity approah in Setion VI. II. NETWORK MODEL The network disussed here onsists of one or more soures and sinks. We denote the network as G = hv;ei, where V and E are the node set and the hannel set, respetively. It is natural to desribe the network with a direted graph [1] [3]. The edges of the graph denote the hannels of the network. The hannels are noisy, memoryless and statistially independent. There might be hannel odes. In this situation, the ode symbols of network oding are enoded by hannel odes, and then transmitted through noisy hannels. In the examples, we assume hannel noise as additive white Gaussian noise (AWGN) with a double-sided power spetral density (PSD) N 0 =2 [19]. The reeived observable (time-disrete vetor) at the end of a hannel is Y = X + n (1) where X (Matries and vetors are underlined in this orrespondene) is the transmitted signal vetor in signal spae [19], and n is the additive noise vetor. If hannel odes are used, the observations from the hannels are input to the hannel deoders, and then the outputs from the hannel deoder are used as network ode symbols. Otherwise, the outputs from the hannels (hard-deision) are diretly used by network oding. The transmitted signal through the hannels an be, for example, multiple phase shift-keying (MPSK), pulse amplitude modulation (PAM) or frequeny shift keying (FSK), et. Below, we will use binary phase shift-keying (BPSK) modulation to simplify the illustration. In a network, a single hannel is usually one setion of the multihop transmission path of a network. Due to noise, the deteted data (or deoded symbols from hannel deoders) at the end of a hannel might differ from its input. This auses an undeteted hannel error. This hannel error may eventually ause an erroneous bit reprodued in the sink. Here the reprodued bits are the output bits from sink deoders of network oding. The hannel error ours with a ertain probability (hannel error probability). Note that here we do not disuss the joint design of network and hannel odes. In the transmitter side of a hannel, network ode symbols are first input to network enoder, and then to the hannel enoder. At the reeiver side, the reeived observations are first deoded by the hannel deoders, and then sent to the network odes. Due to noise, the output symbols from the hannel deoders an be hanged to other values with ertain probabilities. Thus, to the network odes, the ode symbols are transmitted through DMCs, whether they are from the hannel diretly or from a hannel deoder. In the sink, the output from the hannel (or the hannel deoders) are deoded (reprodued) by the network deoder. In addition to the above desription, we make following assumptions: 1) The network is direted, ayli and delay-free, 2) the oding sheme is linear and deterministi. By deterministi, we mean that the ode has been set and remains unhanged during the transmission proess, and 3) the network oding sheme is admissible, i.e., the sink an orretly reprodue soure information if there is no hannel error. III. ERROR PERFORMANCE MEASURE To quantify the error probability performane, we use the BEP in the sink. It is defined as the probability in whih any information bit reprodued in the sink is erroneous. For onveniene, we all it the sink BEP (or briefly BEP) to distinguish it from the hannel BEP. We note that the BEP is a stronger measure than the probability of an error event, whih is defined as the probability in whih any error ours [19]. As we will show, the omputation of the BEP inludes that for the probability of an error event. We assume that in eah time unit, k information bits transmitted from the soure. We use W to denote the sum of bits reprodued (deoded) in all sinks in one time unit. For the multiast network, different sinks normally reeive overlapping information for the k soure bits. Hene, k W. For instane, in Fig. 1, k =2and W =4. Due to the errors in the hannels, part of these W bits might be different from the soure. To ompute the sink BEP, we define an error event as follows. Definition 1: Channel error event: One or more hannels having errors in the network. Briefly, we all it an error event. An error event may ause w (0 w W ) bit errors reprodued in the sinks. w is the error weight (or just weight) of the event. Again, it is the sum of error bits in all sinks. If the probability of the event is P (), the BEP of the event is P b () =P () w W. Thus, the sink BEP an be alulated by adding the BEPs of all distint error events, i.e. P b = P b () = P w () W 2" 2" (2)

3 5190 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 where " is the set of all distint error events. The error events must be distint for the equation to hold. Otherwise, the result is an upper bound ([19] [22]) of the sink BEP. Note that this definition of the BEP is averaged over all sinks. Thus, it is a measure of the whole system. The motivation for us to use the sink BEP is that it atually denotes the quality of servie (QoS) between the transmitters and the reeivers, i.e., how reliable are the reprodued bits in the sinks? Do we have to improve the transmission approah? Also, the BEP onstraint attrats more and more researh effort for Ad Ho wireless networks [23] [26]. This might be one of the most attrative appliations for network oding [27], [28]. We distinguish between a reprodued bit and a reeived bit for the sink. The latter is the bit reeived by the sink from the hannels and it is to be sent to the deoder. The former is the output from the sink network deoding operation. In the notation of [3], the reprodued bits here are the output proesses (for ontinuous transmission) of a sink. The reeived bits are the proesses transmitted (and reeived) in the hannels diretly onneted the sink. IV. BEP CALCULATION FOR BINARY CODES In this setion, we will show how to alulate the sink BEP for binary odes. We separately onsider binary oding shemes sine they are the simplest one (ompared to nonbinary shemes). We an temporarily skip omplex operations (suh as multipliation) from the high alphabet size sheme. We first show exat results with high omplexity, and then give approximate results with low omplexity. We note that though approximate results are easier to alulate and work well in most pratial situations, the exat analysis still has important values. For instane, through exat analysis, we an know how muh we lose from an approximation, and under what ondition approximate results are reliable. Now we give a brief review of the BEP omputation for traditional hannel odes in a single hannel, sine part of our approahes are related to it. A. Review of Channel BEP Calulation In general, the omputation of BEPs (espeially exat results) for hannel odes is a quite diffiult problem, even for short blok odes with hard-deision deoding (see [20] [22], [29], [30]). We assume that a blok ode with a generator matrix G is used. The ode rate is k=n, i.e., k information bits and n oded bits. We assume that hard-deision deoding is used in the reeiver. The odewords C are evaluated as C = I 1 G (3) where I is the information vetor. At the reeiver side, the reeived vetor (after hard-deision) is Y = C + e, where e 2 f0; 1g n is the error vetor due to the hannel noise. A maximum-likelihood deoder finds a odeword (C) with the minimal Hamming distane to Y, and outputs the information vetor (I) orresponding to C. IfI 6= I, then an error ours. The Hamming distane between I and I is alled error weight (w e). To ompute the exat hannel BEPs, we need to know all possible errors (E), their probabilities (P E ), the number of errors (A E ) and error weight (w E) [20], [21], [29], and [30]. Then, the hannel BEP is alulated as w E P e k A EP E : (4) E Equality holds if all E s are distint. To the knowledge of the authors, there is no simple analysis approah to ollet them (E;w E ;P E ;A E ) for most odes (at least for distint E). The analysis equations are only available for the probability of odeword errors for perfet odes (e.g., Hamming odes, repetition odes, and some Golay odes) [20] [22], [29], [30]. Alternatively, one straightforward approah is to find all possible e, and alulate their probabilities and error weights (see [21], [22], [30] for blok odes). The hannel BEP is w e P e = P (e) (5) k e2e where E is the set of all possible e, and P (e) is the probability of e. E an be olleted by enumerating all omponent bits of e. w e an be olleted by performing deoding, sine C is known, and then Y an be found. Thus, it is a brute-fore approah. Equality holds sine different e s are distint (means distint hannel errors). Clearly, there are 2 n 0 1 e s. The omplexity is exponential in odeword length. The approahes above are for exat results. If only approximate results are needed, there are many less-omplex methods to estimate the hannel BEP. For instane, one needs only to onsider the error events with the minimum distane. The results are normally good approximations in the medium-to-high SNR [20] [22], [29]. B. Transfer Matrix For a linear network ode, the enoding and deoding proess is well desribed by the transfer matrix in [3]. Now, we briefly review the transfer matrix, whih denotes the relation between soure information bits (or soure random proesses for ontinuous transmission) and sink reprodued bits (random proesses). The transfer matrix an be evaluated as [3] M = A 1 P 1 B (6) where A is the oding matrix between the soure information bits and the input to the edges onneted to the soure nodes (We all it the network input for onveniene), B is the transfer matrix between the sink reeived bit and reprodued bits, and P is the transfer matrix between the network input and the sink reeived bits. The matrix B determines how the sinks reprodue information bits from the reeived bits. Thus, it an be regarded as the deoding matrix. For onveniene, we all the matrix H = A 1 P the enoding matrix. The reprodued bits at the sinks are alulated as Z = X 1 M: (7) See (8) at the bottom of the page. Here X and Z are soure information bits (row vetor) and sink reprodued bits (row vetor), respetively. From (7), we an see that a row in M orresponds to a soure bit and a olumn to a sink reprodued bit. A nonzero element (say ith row and jth olumn) in M means the ith soure bit is transmitted to the jth reprodued bit (possibly with other e ;1 e ; e ; e ; e ;1 e ;2 M = 1;e 0 0 2;e 1 e ;e e ;e e ;e e ;e e ;e e ;e e ;e 0 0 e ;e e ;e e ;e e ;e e ;e e ;e e ;e 1 : (8)

4 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER bits). Thus, there is a logial path between the ith soure bit and jth sink reprodued bit. Several logial path may share the same physial hannel, if the intermediate node before this physial hannel performs network oding. The transfer matrix for Fig. 1 is shown in (8). This matrix denotes the relation between the two soure bits and the four sink reprodued bits. In a transfer matrix, the indeterminate variables i;e, e ;e and e ;j are the oding variables with the same alphabet size as the ode symbols. From [3], i;e denotes the linear relation between the ith soure symbol and the transmission symbol in hannel e j, and e ;e denotes the linear relation between the input symbol from hannel e i and output symbol to hannel e j in an intermediate node, and e ;j denotes the linear relation between the ode symbols from hannel e i and the jth output symbol (reprodued symbol here) in a sink. Due to limited spae, we do not give any further introdution on the transfer matrix. Readers are referred to [3] for a more detailed desription. C. Error Event and Probability Similar to the BEP omputation of hannel odes, we need to find all distint error events with their probabilities and error weights. Then, (2) an be used to alulate the sink BEP. We have defined an error event to our when one or more hannels have transmission errors. Every hannel is in one of two states: having a transmission error or having no transmission error. Sine hannels are statistially independent, we an ollet all distint error events by enumerating the state of all hannels. The probability of the event is the produt of probabilities of eah hannel state. These events are distint sine the hannels are statistially independent. For example, there are three statistially independent hannels (e 1, e 2, e 3 ) in a network. An error event 1 happens when e 1 is erroneous and e 2, e 3 are orret. Another error event 2 happens when e 1, e 2 are erroneous and e 3 is orret. 1 and 2 are two distint error events, i.e., 1 and 2 annot our simultaneously. Now we ompute the probability of the error event. We assume jej hannels in the network. Sine the hannels are independent, the probability of the error event is P () = e 2E() P e 1 e 2EnE() (1 0 P e ) (9) where P e (i =1;...; jej) is the BEP of the hannel e i, and E() is the set of error hannels for the event. D. Sink Error Weight With the approah to ollet all distint error events and their probabilities, we need to alulate the sink error weight of an event. For this, we need to trak how the errors from the hannels of a network affet the reeived bits (in the intermediate node oding proess and transmission proess along multi-hop paths), and how they are linearly ombined in deoding. Sine the transfer matrix desribes how the information bits are transmitted through the hannels, and how they are enoded and deoded in a oded network, we an ollet the error weight by marking the hannel errors in the transfer matrix, and trak the errors in reprodued bits. First, we define a symbol variable to mark and trak hannel errors. Definition 2: T : A symboli variable used to denote a hannel error bit. It follows the rules: and T 1 T =1 (10) T + T =0: (11) Equation (10) tells that two hannel errors in the same path will anel the transmitted error sine the transmitted symbols are binary. The seond transmitted error will hange the error bit bak to the orret one. Equation (11) means that two bit errors will anel when they are added (XOR) into one bit. We use onstants (0 or 1) to denote no error. When two error bits anel in (10) or (11), 1, and 0 are identity elements ([20], [21]) of multipliation and addition, respetively. In a transfer matrix, a nonzero oding variable, e.g., e ;e, means an input from hannel e i (inluding hannel error) is sent to the output hannel e j.if e ;e =0, the input from hannel e i is not sent to output hannel e j. Thus T 1 e ;e = T; e ;e =1 0; e ;e =0. (12) The same priniples apply to i;e and e ;j. Using T, an error marking algorithm is shown in Algorithm 4.1 to find the error weight in the sinks. The algorithm traks how the hannel errors affet the sink reeived bits and how the errors propagate in the deoding proess. It determines if a reprodued bit is orret by forming the statisti for T. After deoding (multiplied by B), there might be 1+T. It denotes a orret bit added to an erroneous bit. Certainly, the result is an erroneous bit. Thus, we have 1+T = T: (13) In the weight alulation, (13) is not mandatory, sine we only ount the number of T s. It is not affeted by (13). Proposition 1: Algorithm 4.1 alulates the sink error weight for the error event of Definition 1. Proof: Apparently, the sink error weight for an error event an be alulated by heking all reprodued bits (orret or erroneous). This an be ahieved by heking the sink reeived bits and how they linearly ombine in the deoding proess, sine all errors in the sinks are originally from hannel errors. Through the transfer matrix analysis, we know that a nonzero element in the enoding matrix H denotes a logial path between a soure bit and a sink reeived bit. Several logi paths may share the same physial hannels if the intermediate node enodes several inputs into one output. Sine the ode symbols are binary, the nonzero values of ode variable are 1 s. Then, we only need to keep the hannel marks to trak the hannel errors (and ignore the 1 s). Thus, through Step 1 to Step 3, we list all hannels of logial paths from the soure to the sink. For H with hannel marks, one olumn shows how the hannels affet one reeived bit. Sine we assume ayli networks, a hannel error an only distort one sink reeived bit one. They an, of ourse, distort more than one sink reeived bit. Thus, in the algorithm, we only keep one hannel mark for one hannel in one olumn. Hene, in Step 4, we remove the redundant hannel marks. The remaining hannel marks in eah olumn show how the physial hannels (thus hannel errors) affet one reeived bit. Note that the remaining hannel mark an be kept in any logi path (any row), sine if a reeived bit orret or not is only determined by odd or even number of the error mark T in eah olumn of H. For the same reason, the anellation of adding even bit errors is indiated for eah reeived bit (even T s of a olumn in H denote no error, sine the odes are binary). The anellation of even error bits in the same multihop path are denoted by (10). The propagation of a bit error along a path is indiated by (12). Thus, the simplified H in Step 6 an trak if a reeived bit is erroneous or orret, by ounting the number of T s of eah olumn: Odd T s mean an error and even T s denote no error.

5 5192 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 Algorthm 4.1: Error marking algorithm for alulating sink error weight 1: Initialization. Form the matries A, P and B for the network odingsheme. All nonzero elements are temporarily kept as indeterminatevariables as in (8) 2: Calulate the enoding matrix H = A 1 P. 3: Mark hannel. Replae the indeterminate variables ( i;e and e ;e )inh with its seond subsript, whih denotes a hannel. For example, e ;e is replaed by the hannel symbol e 3. 4: Remove redundant hannel symbols: For eah olumn of the matries H, if there are more than one symbols for any single hannel (say e 9 in the seond olumn of (8)), only keep one symbol for this hannel and remove all other symbols for this hannel. 5: Mark error. For eah olumn of the matrix H, if any hannel is assumed to be erroneous in the error event, replae the hannel symbol with T. Set all symbols for the assumed orret hannels to 1s. 6: Calulation of H. Simplify H by using equations (10) (13) 7: Calulation of M. Set all nonzero indeterminate variables in B to 1 and alulate M = H 1 B. Simplify M by equations (10) (13). 8: Make deision for eah reprodued bit. For eah olumn of M,if there are a total of odd T s, then the reprodued bit orresponding to this olumn is erroneous, otherwise, it is orret. 9: Collet error weight and output results. In the deoding proess, no new hannel errors our. The deoding matrix B deides how the reeived bits linearly ombine to form the sink reprodued bit. In deoding, an even number of error bits are anelled by adding them, and one error bit adding one orret bit forms an error bit. These are desribed by (11) (13). Thus, how the errors propagate and anel in the deoding proess are traked by multiplying H with B under the rules (11) (13). Finally, we an determine whether a reprodued bit is erroneous or orret by ounting the number of T s for eah olumn of M. Thus, Algorithm 4.1 alulates the sink error weight of the error event. Q.E.D. We use Fig. 1 for illustration. To simplify the illustration, we assume that all hannels have the same SNR (This is not a requirement for the alulating proedure). First, we assume no hannel ode. Thus, the hannel BEP is evaluated as [19] P e = Q( 2 1 E b =N 0 ) (14) where the Q(1) funtion is the omplementary Gaussian umulative distribution funtion [19], E b is the signal energy per network ode bit. If we assume the hannels e 5, e 6 and e 9 are erroneous and other six hannels are orret, the probability of the event is P (e) =P 3 e (1 0 P e) 6. Now we alulate the sink error weight. First, we replae the indeterminate variables in the enoding matrix with hannel symbols H = e 1e 3 e 1 e 5 e 9 e 7 e 1 e 5 e 9 e e 2 e 6 e 2 e 6 e 2 e 4 : (15) Then, the redundant hannel marks in the same olumn are removed. Then we mark the error hannels e 5, e 6 and e 9 with T. The enoding matrix beomes H = 1 T 1 T T 1 T 0 : (16) 0 T T 1 Then we simplify (16) with (10) (13) and set the nonzero elements in the deoding matrix B to 1s. Multiplying (16) with B, we get the transfer matrix as M = : (17) 0 T T 1 The seond and the third olumn have one T. Thus, we an onlude that the reprodued bits b 2 in the sink y and b 1 in the sink z are erroneous for this event. The error weight is 2. Using a similar proedure, we an find all error events and their weights. Finally, the sink BEP is P b =4P e P e 8 +16Pe 2 P e 7 +40Pe 3 P e 6 +68Pe 4 P e 5 +68P e 5 P e 4 +40Pe 6 P e 3 +16Pe 7 P e 2 +4Pe 8 P e (18) where P e =10 P e. From above analysis, we know that there are 2 jej 01 error events. To have exat results, we must onsider them all. The omplexity is quite high (a more detailed analysis will be given later). Yet, the probability of two or more hannel errors is muh smaller than that of the single hannel error, if the hannel BEPs are low. There are only jej suh error events. The omputational omplexity is greatly redued. Thus, we an get approximate results by only onsidering the events with one hannel error. For the above example, the approximate BEP by only onsidering single hannel error is P b 4P ep e 8 : (19) Fig. 2 shows the alulated and simulation results. In the simulations, two bits (as b 1 and b 2) are ontinuously sent from the soure s to sinks. At the end of eah hannel, the transmitted bits are deteted from the noise disturbed BPSK signals. In node w, two input bits are enoded into one bit. In the sinks y and z, deoding is performed and the results are ompared with the soure bits. The simulation stops until a ertain number of errors (say bit errors) our. In the absissa, we use the SNR of a single hannel. This does not limit the illustration sine all hannels are assumed to have the same SNR. From Fig. 2, we an see that the exat alulated result from (18) mathes the simulation result very well. The two urves oinide in all BEP regions. The approximate result from (19) estimates the simulation result well for the medium-to-low hannel BEP (roughly P e < ). For high BEPs, the approximation loses auray. The simulation BEP is larger than the approximate sine the ignored error events with two or more hannel errors are signifiant in this region. Also, in the approximate results, we an see from Fig. 2 that the alulation auray improves as the hannel ondition improves (low BEP). This is analogous to onventional hannel odes, for whih the minimum distane beomes the dominating event as the hannel ondition improves. As we have disussed, the use of hannel odes does not affet the sink BEP omputation (though it affets the hannel BEP). For the example in Fig. 1, we use the (7; 4) Hamming ode for eah hannel. The generator matrix is ([30], [20]) G = : (20) We hoose this simple ode sine the exat hannel BEP with harddeision maximum-likelihood deoding an be alulated as ([30], [20]) P e = p 2 (9 0 26p +30p p 3 ) (21) where p is the hannel transfer probability, and it an be alulated as p = Q( E ). Inserting (21) into (18), we an get the exat N sink BEP. The numerial results are shown in Fig. 3. We an see that our approah an be used for a oded network with hannel odes. From the above disussion on the exat BEP alulation, we an see that there is a similarity between the approah for the sink BEP and the alulation of the exat hannel BEP for hard-deision deoding (see

6 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER Fig. 2. The simulated and alulated BEP for Fig. 1 E =N is the signal-to-noise ratio of a single hannel. Fig. 3. The simulated and alulated BEP for Fig. 1 with (7; 4) Hamming odes as hannel odes. E =N is the signal-to-noise ratio of a single hannel. (5) or [30], [20]). They both enumerate all possible errors in the hannels, and then alulate the error weight. Yet, the approahes of evaluating the error weight are quite different. Essentially, this is due to one important differene between network oding and hannel oding: Information symbols of network odes are transmitted through hannels and then enoded. Thus, they might be distorted by hannel noise before entering enoders (in intermediate nodes). Yet, for hannel odes, the information symbols are not disturbed by hannel noise before enoding. In the weight enumeration of traditional odes, we only need to add the hannel errors to the odewords, and then deode to get the error weights [30], [20]. The odewords are found by multiplying the information symbols with G (see (3)). Yet, for network odes in noisy hannels, the odewords annot be found in this way, due to distortion before enoding. Thus, we need to trak the ode symbols in the whole network of noisy hannels: How the hannel errors propagate and anel in the enoding proess or along the multihop paths. Our algorithm ahieves this result. Speifially, we use (10) (13) to trak the error bits in the sink weight alulation. We also remove the redundant hannel marks (Step 4 in Algorithm 4.1), to apture intermediate node enoding. There is also a similarity for approximate results between our approah and that for hannel odes. They both only onsider the error events dominant in the low BEP regions. For the sink BEP, single hannel errors are the dominant events. For the hannel BEP, the events with the minimum distane are dominant.

7 5194 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 sequentially. In the intermediate node m4, the output symbols are enoded as a +2b. The symbols transmitted in the hannels e7;e8;e9 and e10 are shown in Fig. 4. Clearly, they are pairwise full-rank. Before being sent to the binary hannels, eah symbol is demapped into two bits. In the sink, the deoding operation is performed in GF(4). For example, in the sink t6, two reeived symbols from hannels e15 and e18 are r6;1 = a + b and r6;2 = a +2b, respetively. Then, the deoding approahes are 2r6;1 + r6;2 = 2(a + b) +a +2b = a, and r6;1 + r6;2 = a + b + a +2b = b. B. Error Event Enumeration Similar to binary odes, the BEP alulation of nonbinary odes follows the steps of the error event enumeration and the weight alulation. An error event of nonbinary odes is also defined as one or more hannels having transmission errors. To desribe the transmission error of a hannel, we make the following definition. Definition 2: Error symbol (T e ) from hannel e : The differene (in GF(2 m )) between the reeived symbol (S r) and the transmitted symbol (S ) for hannel e, i.e. Fig. 4. A network oding sheme needs the alphabet size larger than 2. The network ode works in GF(4). a and b are the symbols in GF(4). a and b (k =1;...; 6) are the reprodued bits of a and b in the sink k. A. Introdution V. BEP CALCULATION FOR NONBINARY CODES We have shown the BEP omputation of binary odes. For some network multiast senarios, a high alphabet size is neessary to ahieve min-ut apaity. Now we will generalize the BEP omputation to nonbinary oding shemes. The approah still uses the proedure of hannel error marking and traking. Thus, the result of this setion is a generalization from binary oding shemes. One example of network oding with a high alphabet size is shown in Fig. 4. This figure is a modified version of ombination networks in [31] [33]. In the figure, the soures s1 and s2 both have a sequene of information bits (denoted as a i and b i, i =1; 2;...is the time index) to be multiast to six sinks (t1;...;t6). Sine the hannels use BPSK modulation, one bit an be transmitted in one hannel in eah time interval. Here we assume again that no hannel odes are used. There is no binary salar network ode meeting the multiast requirement: Eah sink onnets to two of the intermediate nodes m5;m6;m7 and m8. During one time interval, eah sink reprodues two soure bits from two reeived hannel bits, whih ome from two of intermediate nodes. Note that two bits per time interval for one sink is an averaged rate. Atually, it should be four bits every seond time interval. Thus, information transmitted in the hannels e7;e8;e9 and e10 must be pairwise full-rank. Here pairwise full-rank means that one bit (or a symbol for nonbinary odes) annot produe another bit (symbol) by a linear transformation. Pairwise full-rank is impossible for a binary salar oding sheme. For two information bits (a i and b i), there are three possible pairwise full-rank ombinations, i.e., a i, b i and a i + b i. Yet the maximal possible flow from the soures to eah sink is two by the minimal-ut bound [31] [33]. Thus, a higher alphabet size is neessary to ahieve the maximal possible flow. One oding sheme is shown in the figure. For this network, we use the alphabet size four, i.e., GF(4). Though a ternary oding sheme an ahieve maximal possible flow, we use GF(4) to adapt to the binary hannels. Then, eah ode symbol (denote as a and b and ignore the time index) is mapped from two hannel bits in a natural way. These two bits are reeived T e S r + S: (22) Here we use addition instead of substration sine they are interhangeable in GF(2 m ) [20], [21]. We all T e the hannel error symbol. Sine the ode is in GF(2 m ), the range of T e is 0; 1;...; 2 m 0 1. T e = 0 means no error. Using T e, an error event means one or more hannels having nonzero T e. The probability of the event is the produt of the probabilities of the hannel states. The hannel states are identified by the values of T e. One value of T e gives a state of the hannel. Then, we an ollet all distint error events by enumerating the states of all hannels in the network. The memoryless and independene assumption for the hannels ensures the error events being distint. The values and probabilities of T e are determined by the hannel model and transmission sheme. There is no uniform approah. For example, if BPSK is used, one ode symbol needs m hannel transmissions (assuming no hannel odes used). Eah signal transmits one bit information. The error bit () is the reeived bit adding (modulo-2) the transmitted bit. Obviously, 2 f0; 1g. Again, = 0means no error. The hannel error symbol an be evaluated from the error bits by T e = m 2 m0i i (23) i=1 where i is the error bit in the ith transmission of the onsidered symbol. The probability of a hannel error symbol is the produt of the probabilities of error bits, i.e. m P T = P (24) i=1 where P T and P are the probability of a onsidered error symbol and its ith error bit, respetively. Clearly, for eah hannel, T P T =1. For example, in Fig. 4, eah symbol uses two hannel bits. Assume that the transmitted bits are f1; 1g and the reeived bits are f0; 1g. The error bits are f1; 0g. From (23), the hannel error symbol is =2. The probability of the hannel error symbol is P2 = P e 1 (10P e ) from (24), where P e is the hannel BEP. Then, we an ollet hannel error symbols and their probabilities. C. Error Weight Calulation Now we alulate the sink error weight for the error event. Equivalently, we alulate the sink reprodued error symbol. Here a sink reprodued error symbol is the differene between the sink reprodued

8 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER symbol and the orret symbol. The approah also traks error propagation and anellation in the network. For this purpose, we alulate the error symbol for eah sink reeived symbol and trak how these error symbols linearly ombine in the deoding proess. We again use the transfer matrix defined in [3]. Unlike binary oding, we have to onsider the multipliation of nonbinary odes, whih an hange the error symbol. Eventually, it may substantially hange the error weight in the sink. In the algorithm for binary odes, we ignore the multipliation operation and replae the indeterminate variable of the transfer matrix with hannel marks. For nonbinary odes, the indeterminate variables are replaed by the speified value and hannel mark. For example, in Fig. 4, the indeterminate variable e ;e between hannels e 6 and e 10 is replaed by 2e 10. The speified value should preede the hannel mark, sine enoding by ode variables is the operation inside the node, and the hannel mark is for the output hannel of the node. In Fig. 4, the elements of the transfer matrix for the reeived symbol in hannel e 20 is 1;e e ;e e ;e : (25) 2;e e ;e e ;e Then we speify the indeterminate variables and mark hannels (1s are ignored). The result is e 3 e 10 e 20 e 62e 10e 20 : (26) Algorithm 5.1: Error marking algorithm for alulating the sink error weight of nonbinary odes 1: Initialization. Form the matries A, P and B for the network ode. All nonzero elements are temporarily kept as indeterminate variables. 2: Calulate the enoding matrix H = A 1 P. 3: Speify Variables and Mark hannel. Replae the indeterminate variables ( i;e and e ;e )inh with the speified values and their seond subsripts, whih denote hannel symbols. For example, e ;e is replaed by the speified value 2 and hannel symbol e 10. 4: Calulate error symbols for eah sink reeived symbol: Add all rows of the matries H. Simplify the results by ombining the terms with ommon hannel marks. Replae the hannel mark with the hannel error symbol, and simplify by (27). The result is the sink reeived error symbols (row vetor). 5: Calulate the reprodued error symbols. Multiply the sink reeived error symbol vetor with the deoding matrix. 6: Find error events and evaluate their weights. For eah error event, speify the error symbol for all hannels and alulate the sink reprodued error symbol vetor. 7: Collet error weight and output results. After speifying variables and marking hannels, we add the rows of the enoding matrix. The sum is simplified to a produt form and only one hannel mark is kept for eah hannel. This is neessary sine multiple hannel marks of different rows (of the same olumn) atually denote one physial hannel. The ode symbols from different hannels are enoded into one symbol before they are sent to this physial hannel. Thus, the produt form desribes how the hannel errors affet a reeived symbol in the sink. In the example, we add the rows of (26), and simplify the result to the produt form as r e = e 3 e 10 e 20 + e 6 2e 10 e 20 = (e 3 + e 6 2)e 10 e 20. The ode symbols (together with error symbols) from e 3 and e 6 are enoded into one symbol before being transmitted in hannels e 10 and e 20. Thus, the simplified produe form an apture how hannel errors affet the sink reeived error symbol. In the transfer matrix [3], a path is desribed by the multipliation of ode variables along the path. The error at the end of a path is the sum of error symbols of the omponent hannels. This is beause the output symbol of eah hannel is the input symbol adding the error symbol of the hannel. Thus, the errors from asaded hannels are the sum of error symbols of the hannels. Then, the omputing rules of two hannel error symbols follows: T e T e = T e + T e (27) where the operation + is performed in GF(2 m ). In the enoding and deoding proess, the reeived symbols of the node are multiplied (in GF(2 m )) with ode variables (speified values of i;e, e ;e and e ;j). Sine the reeived symbol is the sum of the error symbol and orret symbol, the error symbols are multiplied by the same ode variables in the oding proess too. For the operation of T e e ;e (the same for i;e and e ;j), it just follows the multipliation rule of GF(2 m ). After multiplying the deoding matrix, we an get the sink reprodued error symbol. Similar to (13), after multiplying the deoding matrix, there might be T e + C, where C is a onstant. It also means no error. Thus, T e + C = T e. The algorithm is shown in Algorithm 5.1. Proposition 2: Algorithm 5.1 alulates the sink error weight for the odes in GF(2 m ). Proof: The proof is basially the same as that for Proposition 1, sine Algorithm 4.1 and Algorithm 5.1 follow roughly the same proedure. One major differene for nonbinary odes from binary odes is that the ode symbols (and error symbols) might be hanged by the multipliation of enoding and deoding. This effet is aptured by the multipliation of error symbols and oding variables (in GF(2 m )). Note that the odes disussed here are linear, and transmitted symbols and error symbols (see (22)) are multiplied by the same ode variables. The effet of two symbol errors in the same path is desribed by (27), as we have disussed. The addition of two error symbols in the enoding or deoding proess follows the addition of GF(2 m ). Thus, all types of error propagation and anellation in the enoding and deoding proess are aptured by our algorithm. Step 4 of Algorithm 5.1 is another important differene from Algorithm 4.1. For binary odes, heking an odd or even number of hannel error bits is enough to deide if a sink reeived bit is orret or erroneous. This is not enough for nonbinary odes. We have to trak the whole enoding proess (multipliation in GF(2 m )), and we need to remove redundant hannel marks. Thus, in Step 4, we add the elements of the same olumn, and simplify them to produt forms. The results alulate the reeived error symbols (in the omputing rules defined above). In the deoding proess, how different reeived error symbols linearly ombine to produe reprodued error symbols is desribed by the deoding matrix. Thus, we an get the error symbol for the hannel event. Here we do not give further omments, sine other details are the same as the proof of Algorithm 4.1. Q.E.D. We use the omputation of the error weight in the sink t 3 as an example. From (25) and (26), the reeived error symbol for hannel e 20 is r e =(e 3 + e 6 2)e 10 e 20. In the same way, the reeived error symbol for hannel e 13 is r e = e 1 e 7 e 13. The deoding submatrix for sink t 3 is B t = Thus, the sink reprodued error symbol for t 3 is (a 3 ; b 3 )=(r e ;r e )B t : (28) =(e 1e 7e 13; (e 1e 7e 13 +(e 3 + e 62)e 10e 20)3): (29)

9 5196 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 Fig. 5. The simulated and exat alulated BEP for Fig. 4. E =N is the signal-to-noise ratio of a single hannel. Replaing hannel marks with hannel error symbols and simplifying using (27), we an get the hannel error symbols for the reprodued a as T e + T e + T e, and b as (T e + T e + T e +(T e + T e 3 2) + T e + Te )) 3 3. An error event has Te =2, Te =3 and all other hannel error symbols are zero. After speifying hannel error symbols, we an get the sink error symbols for a as 2, and b as ( ) 3 3=2. The error symbol 2 has a weight 1. Thus, this error event in sink t 3 has the weight 2. In the same way, we an evaluate the error weight for all error events. Now we analyze the omplexity of our approah. We onentrate on nonbinary odes sine they are more general than binary odes. Sine eah edge has 2 m states, and only an all-zero state means no hannel errors, there are 2 mjej 0 1 error events. For the exat result, we need to alulate the weights and probabilities of all error events. For eah event, there are jej01 multipliations to alulate its probability (We assume that the error probabilities of eah hannel are known). Thus, total number of multipliation is upper-bounded by jej2 mjej. Some error events have zero-weight. We do not have to alulate their probabilities. Further, we do not have to alulate the probabilities of eah event from the beginning. We an keep the hannel probabilities as variables, and speify them in the final stage. The omplexity might be greatly redued, sine many terms are anelled or merged (see the example for Fig. 1, only trivial multipliations are used in the final stage). Yet, the omplexity redution with this approah depends on the weight and hannel BEPs, whih depend on network topologies, oding sheme and hannel onditions, et. There is no uniform result for all situations. In the weight alulation, all error events use the same H until Step 4 in Algorithm 5.1. From Step 4 to Step 6, we need to perform addition and multipliation in GF(2 m ). The omplexity is hard to estimate without giving a speifi network topology and oding sheme. We measure the omplexity in the number of ases. It is the number of error events 2 mjej. Thus, the omplexity exponentially inreases with the number of edges in the network. The alulation is hard (non-polynomial) for a network with many hannels. However, if the hannel BEP P e 1, we an use approximate approahes, whih only onsider the errors in single hannel. There are jej(m01) error events. Then, the multipliation for alulating probabilities is upper-bounded by mjej 2. The number of ases for omputing the error weight is about mjej. VI. COMPLEXITY REDUCTION BY SUBGRAPH DECOMPOSITION Above, we showed how to ompute the sink BEP, and the omplexity of the approah is exponential in mjej (for exat results). The problem beomes intratable for a large network. For Fig. 5, there are 22 hannels with four states for eah hannel. Thus, there are totally 4 22 error events to onsider. We have shown a redued-omplexity approah that only onsiders the single hannel error event. Yet, the approah is only useful for the medium-to-low BEP region. Now we show another omplexity redution approah based on subgraph deomposition. The result is exat in all BEP regions. The approah is based on the following observation: When we ompute the sink BEP for a ertain sink, for instane, for sink t 3 in Fig. 5, only part of the network (hannels e 1 ;e 3 ;e 6 ;e 7 ;e 10 ;e 13 and e 20 ) affet its BEP. Other hannels are unrelated, sine no information to t 3 is transmitted over them. The omplexity is exponential in the number of hannels. Thus, we an greatly redue omplexity, by omputing the BEP of an individual sink. Yet, we need to find the relation of the sink BEP and the BEP of the individual sink. Now we give a proposition on this. First, we define the number of the information bits reprodued by sinks. Definition 3. Sink Information Ratio: The ratio of the number of information bits reprodued by the individual sink to that of all sinks (in one time unit). We assume that information is measured in bits, and W bits are reprodued in all sinks in one time unit. Among them, the sink t i reprodues W t bits. Then the sink information ratio for t i is l t W t =W. From the definition of the sink BEP, it is an averaged result over all sinks. For a given sink (say sink t i ), the BEP (P b (t i )) is alled the BEP of an individual sink. It is evaluated by only onsidering the errors in the hannels (denoted as E[i]) related to information transmission to sink t i. Thus P b (t i)= w i;j P ( i;j ) W t (30) where i;j is the error event defined in E[i] (only onsider the hannel states in E[i]), and P ( i;j) and wi;j is the probability and error weight of i;j, respetively.