JAIST Reposi. A Lower Bound Analysis of Hamming Di a Binary CEO Problem with Joint Sour Coding. Title

JAIST Reposi https://dspace.j Title A Lower Bound Analysis of Hamming Di a Binary CEO Prolem with Joint Sour Coding He, Xin; Zhou, Xiaoo; Komulainen, P Author(s) Markku; Matsumoto, Tad Citation IEEE Transactions on Communications, 353 Issue Date 05--09 Type Journal Article Text version author URL Rights http://hdl.handle.net/09/3000 This is the author's version of the Copyright (C) 05 IEEE. IEEE Transa Communications, 64(), 05, 343-353 use of this material is permitted. P from IEEE must e otained for all o any current or future media, includi reprinting/repulishing this materia advertising or promotional purposes, collective works, for resale or redi servers or lists, or reuse of any co component of this work in other work Description Japan Advanced Institute of Science and

A Lower Bound Analysis of Hamming Distortion for a Binary CEO Prolem with Joint Source-Channel Coding Xin He, Xiaoo Zhou, Memer, IEEE, Petri Komulainen, Memer, IEEE, Markku Juntti, Senior Memer, IEEE and Tad Matsumoto Fellow, IEEE Astract A two-node inary chief executive officer (CEO) prolem is investigated. Noise-corrupted versions of a inary sequence are forwarded y two nodes to a single destination node over orthogonal additive white Gaussian noise (AWGN) channels. We first reduce the inary CEO prolem to a inary multiterminal source coding prolem, of which an outer ound for the rate-distortion region is derived. The distortion function is then estalished y evaluating the relationship etween the inary CEO and multiterminal source coding prolems. A lower ound approximation on the Hamming distortion (HD) is otained y minimizing a distortion function suject to constraints otained ased on the source-channel separation theorem. Encoding/decoding algorithms using concatenated convolutional codes and a joint decoding scheme are used to verify the lower ound on the HD. It is found that the theoretical lower ounds on the HD and the computer simulation ased it error rate performance curves have the same tendencies. The differences in the threshold signal-to-noise ratio etween the theoretical lower ounds and those otained y simulations are around.5 db in AWGN channel. The theoretical lower ound on the HD in lock Rayleigh fading channel is also evaluated y performing Monte Carlo simulation. Index Terms Hamming distortion lower ound, inary CEO prolem, inary multiterminal source coding, rate-distortion outer ound. I. INTRODUCTION THE chief executive officer (CEO) prolem where a CEO aims at reproducing a common source which cannot e directly oserved was first introduced y Berger et al. in []. The CEO prolem has attracted a lot of attention, not only from pure information theoretic interest ut also This work has een in part supported y Academy of Finland, and in part performed in the framework of the FP7 project ICT-69555 RESCUE (Linkson-the-fly Technology for Roust, Efficient and Smart Communication in Unpredictale Environments) which is partly funded y the European Union. X. He and T. Matsumoto are with the School of Information Science, JAIST, - Asahidai, Nomi, Ishikawa, Japan 93-9 (e-mail: hexin,matumoto}@jaist.ac.jp) and the Centre for Wireless Communications and Department of Communications Engineering, University of Oulu, FI- 9004 Finland (e-mail: xin.he, tadashi.matsumoto}@ee.oulu.fi). X. Zhou was with the Centre for Wireless Communications and Department of Communications Engineering, University of Oulu and is now with School of Computer Science and Technology, Tianjin University, Tianjin, China (email:xiaoo.zhou@tju.edu.cn). P. Komulainen was with the Centre for Wireless Communications and Department of Communications Engineering, University of Oulu and is now with MediaTek, Oulu, Finland (e-mail: petri.komulainen@mediatek.com). M. Juntti is with the Centre for Wireless Communications and Department of Communications Engineering, University of Oulu, FI-9004 Finland (email: markku.juntti@oulu.fi). from practical application viewpoint such as wireless sensor networks (WSNs). The quadratic Gaussian CEO prolem, a particular case of the CEO prolem, where the source and the multiple oservations are assumed to e jointly Gaussian distriuted, was studied in [], [3], where an explicit form of the ratedistortion function was derived. Chen et al. derived an upper ound on the sum-rate for the CEO prolem and proposed rate allocation schemes y exploiting the contra-polymatroid structure of the achievale rate region in [4]. Besides the theoretical work, the performance on minimum achievale distortion of a successive coding strategy [5] ased on a generalization of Wyner-Ziv source coding was evaluated in [6], and the optimal rate allocation scheme to achieve the minimum distortion under a sum-rate constraint was further proposed in [7] for the successively structured Gaussian CEO prolem. In this work, we focus on a inary CEO prolem, in which a inary source is estimated y the CEO through multiple deployed nodes. There are many practical applications of the inary CEO prolem, for example, a inary data gathering sensor network, distriuted detection using multiple sensors [8], power-distortion tradeoff in sensor networks [9], and wireless mesh network (WMN) with lossy forwarding (LF) [0]. An iterative joint decoding algorithm for the inary data gathering WSN, which is a direct application of the inary CEO prolem, was proposed in [], where a convolutional code is applied at the sensor node. A coding scheme ased on the parallel concatenated convolutional codes was proposed in [], where the extrinsic log-likelihood ratios (LLRs) are weighted y the oservation error proailities at the decoder. Moreover, the capacity of the equivalent parallel channel was derived to verify the it error rate (BER) performance taking into account the error proaility of the oserved data sequence. In [3], an adaptive i-modal decoder for a inary source estimation involving two sensors was proposed ased on the modified extrinsic information transfer (EXIT) chart analysis. A convergence property analysis of the iterative decoding algorithm was presented ased on the modified EXIT chart analysis for inary data gathering WSNs in [4]. It shows that iterative process is less useful if the channel quality is very good or the oservation accuracy is very low. In [0], we proposed an encoding/decoding technique which can significantly improve the BER performance y exploiting the correlation

knowledge through the LLR updating function [5], for oth a inary independently and identically distriuted (i.i.d.) source and a inary Markov source. In [6], we proposed a nonnegative constrained iterative algorithm for estimating the oservation error proailities in a WSN having an aritrary numer of sensors. For the inary CEO prolem, most of the previous works focus on the design of practical encoding/decoding algorithms. This motivates us to analyze a prolem that how small a distortion level the CEO can achieve from the rate-distortion perspective. Hence, the primary goals of this work are to theoretically provide a lower ound on the Hamming distortion (HD), corresponding to the it error proaility (BEP), and to verify the lower ound y making performance comparison with a practical encoding/decoding algorithm. The major contriution is the derivation of a HD approximation ased on an information-theoretic outer ound resulting in a lower ound (approximation) on the HD (or BEP) for a two-node communication network with joint source-channel (JSC) coding. Its ojective is to estimate a single inary source over two orthogonal additive white Gaussian noise (AWGN) channels. For the simplicity of analysis, we focus on orthogonal transmissions from two nodes to the destination, and we sperate the stages for JSC decoding and the final decision on the common source [4], [7], [8]. Hence, deriving the theoretical lower ound on the HD is equivalent to minimizing a distortion function suject to a series of inequalities otained ased on the source-channel separation theorem for lossy source coding. In order to solve the minimization prolem, we first model the source coding of this two-node network y the inary CEO prolem. We then reduce the inary CEO prolem to a inary multiterminal source coding prolem, which plays the core role in solving the main prolem. An outer ound for the ratedistortion region of the inary multiterminal source coding prolem is then derived y providing the converse proof. We estalish the relationship etween the inary CEO prolem and the inary multiterminal source coding prolem in terms of the distortion function. Finally, the minimization prolem is formulated in the framework of convex optimization. It should e emphasized here that our purpose is not to derive a tight rate-distortion ound for the inary CEO prolem. Instead, we focus on the derivation of a lower ound that can e used as a reference of the BER performance curves of the encoding/decoding algorithms, including the technique proposed in [0] and [6]. The rest of the paper is organized as follows. In Section II, the system model and the prolem to e solved are descried. The derivation of the outer ound and its proof for the inary multiterminal source coding prolem is detailed in Section III. The prolem of how to otain the lower ound on the HD is formulated in Section IV. Section V provides the numerical results of simulations as well as their corresponding lower ounds. Finally, we conclude this work in Section VI with several concluding statements. II. PROBLEM STATEMENT Notation. The uppercase and lowercase letters are used to Fig.. The astract system model of estimating a single source through two independent nodes with joint source-channel coding. denote random variales and their realizations, respectively. The alphaet set of a random variale X is denoted y X. Let X n and x n represent a random vector and its realization, respectively, with the superscript n eing the length of the vector (lock length). We use t to denote the time index and i to denote the index of a node. The system model of estimating a single source through two nodes is depicted in Fig.. A common i.i.d. source X produces a sequence x n = x(t)} n t= y taking values from a inary set X = 0, } with equal proaility. Source X is oserved y two nodes and forwarded to a single destination. Due to the inaccuracy of the estimation and/or limited received signal power at nodes, such as in WSN and WMN, the sequences received y the nodes may contain errors, and the nodes still forward the erroneous sequences to the destination, which is referred as LF [0], []. The error proailities Pr(x (t) x(t)) and Pr(x (t) x(t)) are denoted as p and p, respectively, i.e., Pr(z i (t) = ) = p i for the inary noise sequence z n i = z i (t)} n t=, i =,. At the nodes, the noisy versions x n = x (t)} n t= and x n = x (t)} n t= of x n are separately encoded y two JSC encoders to generate symol sequences s k = s (t)} k t= and sk = s (t)} k t= with coding rates r i = n/k i, i =,. The symol sequences s k and s k are then transmitted to the destination over two orthogonal AWGN channels, as y ki i = h i s ki i + w ki i, i =,, () where h i and w ki i = w(t)} ki t= represent the channel gain and the AWGN sequence at the destination, respectively. The orthogonality can e achieved y any scheduled multiple access scheme, like time division multiple access (TDMA), i.e., and sk can e transmitted at different time intervals. The s k destination performs JSC decoding to form estimates ˆx n i of the sequences x n i, i =,. We define the expected Hamming distortion measures E[ n n t= d(x i(t), ˆx i (t))] D i + ϵ to evaluate the error proaility Pr(x i (t) ˆx i (t)) with, if xi (t) ˆx d(x i (t), ˆx i (t)) = i (t), () 0, if x i (t) = ˆx i (t), and ϵ representing an aritrarily small positive numer. Finally, the destination reconstructs the source information x n of which the estimate is denoted as ˆx n ased on a decision rule from ˆx n and ˆx n. Therefore, the distortion measure E[ n n t= d(x(t), ˆx(t))] D + ϵ can e formulated as a In WMN applications, the nodes correspond to the transceivers in the multiple routes. In a WMN, a source communicates with a destination through multiple intermediate nodes if they are not within the communication coverage. If errors are allowed in the messages forwarded y the intermediate nodes, the WMN can e also modeled as the model shown in Fig. [9].

3 Fig.. nodes. The astract model of the inary CEO prolem with two independent Fig. 3. The inary multiterminal source coding prolem for two correlated inary sources. function of D i, i =,, as D = f(d, D ), where function f( ) is detailed in susection IV-A. It should e emphasized here that function D = f(d, D ) limits the decoding scheme to which first reconstructs x n and x n and then makes the decision on x n from those reconstructions (it is referred to as sequential decoding), as shown in Fig.. The optimality of such a decoding scheme is an open prolem, ut it is definitely of interest for practical systems. Furthermore, f(d, D ) largely depends on the decision rule, i.e., there exists different function f(d, D ) for different decision rules. According to the source-channel separation theorem for lossy source coding [3], distortions D and D can e achieved if the following inequalities hold: R (D ) r C(γ ), (3) R (D ) r C(γ ), where R i (D i ) is the rate-distortion function for the source coding and C(γ) is the Shannon capacity using Gaussian codeook 3 with the argument γ denoting the signal-to-noise ratio (SNR) of the channel. As stated aove, r and r represent end-to-end coding rates of two links. Our goal is to derive the theoretical lower ound on the HD for the system shown in Fig.. It is equivalent to minimizing the expected Hamming distortion D through a function f(d, D ) under constraints shown in (3), as min D,D D = f(d, D ) (4) s.t. (3). The minimization eing performed in (4) is for a specific system which maps the average distortions D and D to D, since function f(d, D ) is defined for designated decision rules. To achieve this goal y solving (4), we turn to derive the rate-distortion function R i (D i ) for the prolem shown in Fig. and to estalish the function D = f(d, D ) for the decision rule used at the destination. III. RATE-DISTORTION REGION ANALYSIS A. Source Coding In network information theory, the source coding of the communication system shown in Fig. is modeled y the It has een assumed in this setup that ) each encoder uses joint typicality encoding and inning ased on random coding arguments, and the decoder performs joint typicality decoding with a sufficiently large n to achieve the average distortion D i as in the Berger-Tung source coding prolem []; ) the errors occurring in each sequence x n i are i.i.d. In the practical system, we use random interleavers to asymptotically make this assumption practical. As shown in section V-B, the simulation results are consistent with the lower ound calculation ased on majority logic decision. 3 For one dimensional signal, C(γ) = log ( + γ), and for two dimensional signal, C(γ) = log ( + γ) [4]. inary CEO prolem. The astract model of the inary CEO prolem is illustrated in Fig.. In order to derive the ratedistortion function R i (D i ), we first reduce the inary CEO prolem to a inary multiterminal source coding prolem. An outer ound for the rate-distortion region which is determined y the rate-distortion function R i (D i ) is then derived for the inary multiterminal source coding prolem through the converse proof, as in the Gaussian case [5]. The inary multiterminal source coding prolem which we consider is depicted in Fig. 3. Since random sources X n and X n originate from the common source X n, the random variale pair (X, X ) follows a joint proaility distriution Pr X,X (x, x ) given y Pr X,X (x, x ) = ρ, if x x, ( ρ), otherwise, (5) where ρ = Pr(x x ) is the correlation parameter etween the sources X and X, i.e., X can e seen as the output of a inary symmetric channel (BSC) with the crossover proaility ρ where X is the input. Two encoders separately encode X n and X n at rates R and R as φ :X n M =,,, nr }, φ :X n M =,,, nr }. The encoder output sequences U = φ (X n ) and U = φ (X n ) are transmitted to a common receiver. It jointly decodes the received samples to construct the estimates ( ˆX n, ˆX n ) of the source pair (X n, X n ) denoted as ( ˆX n, ˆX n ) = ψ(φ (X n ), φ (X n )). For given distortion values D [0, ] and D [0, ], the rate-distortion region R(D, D ) is defined as R(D, D ) = (R, R ) : (R, R ) is admissile such that E n } d(x i (t), ˆx i (t)) D i + ϵ, i =,. n t= B. Outer Bound for Rate-Distortion Region We provide a ound R o (D, D ) of the rate-distortion region R(D, D ). Definition : R o (D ) = (R, R ) : R R H [ρ H ( R )] H (D ) }, (6)

4.5.4 Slepian Wolf (ρ=0.5, D =D =0) Berger Tung inner ound (ρ=0.5, D =D =0.005) 0.9 0.8 Wyner Ziv rate distortion region Outer ound.3 Outer ound (ρ=0.5, D =D =0.005) 0.7 R.. 0.9 0.8 0.7 0.6 R 0.6 0.5 0.4 0.3 0. 0. ρ = 0.3 0.5 0.5 0.6 0.7 0.8 0.9...3.4.5 R Fig. 4. The comparison of R o (D, D ), Berger-Tung inner ound and Slepian-Wolf admissile rate region. The correlation ρ etween two sources is set at 0.5. with H (a) = a log (a) ( a) log ( a) and H (a) representing the inary entropy function and its inverse function, respectively 4. The operator calculates the inary convolution of the two variales, i.e., a = a( ) + ( a). R o (D ) = (R, R ) : R } R H [ρ H ( R )] H (D ), (7) R o (D,D ) = (R, R ) : } R + R + H (ρ) H (D ) H (D ). (8) For every D [0, ] and D [0, ], R o (D, D ) = R o (D ) R o (D ) R o (D, D ). (9) In what follows, we prove that R o (D ), R o (D ) and R o (D, D ) are the supersets of the regions of R(D, D ). It means that the following theorem holds. Theorem : R o (D, D ) is an outer ound for the ratedistortion region R(D, D ); i.e., R(D, D ) R o (D, D ). C. Proof Proof of Theorem (Converse): To prove Theorem, the following three different cases of the inary multiterminal source coding prolem are considered. Case. In order to prove that R(D, D ) R o (D ), we assume that the rate pair (R, R ) R(D, D ) and show that this implies that (R, R ) R o (D ). In the proof, X n is first reconstructed without constraint on D which results in (5), and then ˆX n is regarded as the side information to recover 4 The inverse function H (a) : [0, ] [0, ] only takes values from the interval [0, ] since distortion is assumed to e within this range. 0 0 0.05 0. 0.5 0. 0.5 0.3 0.35 D Fig. 5. The comparison of Wyner-Ziv rate-distortion region and derived outer ound. The correlation ρ etween two sources is set at 0.3. X n. Assume that a rate pair (R, R ) achieves distortion D, then n (R + ϵ) H(U ) H(U U ) (0) = I(X n ; U U ) () = I(X n ; U, U ) I(X n ; U ) () I(X n ; ˆX n ) I(X n ; U ), (3) where the steps are justified, ecause (0) conditioning reduces entropy, () U is a function of X n, () the chain rule of mutual information, (3) X n (U, U ) ˆX n forms a Markov chain. Now our aim is to lower ound I(X n ; ˆX n ) and upper ound I(X n ; U ). Since I(X n ; ˆX n ) = H(X n ) H(X n ˆX n ) = n H(X n ˆX n ), to lower ound I(X n ; ˆX n ) is equivalent to upper ound H(X n ˆX n ). According to the Fano s inequality, we have H(X n ˆX n ) n H (D ) + n D log( X ) = n H (D ). (4) On the other hand, since I(X n ; U ) = H(X n ) H(X n U ) = n H(X n U ), an upper ound on I(X n ; U ) corresponds to the lower ound on H(X n U ). Oserving that X n X n U forms a Markov chain, it can e shown that H(X n U ) nh (ρ β) y [6, Corollary 4], where β = n H [H (X n U )]. A more detailed explanation is given in Appendix A. Since the inary convolution is monotonically increasing with respect to β if ρ is fixed, we need to find the minimizing value of β to lower ound H (ρ β) 5. We also have the rate constraint on R as n (R + ϵ) H(U ) = I(X n ; U ). (5) 5 The inary entropy function is a monotonically increasing function in the interval [0, ].

5 Letting ϵ 0, we have n R I(X n ; U ) = H(X n ) H(X n U ) = n nh (β), and hence β H ( R ). Therefore, the lower ound on H(X n U ) is given y H(X n U ) n H [ρ H ( R )]. (6) From (3), (4) and (6), we can otain n (R + ϵ) n n H (D ) n + n H [ρ H ( R )] = n H [ρ H ( R )] n H (D ). (7) The rate-distortion region R o (D ) shown in (6) is otained y letting ϵ 0 in (7). Thus, the rate pair (R, R ) satisfies condition (6); i.e., (R, R ) R o (D ). Case. The source X n acts as a helper to reconstruct X n under the required distortion level D. This is the case symmetric with Case. The rate-distortion region shown in (7) can e proved in the same way as in Case. Case 3. Here, we prove that (R, R ) R(D, D ) implies (R, R ) R o (D, D ). To this end, assume that the rate pair (R, R ) achieves distortion D for X and D for X. In the following proof, decoder ψ jointly reconstructs the sources X n and X n under required distortions D and D. The following inequalities are otained: n (R + R + ϵ) H(U ) + H(U ) H(U, U ) = I(X n, X n ; U, U ) (8) = I(X n ; U, U ) + I(X n ; U, U X n ) = I(X n ; U, U ) + I(X n ; X n, U, U ) n I(X ; X ) I(X n ; ˆX n ) + I(X n ; ˆX n ) n I(X ; X ), (9) where (8) holds since U i is a function of X n i, i =,. Similarly, y utilizing Fano s inequality to lower ound I(X n ; ˆX n ) and I(X n ; ˆX n ), we have n (R +R +ϵ) n+n H (ρ) n H (D ) n H (D ). (0) Letting ϵ 0 in the aove inequality, we conclude that (8) holds. That is, (R, R ) R o (D, D ). Through these three cases, it can e concluded that the admissile rate pair (R +ϵ, R +ϵ) R o (D, D ). Furthermore, the monotonicity of the outer ound R o (D, D ) [7] implies that R o (D, D ) R o (D + ϵ, D + ϵ). Since (R, R ) is admissile, we conclude that R(D, D ) R o (D, D ) y letting ϵ 0. Remark : If either R = 0 or R = 0, R o (D, D ) is consistent with the rate-distortion function H (D i ) for the inary source. Remark : R o (D, D ) reduces to the Slepian-Wolf rate region [8] for correlated inary sources if we set D 0 and D 0. The Slepian-Wolf rate region and R o (D, D ) are shown in Fig. 4. It can e found that y allowing nonzero distortion values, the sources can e further compressed compared to the Slepian-Wolf lossless case. Remark 3: If we are interested in reconstructing only one source of the two sources, say X, and there is no rate limit on descriing X n, i.e., R n H(Xn ), then it is equivalent to the Wyner-Ziv compression prolem []. Fig. 5 plots the ratedistortion ound of the Wyner-Ziv source coding [9] and our derived outer ound. In this case, R o (D ) is not tight, since it can e found from Fig. 5 that the rate-distortion region of the Wyner-Ziv prolem lies inside of R o (D ). Remark 4: As it is known that the exact rate-distortion ound of lossy multiterminal source coding prolem lies etween the Berger-Tung inner and outer ounds []. We also derived the rate-distortion region R i (D, D ) ased on the Berger-Tung inner ound [30] after several steps of elementary calculation in information theory, as R i (D, D ) = R i (D ) R i (D ) R i (D, D ) () with R i (D ) = (R, R ) R H (ρ D D ) H (D )}, R i (D ) = (R, R ) R H (ρ D D ) H (D )}, R i (D, D ) = (R, R ) R + R + H (ρ D D ) i= H (D i )}, for every 0 D, D. In Fig. 4, the Berger-Tung inner ound for inary case R i (D, D ) is also presented as a reference to verify how close the ounds R o (D, D ) and R i (D, D ) are. It can e seen from the figure that they are very close to each other for small values of D and D, i.e., the outer ound can e considered as a useful reference in the evaluation of the BER performance, even though there exists a small gap. However, to resolve this gap, further insightful discussions are still needed [5]. Remark 5: The outer ound R o (D, D ) is otained y assuming that the code length n is sufficiently large. However, the penalty on the rate-distortion function is unavoidale if the code length is finite [3], where the rate-distortion function taking into account n is derived for a conventional point-topoint communication system. According to [3], the impact of n is not significant when n is large enough. Therefore, this work focuses on the infinite code length, while the outer ound R o (D, D ) of finite code length is left as a future study. In summary, the rate-distortion function R i (D i ) is given y R (D ) H [ρ H ( R (D ))] H (D ), R (D ) H [ρ H ( R (D ))] H (D ), () R i (D i ) + H (ρ) H (D i ). i= i= IV. HAMMING DISTORTION LOWER BOUND A. Function D = f(d, D ) As stated in Section II, distortion D is a function of distortions D i, i =,. Function f(d, D ) is otained y evaluating the relationship etween the inary CEO and the inary multiterminal source coding prolems in terms of distortions, where the model of the relationship is shown in Fig. 6. The estimate ˆX is otained ased on the decision rule from the outputs of two parallel BSCs with crossover

6 + Fig. 6. The distortion model of the inary CEO prolem. BMTSC represents inary multiterminal source coding. Fig. 7. Block diagram of the encoding/decoding algorithm shown in [0] and [6]. proailities p D, p D and input X. The distortion D largely depends on the decision rule used y the destination. Here we only consider two decision rules. One is the weighted majority decision and the other the optimal decision. ) Weighted majority decision: Distortion D is otained y evaluating the proaility of an error event. Let θ = p D and θ = p D. Without loss of generality, we assume that θ θ. Hence, the error event is composed of two independent events: node makes a wrong decision and node makes correct decision or oth node and node make erroneous decisions. Therefore, the distortion D in this case is approximated y D = θ ( θ ) + θ θ = θ. It can e found that the corner point θ or θ in the rate-distortion region is achieved. Hence, the weighted majority decision rule can e seen as eing equivalent to that derived from the time sharing method. ) Optimal decision: Since the lock length is assumed to e infinite and the code is random, an optimal lower ound on the distortion D is determined y utilizing the rate-distortion function for the inary source [4], as H ( d) = I(X; ˆX) (3) I(X; ˆX, ˆX ) = H(X) + H( ˆX, ˆX ) H(X, ˆX, ˆX ) = + + H (θ θ ) [H(X) + H( ˆX X) + H( ˆX X)] (4) = + H (θ θ ) [ + H (θ ) + H (θ )] = + H (θ θ ) H (θ ) H (θ ), (5) where d is the Hamming distortion measure etween X and ˆX, and the steps are justified as: (3) rate-distortion function for the inary source, (4) ˆX X ˆX forms a Markov chain. Thus, it is ovious from (5) that for 0 d, d [H (θ )+H (θ ) H (θ θ )]. Therefore, the distortion D is the minimum value of d, as H D = H [H (θ ) + H (θ ) H (θ θ )]. (6) It should e emphasized here that the optimal decision acts as a universal lower ound on the HD for specific schemes which assume sequential decoding. However, in the design of practical encoding/decoding algorithms, we do not consider this decision rule. In summary, the distortion level D of the two decision rules descried aove is given as minθ, θ D = }, majority decision, H [H (θ ) + H (θ ) H (θ θ )], optimal. (7) B. Distortion Minimization By sustituting the rate-distortion function () and (7) into the minimization prolem (4), we have min D,D D (8) s.t. H [ρ H ( C(γ ) )] H (D ) C(γ ), r r H [ρ H ( C(γ ) r )] H (D ) C(γ ) r, + H (ρ) H (D ) H (D ) C(γ ) r + C(γ ) r, D i, i =,, D i 0, i =,. The reason of using the derived outer ound, not the Berger- Tung inner ound is that, the outer ound can e easily formulated as a convex optimization. The Berger-Tung inner ound includes term D D in the inary entropy function which cannot e easily handled in the minimization. It is found that distortion D = f(d, D ) is monotonically increasing function on the intervals D i [0, ], i =, for oth the majority decision and optimal decision rules, and the proof is detailed in Appendix B. Furthermore, since the sequential decoding (first reconstructs X n and X n, then makes decision on X n ) is applied, we first minimize the l -norm of a vector

7 0 0 0 Simulation (p =p =0.05) Lower ound (majority) Lower ound (optimal) Simulation (p =p =0.005) Lower ound (majority) Lower ound (optimal) 0 0 0 BER/HD (BEP) 0 BER/HD (BEP) 0 0 3 0 4 0 9 8 7 6 5 4 3 0 per node SNR (db) 0 3 Simulation (p = 0.06, p = 0.03) Lower ound (majority) Lower ound (optimal) Simulation (p = 0.005, p = 0.05) Lower ound (majority) Lower ound (optimal) 0 4 0 9 8 7 6 5 4 3 0 per node SNR (db) Fig. 8. Symmetric P and SNR. BPSK is used for oth nodes. Fig. 9. Asymmetric P and symmetric SNR. BPSK is used for oth nodes. [D, D ] instead of directly minimizing D, as min D,D [D, D ] (9) s.t. H (D ) H (D ) C(γ ) + C(γ ) H (ρ), r r H (D ) C(γ ) r H (D ) C(γ ) r D i, i =,, D i 0, i =,, H [ρ H ( C(γ ) r )], H [ρ H ( C(γ ) r )], to otain the minimal values of D and D, and then map them to D y using function f(d, D ). It is easily found that the prolem (9) is convex since the ojective function is convex and function H ( ) is also convex. Therefore, it can e efficiently solved using convex optimization tools. Assume that the minimum values of D and D otained through the convex optimization are denoted as D and D, respectively. Sustituting D and D into (7), the minimum distortion value D is then otained through minθ D =, θ}, majority decision, H [H (θ) + H (θ) H (θ θ)], optimal, (30) where θ and θ are p D and p D, respectively. It should e emphasized here that the distortion D or D should e set to 0 in the optimization prolem (8) if C(γ ) r or C(γ ) r is larger than or equal to, which is the inary entropy of the source X and X. The reason is that a source can e reconstructed under an aritrary small error proaility if the source coding rate is larger than its entropy even in the case the helper does not exist [4]. V. BER EVALUATION A. Coding and Decoding Algorithm We riefly review the encoding/decoding algorithm [0], [6] which is illustrated in Fig. 7. This algorithm is used to verify the theoretical HD lower ound. As illustrated in Fig. 7, each node encodes its erroneous sequence y using a serially concatenated memory- convolutional code and an accumulator (ACC). The encoder output sequences are then modulated and transmitted to the destination over statistically independent AWGN and lock Rayleigh fading channels, where the channel gain h i is static within each lock ut varies independently lock-y-lock. At the destination, iterative decoding process is carried out etween the decoders of the convolutional code and the ACC, as well as etween the two decoders of the convolutional codes through the LLR updating function f c to modify the extrinsic LLR, according to the error proailities p and p. B. Numerical Results The lower ounds 6 on the HD for different SNR values γ, γ are otained through solving the convex optimization prolem which we presented in Section IV. The results are shown in Figs. 8 for AWGN channels and Fig. for lock Rayleigh fading channels. The common parameters used in the simulations are Frame length: n = 0000 its for AWGN channels and n = 048 its for lock Rayleigh fading channels. The numer of frames: 000 for AWGN channels and 0000 for lock Rayleigh fading channels. Interleavers: random. Encoder C i : half-rate nonrecursive systematic convolutional code with generator polynomial G = [03, 0] 8, where [ ] 8 represents the argument is an octal numer. Modulation: inary phase-shift keying (BPSK) and quadrature phase-shift keying (QPSK) with coherent detection, where channel state information is assumed to e known to the receiver. Natural mapping is used as the mapping rule in QPSK [3, Example 8.]. Doping ratio of ACC: for BPSK and 8 for QPSK. Decoding algorithm for DCC i and ACC : logmaximum a posteriori (MAP). 6 The terminology lower ound used here is due to the HD is calculated ased on the derived outer ound, even though the approximation of the ojective functions is used.

8 0 0 0 Simulation (p = 0.0, p = 0.08) Lower ound (majority) Lower ound (optimal) Simulation (p = p = 0.005) Lower ound (majority) Lower ound (optimal) 0 0 0 Simulation (p = p = 0.0) Lower ound (majority) Lower ound (optimal) Simulation (p = p = 0.005) Lower ound (majority) Lower ound (optimal) BER/HD (BEP) 0 BER/HD (BEP) 0 0 3 0 3 0 4 0 9 8 7 6 5 4 3 0 SNR of channel # (db) Fig. 0. Asymmetric r and r. The coding rates r and r are set at 4 and, respectively. The transmit power of two nodes are the same. BPSK is used for oth nodes. 0 4 0 9 8 7 6 5 4 3 0 SNR of channel # (db) Fig.. Asymmetric r and r. The coding rates r and r are set at and, respectively. The transmit power of two nodes are the same. QPSK is used for node and BPSK for node. The numer of iterations: 30 times. Fig. 8 shows the error proaility lower ounds and the BER versus SNR when p, p and SNRs of the two nodes are set identically; this is referred as the symmetric case. It can e found that, the BER curves otained y simulations and the theoretical lower ounds on the HD (or BEP) exhiit a similar tendency. Furthermore, it is clearly found that the error floor of the BER otained y the simulation and the lower ound on the HD ased on majority decision match exactly. The reason is that if the SNRs of two nodes are large enough, the distortion levels D and D are almost 0, which results in the error floor eing determined completely y the error proailities p and p. A gap clearly appears etween the HD lower ounds using the majority and optimal decision rules. The reason is twofold: ) the optimality of the majority decision can not e guaranteed; ) optimal decision is derived ased on the assumption of the inary rate-distortion function without considering any loss during processing the information. To find a etter decision rule than majority decision rule is left as a future study. However, it is clear that the HD lower ound deriving from the optimal decision cannot e exceeded. The impact of the variation of the error proailities p, p and the coding rates r i are evaluated in AWGN channels. Fig. 9 shows the results for asymmetric p and p ut symmetric SNRs. When the coding rates 7 r and r are set as 4 and, respectively, the BER performance shown in Fig. 0 is otained. We further consider using different modulation schemes for the nodes to achieve different rates of the channel code in Fig., where QPSK is used for node and BPSK for node. Even in these asymmetric cases, the theoretical lower ounds on the HD can still provide us with a useful reference when we evaluate the BER performance of practical systems. Furthermore, the theoretical lower ounds on the HD otained ased on our derived outer ound exhiit similar ehaviors to those of the BER curves found y simulations. 7 We simply transmit the output of ACC without doping to achieve rate 4. No optimized design of the channel code is considered. In oth the symmetric and asymmetric cases, the threshold SNR value at which turo cliff in the BER otained y the simulation is around.5 db larger than that oserved in the theoretical lower ounds in static AWGN channels. In addition, since the lower ounds on the HD plateaus at a certain level even if the power is increased at high SNR regime, increasing the numer of nodes is a proper way to improve performance in the practical deployment. In Fig., the channels etween two nodes and the destination experience independent lock Rayleigh fading. Therefore, the instantaneous SNRs of two nodes are different while the average SNRs of the two channels are the same. The lower ounds on the HD shown in Fig. are calculated as D fading = + + 0 0 D (γ, γ ) Pr(γ ) Pr(γ )dγ dγ, (3) where D (γ, γ ) is the result of (30), otained for static AWGN channels. Pr(γ i ) is the proaility density function of the SNR γ i, which follows the Rayleigh distriution. We use Monte Carlo method to otain the lower ounds on the average HD Dfading instead of theoretically calculating (3). In the Rayleigh fading case, the shape of the BER curves and the lower ounds on the HD are almost the same. VI. CONCLUSION We examined theoretically the lower ound on the HD for the inary CEO prolem, where two independent nodes forward the erroneous versions of a common inary source to the destination over static AWGN and lock Rayleigh fading channels. The inary CEO prolem was first formulated as the inary multiterminal source coding prolem, which is the core part of the inary CEO prolem. The outer ound for ratedistortion region of the inary multiterminal source coding prolem was then derived ased on the converse proof of the ound. The relationship etween the inary CEO prolem and the inary multiterminal source coding prolem in terms of the distortion function has een estalished. According to the lossy source-channel separation theorem, the lower ound on

9 Average BER/HD (BEP) 0 0 0 0 0 3 Simulation (p = p = 0.0) Lower ound (majority) Lower ound (optimal) Simulation (p = 0.008, p = 0.005) Lower ound (majority) Lower ound (optimal) 0 4 0 8 6 4 0 4 6 8 0 4 per node average SNR (db) Fig.. Asymmetric SNR and P. BPSK is used for oth nodes. Channels are assumed to suffer from lock Rayleigh fading. the HD was formulated y minimizing the distortion function suject to the inequalities etween the derived outer ound and the channel capacities. The prolem of otaining the lower ound on the HD was solved in the framework of convex optimization, and the results of HD lower ounds only apply to schemes which use sequential decoding. Through a series of simulations, it has een shown that the BER curves otained with a practical encoding/decoding algorithm is consistent with the result of the theoretical lower ounds on the HD. Even though we only solved the inary CEO prolem having two nodes in this work, it shed light on fully theoretically analyzing the performance of more generic inary CEO prolems such as that with an aritrary numer of nodes. ACKNOWLEDGEMENTS The authors wish to acknowledge the helpful discussions with Prof. Yasutada Oohama. APPENDIX A LOWER BOUNDED H(X n U ) For the completeness of the paper, we riefly provide the core of [6, Corollary 4]. Let (X n, Y n, W ) e a triple of random variales, where X n and Y n take values from a set X n = 0, } n, and W elongs to an aritrary discrete set W. The joint proaility mass function of this triple is given y Pr X,Y,W (x, y, w) = PrX = x, Y = y, W = w} n = Pr(y(t) x(t)) Pr X,W (x, w). (3) t= In other words, Y n can e seen as the output of a BSC channel with crossover proaility p 0 when X n is the input, and W is conditionally independent of Y n given X n. Then Corollary 4: If H(X n W ) n v, then H(Y n W ) n H [p 0 H (v)]. When we sustitute X n, X n and U into the corollary, we have H(X n U ) nh (ρ β) since X n and X n is the input and the output of a BSC with crossover proaility ρ, and as we assumed H(X n U ) H (β). APPENDIX B MONOTONICITY OF DISTORTION D Majority decision. D = minθ, θ }. Since θ i, i =, is the result of the inary convolution on p i and D i, θ i is oviously increasing as D i is increasing, when p i is fixed. Optimal decision. D = H [H (θ )+H (θ ) H (θ θ )]. In this case, D is a composite function of H ( ) and H (θ ) + H (θ ) H (θ θ ). Since the function H ( ) is monotonically increasing, we only need to prove that g(θ, θ ) = H (θ )+H (θ ) H (θ θ ) is also an increasing function of θ and θ. Assume θ is fixed. The partial derivative g(θ,θ ) θ on θ is g(θ, θ ) θ = log θ θ ( θ ) log θ θ θ θ. (33) In order to prove that (33) is nonnegative, we should prove θ θ ( θ θ θ θ ) ( θ ). (34) The aove always holds according to the monotonically increasing property of function log( ). As 0 θ i, i =, and 0 θ θ is assumed, the following inequalities are otained after several steps of elementary calculation θ θ ( θ θ ) ( θ) ( θ θ θ θ ) ( θ). (35) Therefore, it is found that (33) can not take negative values according to (35). Symmetrically, we can assume θ is fixed, and show that the partial derivative g(θ,θ) θ on θ is also nonnegative. Hence, g(θ, θ ) is increasing in the dimension of θ and θ, respectively. Based on the aove two cases, it is concluded that the distortion D is increasing with respect to D and D. REFERENCES [] T. Berger, Z. Zhang, and H. Viswanathan, The CEO prolem, IEEE Trans. Inform. Theory, vol. 4, no. 3, pp. 887 90, May 996. [] H. Viswanathan and T. Berger, The quadratic Gaussian CEO prolem, IEEE Trans. Inform. Theory, vol. 43, no. 5, pp. 549 559, Sept. 997. [3] Y. Oohama, The rate-distortion function for the quadratic Gaussian CEO prolem, IEEE Trans. Inform. Theory, vol. 44, no. 3, pp. 057 070, May 998. [4] J. Chen, X. Zhang, T. Berger, and S. Wicker, An upper ound on the sum-rate distortion function and its corresponding rate allocation schemes for the CEO prolem, IEEE J. Select. Areas Commun., vol., no. 6, pp. 977 987, Aug. 004. [5] S. Draper and G. W. Wornell, Side information aware coding strategies for sensor networks, IEEE J. Select. Areas Commun., vol., no. 6, pp. 966 976, Aug. 004. [6] H. Behroozi and M. Soleymani, Performance of the successive coding strategy in the CEO prolem, in Proc. IEEE Gloal Telecommun. Conf., vol. 3, 8 Nov.- Dec. 005, pp. 347 35. [7], Optimal rate allocation in successively structured Gaussian CEO prolem, IEEE Trans. Wireless Commun., vol. 8, no., pp. 67 63, Fe. 009. [8] R. Viswanathan and P. Varshney, Distriuted detection with multiple sensors I. Fundamentals, Proceedings of the IEEE, vol. 85, no., pp. 54 63, Jan. 997. [9] J. Yuan and W. Yu, Joint source coding, routing and power allocation in wireless sensor networks, IEEE Trans. Commun., vol. 56, no. 6, pp. 886 896, June 008. [0] X. Zhou, X. He, K. Anwar, and T. Matsumoto, GREAT-CEO: large scale distriuted decision making Technique for wireless Chief Executive Officer prolems, IEICE Trans. Commun., vol. E95-B, no., pp. 3654 366, Dec. 0.

[] J. Haghighat, H. Behroozi, and D. Plant, Iterative joint decoding for sensor networks with inary CEO model, in Proc. IEEE Works. on Sign. Proc. Adv. in Wirel. Comms., July 008, pp. 4 45. [] A. Razi, K. Yasami, and A. Aedi, On minimum numer of wireless sensors required for reliale inary source estimation, in Proc. IEEE Wireless Commun. and Networking Conf., Mexico, Mar. 0, pp. 85 857. [3] A. Razi and A. Aedi, Adaptive i-modal decoder for inary source estimation with two oservers, in Proc. Conf. Inform. Sciences Syst. (CISS), Princeton, NJ, USA, Mar. 0, pp. 5. [4], Convergence analysis of iterative decoding for inary CEO prolem, IEEE Trans. Wireless Commun., vol. 3, no. 5, pp. 944 954, May 04. [5] J. Garcia-Frias and Y. Zhao, Near-Shannon/Slepian-Wolf performance for unknown correlated sources over AWGN channels, IEEE Trans. Commun., vol. 53, no. 4, pp. 555 559, Apr. 005. [6] X. He, X. Zhou, K. Anwar, and T. Matsumoto, Estimation of Oservation Error Proaility in Wireless Sensor Networks, IEEE Commun. Lett., vol. 7, no. 6, pp. 073 076, June 03. [7] W. Zhong and J. Garcia-Frias, Comining data fusion with joint sourcechannel coding of correlated sensors, in Proc. IEEE Inform. Theory Workshop, Oct. 004, pp. 35 37. [8] M. Sartipi and F. Fekri, Source and channel coding in wireless sensor networks using LDPC codes, in Proc. IEEE Conf. Sensor Ad Hoc Commun. Networks, Oct. 004, pp. 309 36. [9] X. He, X. Zhou, K. Anwar, and T. Matsumoto, Wireless mesh networks allowing intra-link errors: CEO prolem viewpoint, in Proc. IEEE Int. Symp. Inform. Theory and its Applications, Hawaii, USA, Oct. 0, pp. 6 65. [0] X. Zhou, M. Cheng, X. He, and T. Matsumoto, Exact and approximated outage proaility analyses for decode-and-forward relaying system allowing intra-link errors, IEEE Trans. Wireless Commun., vol. 3, no., pp. 706 707, Dec. 04. [] P.-S. Lu, X. Zhou, K. Anwar, and T. Matsumoto, Joint adaptive network-channel coding for energy-efficient multiple-access relaying, IEEE Trans. Veh. Technol., vol. 63, no. 5, pp. 98 305, June 04. [] A. Gamal and Y. Kim, Network Information Theory. Camridge University Press, 0. [3] J.-J. Xiao and Z.-Q. Luo, Multiterminal source-channel communication over an orthogonal multiple-access channel, IEEE Trans. Inform. Theory, vol. 53, no. 9, pp. 355 364, Sept. 007. [4] T. Cover and J. Thomas, Elements of Information Theory, nd ed. USA: John Wiley & Sons, Inc., 006. [5] Y. Oohama, Gaussian multiterminal source coding, IEEE Trans. Inform. Theory, vol. 43, no. 6, pp. 9 93, Nov. 997. [6] A. D. Wyner and J. Ziv, A theorem on the entropy of certain inary sequence and applications: Part I, IEEE Trans. Inform. Theory, vol. 9, no. 6, pp. 769 77, Nov. 973. [7] S.-Y. Tung, Multiterminal Source Coding, Ph.D. dissertation, Cornell University, 978. [8] D. Slepian and J. Wolf, Noiseless coding of correlated information sources, IEEE Trans. Inform. Theory, vol. 9, no. 4, pp. 47 480, July 973. [9] A. Wyner and J. Ziv, The rate-distortion function for source coding with side information at the decoder, IEEE Trans. Inform. Theory, vol., no., pp. 0, Jan. 976. [30] ICT-69555 RESCUE Deliverale, D..-Assessment on Feasiility, Achievaility, and Limits, Technical Report, May 05. [3] V. Kostina and S. Verdu, Fixed-length lossy compression in the finite locklength regime, IEEE Trans. Inform. Theory, vol. 58, no. 6, pp. 3309 3338, June 0. [3] S. Lin and D. J. Costello, Error Control Coding, nd ed. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 004. 0