Utilization of multi-dimensional sou correlation in multi-dimensional sin check codes. Izhar, Mohd Azri Mohd; Zhou, Xiaobo; Author(s) Tad

Transcription

1 JAIST Reposi Tite Utiization of muti-dimensiona sou correation in muti-dimensiona sin check codes Izhar, Mohd Azri Mohd; Zhou, Xiaobo; Author(s) Tad Citation Teecommunication Systems, 62(4): 73 Issue Date Type Journa Artice Text version author URL Rights This is the author-created version o Mohd Azri Mohd Izhar, Xiaobo Zhou, T Teecommunication Systems, 62(4), 20 The origina pubication is avaiab Description Japan Advanced Institute of Science and

2 Teecommun Syst manuscript No. (wi be inserted by the editor) Utiization of muti-dimensiona source correation in muti-dimensiona singe parity check codes Mohd Azri Mohd Izhar Xiaobo Zhou Tad Matsumoto Received: date / Accepted: date Abstract This paper proposes a joint source-channe coding (JSCC) technique that we utiizes muti-dimensiona (MD) source correation using MD singe parity check codes (MD-SPCCs). The source is assumed to be described by the couping of mutipe first-order binary Markov processes. The knowedge about the source correation is utiized in the channe decoding process where each component decoder utiizes a singe dimension correation of the MD source. To enhance performance and reduce the error foor, a rate-1 recursive systematic convoutiona code (RSCC) is seriay concatenated to the MD-SPCC via a random intereaver. Two decoding techniques are proposed for each component decoder, and the seection of the decoding technique depends on the strength of the source correation, which may further enhance the performance of the proposed JSCC technique. Simuation resuts revea that a significant performance gain can be achieved by expoiting the MD source correation with the proposed JSCC technique compared with the case in which the source correation is not utiized; more significant gains can be achieved with stronger source correation, and with a arger dimensionaity source correation as we. M. A. M. Izhar UTM-MIMOS Center of Exceence, Facuty of Eectrica Engineering, Universiti Teknoogi Maaysia (UTM), Skudai, Johor, Maaysia E-mai: mohdazri.k@utm.my X. Zhou Schoo of Computer Science and Technoogy, Tianjin University, Weijin Road 92, Nankai, Tianjin , China E-mai: xiaobo.zhou@tju.edu.cn T. Matsumoto Japan Advanced Institute of Science and Technoogy (JAIST), 1-1 Asahidai, Nomi, Ishikawa , Japan E-mai: matumoto@jaist.ac.jp Keywords Joint source-channe coding Singe parity check codes Muti-dimensiona source correation Turbo coding 1 Introduction Shannon s separation theorem [18] states that if source entropy is ess than the channe capacity, source and channe codes can be independenty designed; if the designed code is optima both in terms of source and channe coding independenty, arbitrariy sma error probabiity can be achieved. However, the theorem assumes infinite codeword engths and hence, in theory, communication systems designed based on the separation theorem require infinite atency. In practica communication systems, however, the atency requirement does not aow the design of source and channe coding processes based on the separation theorem as the computationa compexity of the systems is finite and the systems are not free of atency constraints. Furthermore, athough existing source coding techniques for specific appications such as voice, image, and/or video communications are quite efficient, redundancy of the source sti remains at the output of the source encoders. The shortcomings described above have motivated the need for joint optimization of source and channe coding (JSCC), particuary in iterative sourcechanne decoding approach [1, 12 14, 16] and aso, in utiizing source redundancy with channe coding [3, 6 8, 10, 11, 17, 19, 22 26]. The JSCC techniques utiizing source redundancy with channe coding may be cassified into two categories, depending on the characteristics of the source: one utiizes non-uniform source distribution knowedge [3, 19, 24, 26]; the other utiizes the time-domain source correation knowedge, for exampe,

3 2 Mohd Azri Mohd Izhar et a. described by hidden Markov mode [6, 7, 17] or Markov process [10, 11, 22, 23, 25] to describe the source behavior. Most of the existing works described above ony assume the source to be one-dimensiona (1D) correated and in [10,11], the source was assumed to be expressed by a 2D couped Markov mode. In this paper, we extend the work [10, 11] to consider sources with mutidimensiona (MD) correation. The source is modeed by a couping of mutipe first-order binary Markov processes [5]. In many wireess appications, such as transmission of mutimedia contents (e.g. 3D video), the pixes/symbos within the source are correated in various dimensions. Sources with higher dimensiona correation can provide additiona information and this information can be utiized to further improve the error correction capabiity of a channe code. To the best of the authors knowedge, the utiization of MD source correation has never been considered in any previous pubications. The main chaenge of this work woud be on the compexity issue. References [10, 11] use turbo bock codes composed of two Bose, Chaudhuri, Hocquenghem (BCH) codes as the component codes, and a modified version of the Bah-Cocke-Jeinek-Raviv (BCJR) agorithm is performed at the receiver to better expoit the source correation. Hence, the arger the parity ength of the BCH codes, the heavier the computationa compexity required to decode the codes. It is therefore not practica to empoy this BCH-based JSCC system for the expoitation of higher dimensiona source correation due to the high computationa compexity required. This paper repaces the parae concatenated BCH code by an MD singe parity check code (MD- SPCC), which is composed of an MD parae concatenation of memory-1 SPCCs, and hence the decoding compexity is significanty reduced compared with the technique presented in [10, 11]. Besides the chaenge in terms of compexity, this work needs to address the negative effect of the correation among the extrinsic og-ikeihood ratios (LLRs) which can cause degradation in the system performance [10]. This paper proposes an aternative decoding technique that based on LLR modification technique in order to avoid the performance degradation. Moreover, a switching agorithm between the modified BCJR agorithm and the LLR modification agorithm is proposed to improve further the performance. The rest of the paper is organized as foows. In Section 2, we provide the information theoretic property of sources generated from couped MD Markov processes. The system design of the utiization of MD source correation in the framework of MD-SPCC is expained p 0 D 1 p 0 D S 0 S 1 1 p 1 D Fig. 1 Two-state Markov process. State S i emits binary output i, i {0, 1} in Section 3. Decoding techniques used to utiize the source correation in the SPC decoding are presented in Section 4. Simuation resuts of the proposed JSCC technique are presented and discussed in Section 5. Finay, concuding remarks are provided in Section 6. 2 Muti-dimensiona Markov sources In this paper, sources with M-dimensiona correation are characterized by the couping of M first-order binary state-emitting Markov processes. Figure 1 describes the source behavior for the -th dimension (D ), where = 1, 2,.., M. S 0 and S 1 are the states that emit binary source information 0 and 1, respectivey. p D 0 and p D 1 are the transition probabiities from S 0 to S 0 and from S 1 to S 1, respectivey. There are M different 1D sequences that can be formed from a source having M- dimensiona source correation because of the source couping. Each of the 1D sequences corresponds to the different dimensions of the source correation. Figure 2 iustrates an exampe of a source with 3D correation couped by three different 1D sequences. The transition probabiities of a source in th dimension, can be represented in matrix form as A D [ = [a D i,i ] = p D 0 1 p D 0 1 p D 1 p D 1 ] p 1 D i, i {0, 1}, (1) where i and i are the previous and current binary vaue emitted by a Markov source, respectivey. The vaue of a current source U depends ony on its immediate previous vaues in a the M dimensions. The previous vaue of U in the -th dimension is denoted as U D, where = 1, 2,..., M. Based on the couped Markov chain (CMC) mode in [5], the transition probabiity of the source U given the previous vaues in a the M dimensions, Pr(U U D1, U D2,..., U DM ) can be represented in a matrix form as B = [b i 1,i 2,...,i M,i ], (2)

4 Utiization MD source correation in MD SPCCs 3 Tabe 1 Entropy rate of 1D, 2D, 3D and 4D source correation with various p vaues Correation in 2 nd dimension: p 0 D 2 D 2 Correation in 1 st dimension: p 0 D 1 D 1 Source U Fig. 2 A source with 3D correation where b i 1,i 2,...,i M,i Correation in 3 rd dimension: p 0 D 3 D 3 = Pr(U = i U D1 = i 1, U D2 = i 2,..., U DM = i M ) = M =1 ad i,i 1 M, i 1, i 2,..., i f=0 =1 ad M, i {0, 1}. (3) i,f The entropy rate of U given U D1, U D2,..., U DM can be derived as H(U U D1, U D2,..., U DM ) = H(U D1, U D2,..., U DM U) + H(U) H(U D1, U D2,..., U DM ), (4) where U D1, U D2,..., and U DM are assumed to be statisticay independent to each other given U. Thus, (4) can be simpified to H(U U D1, U D2,..., U DM ) = H(U D1 U) + H(U D2 U) H(U DM U) + H(U) H(U D1, U D2,..., U DM ) = H D1 (S) + H D2 (S) H DM (S) + H(U) H(U D1, U D2,..., U DM ) M = H D (S) + H(U) H(U D1, U D2,..., U DM ), (5) =1 where the vaues of H(U) and H(U D1, U D2,..., U DM ) can be cacuated empiricay by measurements. H D (S) is the 1D entropy rate of the Markov source in the -th dimension which can be cacuated by [4]: H D (S) = i,i {0,1} µ D i ad i,i og 2 a D i,i, (6) where S is the stochastic process of source U and µ D i is the stationary state distribution. In the case of symmetric Markov sources where p D 0 = p D 1 = p D, then µ D i p H (bit) 1D 2D 3D 4D = 0.5; If the Markov source is symmetric in a M dimensions, then the entropy rate H(U) = 1 bit. The entropy rates for the sources with 1D, 2D, 3D and 4D source correation are summarized in Tabe 1. For simpicity we assume the Markov sources are symmetric in a dimensions and the transition probabiities in a dimensions have identica vaue, e.g. for 4D source correation case, p D1 = p D2 = p D3 = p D4 = p. It is found in Tabe 1 the entropy rate becomes ower as the dimensionaity and the strength of the source correation increases. The owest entropy rate is 0.23 for sources with 4D source correation and p = 0.9, whereas the highest entropy rate is 0.88, in the case of 1D source with p = 0.7. The difference in entropy rate resuted by the increase in the dimensionaity, which acquired by observing the source from mutipe dimensions, however, becomes smaer as the tota dimensionaity becomes arger. Hence, it can be expected that the reative difference in the entropy rates becomes even smaer for sources with 5D and arger dimensionaity. 3 System mode 3.1 Transmitter In this work, we extend the system mode in [11] by repacing the turbo BCH code with the MD SPCC to avoid the compexity exposion in the decoding process, and moreover, mutipe (not necessary 2) component codes are considered. The proposed JSCC system utiizing M-dimensiona source correation by an M- dimensiona SPCC at the transmitter side is iustrated in Fig. 3. It shoud be emphasized that the dimensionaity of the SPCC shoud not necessariy be equa to the number of dimensions of the source correation. More specificay, the proposed system can work we with any dimensiona SPCC as ong as the number of dimensions of the SPCC is not ess than M. Throughout this section, the SPCC dimensionaity is assumed to be equa to M. Sources with M-dimensiona correation are considered. The ength of the -th dimension of the source in bits is denoted as L, with = 1, 2,.., M. It is aso assumed that the receiver knows the transition proba-

5 4 Mohd Azri Mohd Izhar et a. u S Transition Probabiities for Sources with M-Dimensiona Source Correation: p 0 D 1 D 1, p 0 D 2 D 2,, p 0 D M D M M-Dimensiona SPC Encoder u u 1 C 1 v 1 Π 2 u 2 C 2 v 2 M U X w Π in C in c BPSK Moduator x Π M u M C M v M Fig. 3 Bock diagram of the proposed JSCC system utiizing M-dimensiona source correation using an M-dimensiona SPCC at the transmitter side. biities, p D 0 and p D 1 for a the dimensions of the source. The M-dimensiona source is transformed into a 1D sequence u before being fed to the channe encoders for which the frame ength L T of u is given as L T = M L. (7) =1 In our proposed system, there are two channe codes empoyed. These two codes are seriay concatenated via a random intereaver, Π in separating the two codes. The outer channe code is an M-dimensiona SPCC, consisting of M SPC component codes, C 1, C 2,..., C M arranged in parae structure. SPCC is one of the simpest codes, and it has very imited error correction capabiity. However, by combining mutipe SPCCs, powerfu error correction capabiity can be achieved [15, 20]. Every SPC component code, C, adds a singe parity check bit for every K ength information bits; hence, the codeword ength for C is N = K + 1. The SPCC with this set of parameters is denoted as SPC(N,K ) code. Π 2,Π 3,...,Π M are bock intereavers, which are used to re-arrange the ong 1D source sequence u to a sequence foowing each dimension s 1D correation property of the source correation. For instance, u 2 foows the sequence of the source correation in D 2, u 3 in D 3 and so on. The ony exception is u 1, which has the same sequence as u since it foows the sequence of the source correation in D 1. Each u sequence is fed to the corresponding SPC encoder C to generate parity bits. The SPCC-coded 1D sequences having corresponding parity bits, v 1, v 2,..., and v M, are then mutipexed with the origina source sequence u before intereaved by Π in. The intereaved sequence is then encoded by an inner code, C in, which is a rate-1 recursive systematic convoutiona code (RSCC). We found that adding C in to the proposed system heps eiminating the high error foor probem inherenty invoved in MD-SPCC without sacrificing the bandwidth efficiency. Moreover, it enhances the performance of the proposed system especiay for sources with strong correation. The encoded sequence from C in, c, is then moduated using binary-phase shift keying (BPSK) moduator before being transmitted over an additive white Gaussian noise (AWGN) channe. The overa code rate of the proposed system is R c = M =1 1 K. (8) 3.2 Receiver At the receiver, the received sequence r is demoduated to produce a channe output sequence y as depicted in Fig. 4. Iterative decoding process foowing the turbo principe is then invoked between the inner

6 Utiization MD source correation in MD SPCCs 5 M-Dimensiona SPC Decoder M U X L 1 e v 1 L 2 e v 2 L M e v M + L e M Π M 1 (u M ) L 1 e u 1 L 2 e Π 1 2 (u 2 ) p 0 D 1 D 1 r BPSK Demod. y Π in C in 1 L a in Π in (w) Standard BCJR Ag. L e in Π in (w) Π in 1 Adaptive Decoding Technique: (1) Modified BCJR Agorithm or (2) Modified LLR Technique D E M U X L a 1,in u 1 L a 1,in v 1 L a 2,in u 2 Π 2 L a 2,in v 2 Π M L a M,in u M C 1 1 C 2 1 C M 1 L 1 a u 1 L 1 e u L 1 1 e v 1 D p 2 D 0, p 2 1 L 2 a u 2 L 2 e u L 2 2 e v 2 D p M D 0, p M 1 L M a u M L e M u M L e M v M L a M,in v M L M app u M Π M 1 Sign u Fig. 4 Bock diagram of the proposed JSCC system utiizing M-dimensiona source correation using an M-dimensiona SPCC at the receiver side. decoder, C 1 in and the M SPC component decoders, C 1, = 1, 2,..., M. There are two inputs to C 1 in, one is the input from the channe observation and the other input is the a priori LLR L in a fed back from the M SPC component decoders. During the first iteration, the a priori LLR input L in a has zero vaue and C 1 in performs decoding ony with the input from the channe using the standard BCJR agorithm [2] to produce the extrinsic LLR output, denoted as L in e. Each SPC component decoder has two inputs from C 1 in, the a priori LLR corresponding to its information bits, and another a priori LLR corresponding to its parity bits. In addition, there is aso a priori LLR input from the other SPC component decoders cacuated as L a(u ) = M j=1,j L j e(u ), (9) for SPC decoder C 1. The source memory structure (given in transition probabiities) is assumed to be known to the corresponding SPC component decoders and the knowedge of the memory structure is utiized in the channe decoding process. In order for the SPC decoders to utiize the source memory structure, the SPC component decoders adopt either modified version of the BCJR agorithm [11, 22], to be expained in Section 4.1, or the LLR modification technique [25] together with simpified maximum a posteriori (MAP) agorithm [9], to be presented in Section 4.2. The modified BCJR agorithm can achieve better performance than the LLR modification technique, however, in our previous research work [10], it was found out that empoying modified BCJR agorithm in mutipe component decoders may resut in performance degradation due to the correation between the extrinsic LLRs, especiay for sources having strong correation. It shoud be noticed that the LLR correation is caused by the insufficient randomness of the intereaver, which is different from the source correation. This performance degradation can be avoided by using the second decoding technique that based on LLR modification technique. In order to fuy expoit the performance advantage of using the modified BCJR agorithm whie avoiding the performance degradation effect, an agorithm for the seection between the two decoding techniques is proposed in Section 4.3. The proposed seection agorithm, however, is an empirica technique, and optima scheduing of decoder activation with the specified agorithms is sti eft as a future study.

7 6 Mohd Azri Mohd Izhar et a. When the source correation is not utiized in the decoding of the component codes, simpified MAP agorithm is used instead of the standard BCJR agorithm because it can offer amost equivaent performance but require significanty ower decoding compexity than the standard BCJR agorithm. After the decoding process for a M SPC component decoders are competed, the extrinsic LLRs output from C1 1 to C 1 M corresponding to the bits in the information sequence u are summed up and then, mutipexed with the extrinsic LLRs corresponding to the parity bits, v 1, v 2,..., v M. The reconstructed sequence obtained by combining the information and parity parts is then intereaved using Π in before they are fed back to C 1 in. The LLR exchange is repeated for a number of iterations and after the fina iteration, the a posteriori LLRs from the ast component decoder, L M app, are de-intereaved by Π 1 M and hard-decision is then made to obtain the estimated information bits sequence û. 4 Decoding techniques 4.1 Modified BCJR agorithm In [11,22], the standard BCJR agorithm [2] is modified in order to take into account the tempora correation of the source during the decoding process of a convoutiona code and BCH code, respectivey. The modified BCJR agorithm [11, 22] can be empoyed for the decoding of an SPCC. Since SPCC has two states in the treis diagram, there are four states when it is combined with a two-state Markov source in the treis diagram. However, this is not the case when a BCH code is used because the number of states in the treis diagram is arger than two and it depends on the parity ength of the code. 4.2 LLR modification We must use a bock intereaver to preserve the source correation property; however, this contradicts the turbo principe. In fact, we have observed the cases where performance is degraded by iteration, especiay when the correation is arge. In order to avoid this probem, an aternative decoding technique is suggested based on the LLR modification technique [25]. It updates the vaue of the a priori LLR corresponding to the information sequence by considering the source correation property before it is fed to the respective SPC decoder. The a priori LLR L a(u (t)) to be fed to C 1 at a time index t can be updated as [25] L a,mod(u (t)) = (1 α)l a(u (t)) + α [ ] (1 p D 0 n )P (u (t 1) = 0) + p D 1 P (u (t 1) = 1) p D 0 P (u (t 1) = 0) + (1 p D 1 )P (u (t 1) = 1) (10) where α is a constant that specifies the weighting factor for the correction term (the second term of (10)) over the origina term (the first term of (10)) and the vaue range of α is from 0 to 1. The optima vaue of α can be determined empiricay. For t 2, P (u (t 1) = 1) and P (u (t 1) = 0) can be determined as el a (u (t 1)) P (u (t 1) = 1) = 1 + e, (11) L a (u (t 1)) and 1 P (u (t 1) = 0) = 1 + e, (12) L a (u (t 1)) respectivey, and for t = 1, P (u (0) = 0) = 1 P (u (0) = 1) = µ D 0. (13) Simiary, the a priori LLR, L,in a (u (t)) is aso modified in the same way as L a(u (t)) was modified. In this case, we do not need to use the BCJR agorithm based on the memory-extended treis diagram agorithm because the source statistics have been utiized when modifying the a priori LLRs. Simpified MAP agorithm [9] is empoyed for the decoding of SPCCs and the modified a priori LLRs, according to (10) are input to the simpified MAP agorithm. The extrinsic LLR for every K information bits sequence and the corresponding parity bit output from decoder C 1 can be cacuated as [9] L e(u (q)) = 2 arctanh ( K L,in a,mod tanh (u (j)) + L a,mod (u ) (j)) 2 j=1,j q ( )) L,in a (v ) tanh, (14) 2 and L e(v ) = 2 arctanh ( K L,in a,mod tanh (u (j)) + L a,mod (u ) (j)), 2 j=1 (15) respectivey, where q = 1, 2,..., K is the index indicating the bit position in the K ength information bits sequence u and v is the corresponding parity bit.

8 Utiization MD source correation in MD SPCCs Seection of decoding techniques Ideay, the use of the modified BCJR agorithm shoud resut in optima performance, however, when the source has strong correation, performance improvement from the utiization of higher dimensiona source correation cannot be achieved if the modified BCJR agorithm is used by a the SPC component decoders. This probem is primariy due to the correation among the extrinsic LLRs resuted from using the modified BCJR agorithm for more than one component decoder. A negative effect of the correated extrinsic LLRs becomes more significant when utiizing strong source correation. It is we known that highy correated extrinsic LLRs are not desirabe for turbo decoding. Therefore, the LLR modification technique is better suited for reducing the negative impact of the correated extrinsic LLRs and thus, heps to enhance the performance. In this section, we propose a method to seect the decoding technique for each SPC component decoder, so that performance enhancement can be we achieved. However, as stated before, it is sti an empirica method and no mathematica proof of the optimaity is provided. The genera idea is to fuy expoit the performance advantage of using the modified BCJR agorithm whie no performance degradation is imposed. The seection agorithm is described beow: Step-1: Step-2: Step-3: Step-4: Identify and sort C 1 corresponding to the strongest to the weakest correation of u. C 1 that corresponds to the strongest, the second, and so on unti the Mth strongest source correation are denoted as C 1 S:1, C 1 S:2,..., and C 1 S:M, respectivey Empoy the modified BCJR agorithm to C 1 S:1 Check if the average vaue of the transition probabiities for 2D source correation p 2D exceeds or not exceeds a pre-determined threshod vaue p 2D T, then modified BCJR agorithm is empoyed by. If p2d p 2D T C 1 S:2. On the other hand, if p2d > p 2D T, then the LLR modification technique is empoyed at C 1 S:2. The threshod p2d T is determined empiricay beforehand by simuations Repeat Step-3 for other component decoders, C 1 S:3 unti C 1 S:M for 3D to MD source correation, respectivey Once the LLR modification technique is seected at a particuar dimension of source correation, the LLR modification technique is commony used for the rest of the decoders, and the modified BCJR agorithm is no onger used for higher dimension sources. As stated be- Tabe 2 Simuation parameters Parameter Outer Code SPCC Type Inner Code Frame Length Π 2, Π 3, Π 4 Π in Vaue Code Rate, R c 0.64 No. of Iterations 25 α 0.2 4D MD-SPCC SPC(8,7) Code Rate-1 RSC(3, 2) 8 Code bits Bk. Int bits Random Int. 965,888 bits ength fore, this scheduing technique is rather empirica and optimaity is not guaranteed. Nevertheess, as shown in the next section, it can achieve exceent performance. 5 Numerica resuts 5.1 BER performance evauation A series of simuations was carried out to evauate the performance of the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation using a 4D MD- SPCC as the outer code and a rate-1 RSC(3, 2) 8 code as the inner code. The parameters that we used in the simuations are summarized in Tabe 2. The threshod vaues, p 2D T, p3d T and p4d T for determining the decoding technique of C2 1, C 1 3 and C4 1, respectivey, were determined empiricay by the preiminary simuations; They were found to be p 2D T = 1, p3d T = 0.72 and p4d T = 0.67, respectivey, where as noted before, the sources are symmetric having an identica transition probabiity vaue in a source correation dimensions. Athough the same outer code is used for 1D, 2D, 3D and 4D source correation, the decoding techniques are different for different source dimensions. For 1D source correation, ony C1 1 uses modified BCJR agorithm, whereas the other decoders use simpified MAP agorithm, which means that the utiization of the source correation is ony performed at C1 1. For 2D source correation, C1 1 empoys the modified BCJR agorithm, C2 1 empoys the modified BCJR agorithm or LLR modification technique (depending on the strength of the source correation) and the other two decoders empoy simpified MAP agorithm. For 3D source correation, the modified BCJR agorithm is empoyed for C1 1, the modified BCJR agorithm or LLR modification technique is empoyed for C2 1 and C3 1, and the simpified MAP agorithm is empoyed at C4 1. Finay, for

9 8 Mohd Azri Mohd Izhar et a Non JSCC 1D 2D 3D 4D D 2D 3D 4D BER 10 4 BER E b /N 0 (db) Fig. 5 Comparison of BER performance between the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation over the conventiona non-jscc system for identica p = 0.7 after 25 iterations 4D source correation, the modified BCJR agorithm is empoyed for C1 1 and the modified BCJR agorithm or LLR modification technique is empoyed for C2 1, C 1 3 and C 1 4. As stated before, p D1 0 = p D1 1 = p D2 0 = p D2 1 = p D3 0 = p D3 1 = p D4 0 = p D4 1 = p is aso assumed in the simuations. The bit error rate (BER) performance of the proposed JSCC system utiizing 1D, 2D, 3D and 4D source correation for sources with p = 0.7 are shown in Fig. 5. Significant gain achieved by using the proposed JSCC system over the conventiona system that does not utiize source correation (where a component decoders of the conventiona MD-SPCC empoy simpified MAP agorithm). The conventiona system achieves BER 10 5 at E b /N 0 = 2.15 db whie for the JSCC system utiizing 1D source correation, 1.47 db, yieding 0.68 db gain. The gain becomes arger as higher dimensiona source correation is utiized and the gain achieved by utiizing 4D source correation is 1.34 db. It is worth emphasizing here that in this case, since p < p 2D T, the modified BCJR agorithm is empoyed for C2 1 to efficienty utiize the 2D source correation. Simiary, since p < p 3D T, the modified BCJR agorithm is empoyed for C3 1 to efficienty utiize the 3D source correation, and the modified LLR technique is empoyed for C4 1 to efficienty utiize the 4D source correation because p > p 4D T. Simiar observations are found when evauating the BER performance for sources with p = 0.8 and 0.9 as shown in Fig. 6 and Fig. 7, respectivey. In the same way as in the case of p = 0.7, better performance can be achieved by utiizing higher dimensiona source correation, and moreover, utiizing sources with arger correation aso improves the performance. The decoding E b /N 0 (db) Fig. 6 BER performance of the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation for identica p = 0.8 after 25 iterations BER D 2D 3D 4D E b /N 0 (db) Fig. 7 BER performance of the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation for identica p = 0.9 after 25 iterations scheduing was determined in the same way as in the case p = 0.7, according to the technique presented in Section 4. The benefit of using the seection decoding technique is shown Fig. 8 as the system that based on modified BCJR agorithm performs worse than the proposed system especiay when utiizing higher dimensiona source correation. The performance gains with the proposed JSCC system over the conventiona system are summarized in Tabe 3. Among those systems evauated, the argest improvement that can be observed over the conventiona system is by utiizing 4D source correation for sources with p = 0.9, and the gain is 4.86 db, whie the utiization of the 1D source correation with p = 0.7 achieves the smaest gain of 0.68 db. Thus, it is reasonabe to expect that utiizing 5D or even arge source cor-

10 Utiization MD source correation in MD SPCCs 9 Tabe 3 Gain in performance of the proposed JSCC system over the conventiona system p E b /N 0 at BER 10 5 (db) Gain (db) 1D 2D 3D 4D 1D 2D 3D 4D Tabe 4 Gap to theoretica imit of the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation for p = 0.7, 0.8 and 0.9 p Theoretica Limit (db) Gap to Limit (db) 1D 2D 3D 4D 1D 2D 3D 4D BER D (Proposed) 3D (Proposed) D (A Mod. BCJR) 4D (Proposed) 4D (A Mod. BCJR) E b /N 0 (db) Fig. 8 Comparison in BER performance between the proposed JSCC system (with seection decoding technique) and the JSCC system based on modification BCJR agorithm (without seection decoding technique) to utiize MD source correation with identica p = 0.9 after 25 iterations reation using the proposed JSCC system can achieve further gain. The theoretica imits were cacuated using channe constraint capacity (CCC) [21] in order to evauate the gap in E b /N 0 between the performance of the proposed JSCC system at BER eve 10 5 and the threshod E b /N 0 cacuated from the theoretica CCC. In this case, where sources with memory are considered, the capacity, C is subjected to the constraints of H R c C where R c is the code rate of the system and H is the entropy rate. By using the vaues of the entropy rate shown in Tabe 1, the theoretica imit was cacuated for the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation with p as a parameter. The gap to the theoretica imit was then evauated based on the curves shown in Fig. 5, Fig. 6 and Fig. 7. Tabe 4 tabuates the theoretica imit and the gap to the imit of the proposed JSCC system utiizing 1D, 2D, 3D, and 4D source correation for p = 0.7, 0.8 and 0.9. The theoretica imit for the conventiona system with R c = 0.64 and H = 1 is at 0.89 db and hence, the gap to the theoretica imit with the conventiona system is 1.26 db. Utiizing arger dimensionaity and stronger source correation using the proposed JSCC system resuts in ower theoretica imit, however, the gap to the imit becomes arger. For exampe, utiizing 1D source correation with p = 0.7 achieves the smaest gain compared to the conventiona system, however the gap to the imit is 1.54 db which is aso the smaest among those systems compared. On the other hand, utiizing 4D source correation with p = 0.9 achieves the argest gain compared to the conventiona system, however, gap to the imit of 4.62 db is aso the argest. Hence, it is reasonabe to predict that the utiization of arger than four correation dimensions and p arger than 0.9 even increase the gap to the theoretica imit. The design of a more sophisticated JSCC system that can achieve cose-imit performance without requiring high computationa compexity for sources having high dimensionaity and strong source correation is eft as future study. 5.2 Computationa compexity evauation In [10, 11], it is shown that significant improvement can be achieved by expoiting 1D and 2D source correation using turbo BCH codes. Simiar to the JSCC system proposed in this paper, the modified BCJR agorithm is empoyed to expoit the source correation during the decoding of a BCH code in [10, 11]. However, unike the SPCC, number of the states in the treis diagram for the BCH code is arger than 2 2 1

11 10 Mohd Azri Mohd Izhar et a. Tabe 5 The fine-tuned code parameters, the corresponding code rate and theoretica imit for the BCH-based and the proposed JSCC systems expoiting 2D source correation for p = 0.7, 0.8 and 0.9 JSCC System p Codes (C 1,C 2,C in ) R c Limit(dB) BCH 0.7 BCH(255, 247),BCH(15, 11),RSC(77, 40) Proposed 0.7 SPC(128, 127),SPC(4, 3),RSC(77, 40) BCH 0.8 BCH(255, 247),BCH(15, 11),RSC(777, 400) Proposed 0.8 SPC(128, 127),SPC(5, 4),RSC(177, 100) BCH 0.9 BCH(127, 120),BCH(15, 11),RSC(37, 20) Proposed 0.9 SPC(128, 127),SPC(7, 6),RSC(77, 40) Tabe 6 Comparison of the BCH-based and the proposed JSCC systems expoiting 2D source correation p Code Rate Gap to Limit(dB) No. of Treis States BCH Proposed BCH Proposed BCH Proposed = = = = = = 40 BER p = 0.9 p = 0.8 p = 0.7 BCH Based JSCC p=0.7 Proposed JSCC p=0.7 BCH Based JSCC p=0.8 Proposed JSCC p=0.8 BCH Based JSCC p=0.9 Proposed JSCC p= E b /N 0 (db) Fig. 9 BER performance of the fine-tuned BCH-based and the proposed JSCC systems expoiting 2D source correation with p=0.7, 0.8 and 0.9 after 40 iterations (mutipy by 2 to incude the Markov states) depending on the ength of the parity-check part of the BCH code. For instance, there are states in the treis for BCH(N = 127,K = 120). The compexity is determined by the number of treis states invoved during the decoding process and therefore, it is not desirabe to empoy a code requiring arge number of the treis states. The code parameters for the BCH-based JSCC system are fine-tuned to yied cose-imit performance as presented in [11]. Simiary to the expoitation of 2D source correation, the code parameters for the proposed SPCC-based JSCC system (with 2D MD-SPCC used as the outer code) were fine-tuned as we. For the codes with the fine-tuned parameters, their corresponding code rates and the theoretica imit for both the proposed and BCH-based JSCC schemes are presented in Tabe 5. The BER performance curves with fine-tuning of parameters for the BCH-based [11] and the proposed JSCC systems expoiting 2D source correation are compared in Figure 9. To achieve BER 10 5, the BCHbased JSCC system requires E b /N 0 vaue of 0.11 db, db and db for p=0.7, 0.8 and 0.9, respectivey. The proposed JSCC system on the other hand, the required E b /N 0 vaues needed to achieve BER 10 5 are 0.55 db, db and db for p=0.7, 0.8 and 0.9, respectivey. It shoud be noticed that the BCHbased JSCC system achieves sighty better performance than the proposed JSCC system in terms of turbo ciff and hence, smaer gap to the theoretica imit, as shown in Figure 9 and Tabe 6. However, the BCH-based JSCC system requires arger number of treis states invoved in the decoding process compared with the proposed JSCC system, as indicated in Tabe 6. Since the same iteration rounds were performed in both systems, it can be concuded that the proposed JSCC system, decoded with much smaer number of treis states requires ower computationa compexity, resuting in much ower atency. 6 Concusions In this paper, the JSCC technique proposed in [11] for the utiization of 2D source correation has been extended to better utiize MD source correation using MD-SPCCs. The source is characterized by the couping of mutipe first-order Markov processes. The source statistics, dimension-by-dimension of the source correation are utiized during SPC decoding process by using either the modified BCJR agorithm or the LLR modification technique. An empirica but yet efficient method has been proposed to seect the suitabe decoding technique for each component decoder of the MD- SPCC. It has been shown through simuations that the

12 Utiization MD source correation in MD SPCCs 11 utiization of higher dimensiona and stronger source correation with the proposed JSCC technique achieves arger performance gain over the conventiona system that does not utiize source correation. The proposed JSCC technique can be appied in a number of emerging wireess mutimedia appications, especiay for the transmission of MD mutimedia contents (e.g. 3D video). Acknowedgements This research has been supported in part by Ministry of Education (MOE) Maaysia and Research Management Center (RMC), Universiti Teknoogi Maaysia under Fundamenta Research Grant Scheme (FRGS) No. R.K F595, and in part by the Japan Society for the Promotion of Science (JSPS) KIBAN (B) No References 1. Adrat, M., Picard, J.M., Vary, P.: Efficient near-optimum softbit source decoding for sources with inter- and intraframe redundancy. In: Proceedings of IEEE Internationa Conference on Acoustics, Speech and Signa Processing, pp Montrea, Quebec, Canada (2004) 2. Bah, L., Cocke, J., Jeinek, F., Raviv, J.: Optima decoding of inear codes for minimizing symbo error rates (corresp.). IEEE Transactions on Information Theory 20(2), (1974) 3. Cabarcas, F., Souza, R., Garcia-Frias, J.: Sourcecontroed turbo coding of non-uniform memoryess sources based on unequa energy aocation. In: Proceedings of Internationa Symposium on Information Theory (ISIT), p Chicago, Iinois, USA (2004) 4. Cover, T.M., Thomas, J.A.: Eements of Information Theory 2nd Edition. John Wiey and Sons, USA (2006) 5. Efeki, A.M.M., Dekking, F.M.: A markov chain mode for subsurface characterization: Theory and appications. Mathematica Geoogy 33(5), (2001) 6. Garcia-Frias, J., Viasenor, J.D.: Combining hidden markov source modes and parae concatenated codes. IEEE Communications Letters 1(4), (1997) 7. Garcia-Frias, J., Viasenor, J.D.: Joint turbo decoding and estimation of hidden markov sources. IEEE Journa on Seected Areas in Communications 19(9), (2001) 8. Hagenauer, J.: Source-controed channe decoding. IEEE Transactions on Communications 43(9), (1995) 9. Hagenauer, J., Offer, E., Papke, L.: Iterative decoding of binary bock and convoutiona codes. IEEE Transactions on Information Theory 42(2), (1996) 10. Izhar, M.A.M., Fisa, N., Zhou, X., Anwar, K., Matsumoto, T.: Utiization of 2-d markov source correation using bock turbo codes. In: Proceedings of 7th Internationa Symposium on Turbo Codes and Iterative Information Processing. Gothenburg, Sweden (2012) 11. Izhar, M.A.M., Fisa, N., Zhou, X., Anwar, K., Matsumoto, T.: Expoitation of 2d binary source correation using turbo bock codes with fine-tuning. EURASIP Journa on Wireess Communications and Networking 2013(89) (2013) 12. Kiewer, J., Görtz, N.: Soft-input source decoding for robust transmission of compressed images using twodimensiona optima estimation. In: Proceedings of IEEE Internationa Conference on Acoustics, Speech and Signa Processing, pp Sat Lake City, Utah, USA (2001) 13. Kiewer, J., Görtz, N.: Two-dimensiona soft-input source decoding for robust transmission of compressed images. Eectronic Letters 41(4), (2005) 14. Kiewer, J., Görtz, N., Mertins, A.: Iterative sourcechanne decoding with markov random fied source modes. IEEE Transactions on Signa Processing 54(10), (2006) 15. Rankin, D.M., Guiver, T.A.: Singe parity check product codes. IEEE Transactions on Communications 49(8), (2001) 16. Schmaen, L.: Iterative source-channe decoding: Design and optimization for heterogeneous networks. Ph.d. thesis, RWTH Aachen University (2001) 17. Ser, J.D., Crespo, P.M., Esnaoa, I., Garcia-Frias, J.: Joint source-channe coding of sources with memory using turbo codes and the burrows-wheeer transform. IEEE Transactions on Communications 58(7), (2010) 18. Shannon, C.E.: A mathematica theory of communication. Be System Technica Journa 27(3) (1948) 19. Souza, R., Shamir, G.I., Garcia-Frias, J., Xie, K.: Nonsystematic turbo coding with unequa energy aocation for nonuniform memoryess sources. In: Proceedings of Internationa Symposium on Information Theory (ISIT), pp Adeaide, Austraia (2005) 20. Tee, J.S.K., Tayor, D.P., Martin, P.A.: Mutipe seria and parae concatenated singe parity-check codes. IEEE Transactions on Communications 51(10), (2003) 21. Ungerboeck, G.: Channe coding with mutieve/phase signaing. IEEE Transactions on Information Theory IT-28(1), (1982) 22. Zhou, X., Anwar, K., Matsumoto, T.: Seriay concatenated joint source-channe coding for binary markov sources. In: 6th Internationa ICST Conference on Communications and Networking (CHINACOM). Harbin, China (2011) 23. Zhou, X., Anwar, K., Matsumoto, T.: Exit chart based joint source-channe coding for binary markov sources. In: Proceedings of IEEE Vehicuar Technoogy Conference (VTC Fa), pp Quebec City, Canada (2012) 24. Zhu, G., Aajaji, F.: Turbo codes for nonuniform memoryess sources over noisy channes. IEEE Communications Letters 6(2), (2002) 25. Zhu, G., Aajaji, F.: Joint source-channe turbo coding for binary markov sources. IEEE Transactions on Wireess Communications 5(5), (2006) 26. Zhu, G., Aajaji, F., Bajcsy, J., Mitran, P.: Transmission of nonuniform memoryess sources via nonsystematic turbo codes. IEEE Transactions on Communications 52(5), 855 (2004)