Random Sparse Linear Systems Observed Via Arbitrary Channels: A Decoupling Principle

Transcription

1 Random Spare Linear Sytem Oberved Via Arbitrary Channel: A Decoupling Principle Dongning Guo Department of Electrical Engineering & Computer Science Northwetern Univerity Evanton, IL 6008, USA. Chih-Chun Wang Center for Wirele Sytem and Application CWSA School of Electrical & Computer Engineering Purdue Univerity, Wet Lafayette, IN 47907, USA. Abtract Thi paper tudie the problem of etimating the vector input to a pare linear tranformation baed on the obervation of the output vector through a bank of arbitrary independent channel. The linear tranformation i drawn randomly from an enemble with mild regularity condition. The central reult i a decoupling principle in the large-ytem limit. That i, the optimal etimation of each individual ymbol in the input vector i aymptotically equivalent to etimating the ame ymbol through a calar additive Gauian channel, where the aggregate effect of the interfering ymbol i tantamount to a degradation in the ignal-to-noie ratio. The degradation i determined from a recurive formula related to the core function of the conditional probability ditribution of the noiy channel. A ufficient condition i provided for belief propagation BP to aymptotically produce the a poteriori probability ditribution of each input ymbol given the output. Thi paper etend the author previou decoupling reult for Gauian channel to arbitrary channel, which wa baed on an earlier work of Montanari and Te. Moreover, a rigorou jutification i provided for the generalization of ome reult obtained via tatical phyic method. I. INTRODUCTION Conider the etimation of a vector input ignal X which travere a linear ytem S in the Euclidean pace and i then oberved through a noiy channel: SX Y. Thi imple model i widely ued in communication, control, and ignal proceing. Mot work conider the vector Gauian noie channel: Y = SX + N. A ueful a the Gauian model i, it doe not apply to many intereting application. One eample i the Poion multiple-acce channel, in which Y ha Poion ditribution conditioned on SX. Auming that SX Y conit of a bank of arbitrary independent channel, thi paper tudie the optimal a poteriori etimation a well a efficient belief propagation BP etimator. Early work along thi direction conider the vector Gauian channel Y = SX + N with linear minimum meanquare error MMSE etimation. In cae of a large, randomly generated linear tranformation S, the mean-quared error can be computed uing random matri theory. For the ame channel, optimal nonlinear etimation of dicrete input i Thi reearch wa in part upported by the National Science Foundation under Grant CCF and DARPA under Grant W9NF Thi paper adopt the following notational convention unle noted otherwie. Determinitic and random variable are denoted by lowercae and uppercae letter repectively, and calar, vector and matrice are ditinguihed uing normal, bold and underlined bold font repectively. a dicrete optimization problem and random matri theory i not applicable. A breakthrough in performance analyi wa made by Tanaka uing the replica method [], a non-rigorou technique commonly adopted in tatitical phyic. Guo and Verdú ubequently generalized Tanaka reult to arbitrary input and a family of detector []. It i claimed that for vector Gauian channel, the optimal etimate for each individual ymbol i of identical quality a the etimate of the ame ymbol through a calar Gauian channel independent of all other ymbol []. Thi reult i referred to a the decoupling principle for vector Gauian channel. The correponding ignal-to-noie ratio SNR of each equivalent calar Gauian channel, or rather, the degradation in SNR for each ymbol termed the efficiency, i determined by a fied-point equation. Unfortunately, the replica method relie on intractable aumption and the reult in [] and [] are ubject to doubt. In recent year, etimator baed on BP and it approimation have received great attention [3] [5]. By comparing BP and maimum a poteriori MAP detection, Montanari and Te [6] jutified Tanaka reult for the firt time in the pecial cae of pare preading matri S with a relatively mall ytem load. Their reult upport the conjecture from everal work that the large-ytem performance of the MAP detector i determined by the olution of the fied-point equation for efficiency in [], []. Previouly, under eentially the ame etting of large pare linear ytem and vector Gauian channel, the author [7], [8] have generalized the reult of [6] to arbitrary a priori ditribution of the input X and the linear tranformation S uing the generalized denity evolution [9], [0]. Both the decoupling principle and the fied-point characterization in [] are rigorouly proved, and a andwiching argument i ued to obtain the tronget, poterior-probability-baed characterization of the etimation of individual ymbol X k and linearly tranformed ymbol SX l. Thi paper generalize the above vector Gauian channel reult to arbitrary channel characterized by their conditional probabilitie. Remarkably, the decoupling into calar Gauian channel till hold. In particular, the etimation of any individual ymbol uing BP i equivalent to etimating the ame ymbol through a calar additive Gauian noie channel even though the vector channel SX Y i neither additive nor Gauian. The SNR degradation in the /07/$5.00 c 007 IEEE 946

2 equivalent calar Gauian channel, which can be regarded a the generalized efficiency of the underlying ytem, i determined by the ytem load and the core function of the channel conditional probability ditribution through a double recurion. If the ytem load i ufficiently mall uch that the fied point of the recurive formula i unique, the tronget poterior-probability-equivalency between BP and a poteriori etimation can be etablihed by the andwiching argument. Interetingly, the reult in thi work are conitent with the reult developed uing the replica method in []. A. Linear Sytem II. MODEL AND FORMULATION Let X = [X,..., X K ] denote the input vector to a linear tranformation characterized by a random L K matri S. The output of the tranformation, SX, i oberved through arbitrary parallel independent noiy channel of the ame type: p Y X,S y, = L f y l l where fy w = p Y W y w denote the conditional probability denity function of each calar channel. A familiar pecial cae of the ytem with f being conditionally Gauian i decribed by Y = SX + N where N i a Gauian vector. Indeed, the model can be regarded a a generalization of to arbitrary channel. Conider any realization of S =. Following the convention in graphical modeling of communication ytem, a bipartite factor graph of the ytem i illutrated in Figure, where each pair of ymbol node X k and chip node Y l are connected by an edge if lk 0. The tak of the etimator i to infer the calar value of X k for ome k, given the obervation Y, where the channel characteritic f, the input ditribution P X, and the realization of the matri S = are known to the etimator. We aume that the ymbol X k are independent and identically ditributed i.i.d. and take value in the alphabet χ R, which may be dicrete or continuou. Let P X denote the cumulative ditribution function cdf of X k, which i of zero mean and finite variance. The k-th column of S i denoted by S k = [S k, S k,..., S Lk ] and Λ k i a normalization factor. l= B. Random Enemble and Large-Sytem Limit The enemble of linear tranformation i decribed in the following. Firt, an L K binary incidence matri H = H lk i randomly picked from a certain enemble to be decribed hortly. For all l, k with H lk = 0, et S lk = 0. For all l, k with H lk =, S lk are i.i.d. and equally likely to be ±. The normalization factor for each preading equence k i / Λ k, where Λ k = L l= H lk i the ymbol degree of Here we ue a term originated in CDMA. Y Y Y 3 Y 4 Y L Y L X X X 3 X K X K Fig.. The Forney-tyle factor graph for the pare linear ytem. The quare and the circle correpond to the factorization aociated with the noiy obervation p Y W and the vector repetition k X k of X k repectively. X k. Let Γ l := K k= H lk denote the chip degree of Y l and Γ := L L l= Γ l denote the average chip degree. For the random matri enemble of H, the doubly Poion enemble i ued a an illutrative eample in thi work, for which each entry H lk i i.i.d. Bernoulli with P H lk = = Γ/K. Thi paper conider the large-pare-ytem limit, in which K, L, Γ with K/L β < and Γ = ok /, the lat condition of which enure that the bipartite graph of H ee Figure i free of cycle of length maller than any given number in probability. More detailed decription of the enemble of interet can be found in [8]. It i worth noting that the reult in thi paper can be eaily etended to accommodate more general enemble [], a demontrated in [8], for which ome other regular condition, in addition to the aymptotic hort-cyclefree property, are required, including the chip-emi-regularity and the balanced-ymbol-degree condition. A a final remark, the input ditribution P X and the ytem load β are fied ytem parameter and do not change with repect to K, L, and Γ. III. MAIN RESULTS Let u introduce the canonical calar Gauian channel: Z = gx + N 3 where X P X and N N 0, are independent, and g denote the gain of the channel in SNR. Throughout thi paper, we ue P X Z;g to denote the cdf of the poterior ditribution of the input X given Z, according to the Gauian model 3, which i parameterized by g. Conider firt the problem of etimating an individual ymbol X k given all oberved chip in it upporting tree of depth t, denoted by Y t. Preciely, Y t conit of all chip Y l within ditance t to X k on the factor graph. A key reult in thi paper tate that the poterior of X k given Y t eentially converge to the poterior of the calar Gauian channel, a the ize of the linear ytem increae. Theorem : For every k and where P X i continuou, P Xk Y t,s Y t, S P X Z;η t h Y t 4 in probability in the large-pare-ytem limit, where η t i ome poitive number, and h i ome function uch that, conditioned on X k = a, h Y t N a η t,. 947

3 Theorem tate that the problem of etimating each individual ymbol X k from Y t i.e., uing t iteration of belief propagation i aymptotically equivalent to that of etimating the ame ymbol through a calar Gauian channel with SNR equal to η t. Thu the collective effect of the noie and the interference of other ymbol to the deired ymbol i equivalent to an additive Gauian noie. The reult i ignificant and rather urpriing becaue the pare linear ytem and noiy channel are arbitrary. We relegate dicuion of the function h to Section IV. In the following we decribe the olution to η t, which uniquely characterize the etimation problem. Preciely, η t, t =,,..., are determined by the following recurion: η t+ = E E log f Y βν t W Y 5 ν t = E X E X η t X + N 6 where η 0 = 0, gy u = ugy u for arbitrary bivariate function gy u, and the random variable are defined in the following: X P X and N N 0, are independent; Y and W are jointly ditributed with W N 0, and P Y W y w = f y βν t w. The relationhip between the random variable and parameter ν t, η t i illutrated in Figure. The above decription completely determine the joint ditribution of W, Y, which i parameterized by ν t and hence η t. In particular, ν t i the average variance of X conditioned on it noiy obervation Z through a Gauian channel with SNR equal to η t, where η t i the variance of the conditional mean of a core function of the channel characteritic f. W N 0, X P X β ν t η t fy Y N N 0, Z Fig.. The relationhip between random variable in 5 and 6. Note that ν t i the average variance of the input to the Gauian channel with SNR equal to η t conditioned on it output. Theorem : Suppoe the recurive formula 5 and 6 have a unique fied point η, ν. Then for every k and where P X i continuou, P Xk Y,S Y, S PX Z;η hy 7 in probability in the large-pare-ytem limit, where h i uch that, conditioned on X k = a, hy N a η,. Theorem tate that the problem of etimating each X k given the entire obervation Y, i alo aymptotically equivalent to etimating the ame ymbol through a calar Gauian channel, the SNR of which i equal to η = lim t η t. In Fig. 3. X k Y l Yl... Y l.. X k X k. X kγ Statitical inference over the correponding tree tructure. view of Theorem, it implie that oberving Y t become a good a oberving Y a t, even though the ratio of the dimenion of Y and Y t approache infinity. Remarkably, thi implie that BP i aymptotically a good a the optimal a poteriori etimation. In the pecial cae where f repreent a Gauian channel, i.e., log fy w = y w, 5 become η t+ = E E Y W βν t Y 8 = E Y βνt + βν Y 9 t =. 0 + βν t Together with 6, we find the following recurion = + βe X E X η η t X + N t+ which wa firt obtained in [7], [8], where η t i the multiuer efficiency achieved by BP after t iteration. Note that Boutro and Caire obtained a imilar formula in the contet of iterative decoding of coded CDMA uing an empirically inpired Gauian approimation [3]. The fied-point equation correponding to wa originally obtained in []. A. Notation IV. PROOF Conider the inference tree illutrated in Figure 3, which i a ubgraph of the factor graph depicted in Figure. The node correpond to random variable, and the edge correpond to dependencie between the node. Let X k be the root, which ha children Y l,..., Y l, where Λ k i the node degree of X k. Suppoe further that Y l ha children X k,..., X kγ where Γ+ = Γ l i it node degree. Suppoe the ubgraph contain all node within ditance t to X k, while only the firt 3 layer of the ubgraph are hown. It i undertood that for the random enemble of our interet, the graph i a tree i.e., cycle-free in the large-pare-ytem limit with probability. Ue Y l to denote the collection of all chip node Y m in the ubtree with Y l a the root. Ue Y k to denote the collection of Y m in the ubtree with X k a the root. Let Y km = Y k,..., Y kγ. The letter l and k here are deignated to indeing chip and ymbol repectively. 948

4 Conider a fied reference value 0 χ. In general, define the LLR function of X given ome obervation U = u a B. Denity Evolution L U X u = log p U Xu, χ. p U X u 0 Conider the etimation of X k given the ubtree Y k of depth t with X k a it root. An optimal cheme i to pa meage upward tarting from the leave. Each ymbol node X ki end to it parent the LLR function L Y k i Xki y ki, χ, while each chip node Y lj end to it parent in thi cae X k L Y l j Xk y lj, χ. The meage i in general a function defined on χ. At the final tage, the LLR given the entire ubtree Y k = y k i Λ k L Y k X k y k = L Y ln X k y ln 3 n= becaue the ubtree Y ln are independent conditioned on X k. By the central limit theorem, the LLR 3 a a um of i.i.d. random function i aymptotically Gauian for every and in fact a Gauian random proce indeed by χ. In the following we derive the mean and variance of the LLR in the large-ytem limit, which determine the equivalent channel between the deired ymbol X k and the obervation Y k. Conider L Y l Xk, which i obtained from p Y l Xk y l. The conditional ditribution p Yl X k,x km i determined by f. We average over X km conditioned on their repective ubtree to obtain p Y l Xk y l, which can be epreed uing Taylor erie epanion in Λ / k m in 4 6. Sufficient regularity condition are ued to guarantee uniform convergence o that o/λ km can walk in and out of the integral. The LLR i obtained by plugging 6 into Let L Y l Xk y l = log p Y l X k y l p Y l Xk y l 0. 7 W l = Γ m= lk m X km 8 and define a random variable Y independent of everything ele conditioned on W l, where p Y l W l y w = fy w. Clearly, Y X km Y km form a Markov chain. The integral in 6, taken over the variable in the middle of the Markov chain, i proportional to p Y l,y km y l, y km. In fact, we can write 9 in below for arbitrary g. Conequently, the LLR 7 can be epreed conciely uing conditional epectation. By 6 9, the LLR L Y l Xk Y l X k i epreed in the form of Taylor erie epanion in 0, where two of the three ignificant component involve the conditional mean of the core function log f Y W l. p Y l Xk y l = p Yl X k,x km y l, = lk f y l + [ = f y l m= Γ m= + l k Λ f y l k Γ p Y km X km y km m= lk m Γ m= lk m + l k f y l Γ m= lk m + o Λ km Γ m= dp Xkm 4 py km X km ykm dp Xkm k m 5 m= lk m ] Γ py km X ykm km dp Xkm k m 6 m= g f y l f y l m= Γ lk Γ m m= lk m m= Γ m= p Y km X km y km dp Xkm p Y km X km y km dp Xkm L Y l Xk y l = l k 0 E log fy W l Y = y l, Y km = y km + l k 0E l k 0 fy l W l fy l W l = E gx km Y l = y l, Y km = y km Y = y l, Y km = y km E log fy W l Y = y l, Y km = y km + o Λ k

5 Conider now L Y l Xk Y l a a random variable for any given, which conit of three component according to 0. We etimate it mean and variance to the firt order of /Λ k, conditioned on that the true value of X k = k. Only the firt term on the right hand ide of 0 contribute to the variance of the LLR in the order of O/Λ k. The mean of the LLR i a little more involved, becaue the prior of Y l i lightly different than that of Y. If they were the ame, the epectation of the firt two term on the right hand ide of 0 i 0 due to propertie of the core function. While the third term contribute in the order of O/Λ k, the difference between the prior of Y l and Y i of ize O/ Λ k. Taking into account the mall correction, the firt term alo contribute to the mean in the order of O/ Λ k, while the econd term remain inignificant. Let u define η = lim K E E log fy l W l Y l, Y km Conditioned on X k = k, the mean of the LLR i then E L Y l Xk Y l Xk = k = η 0 k 0 + o Λ k where we ue the fact that lk =, and the variance i E L Y l X k Y l X k = k = η 0 +o 3 Λ k Λ k In view of 3, the aymptotic tatitic of the Gauian LLR L Y k X k Y k are var L Y k X k Y k = η 0 4 E L Y k X k Y k = η k 0 η 0. 5 Propoition : The LLR L Y k X k Y k i aymptotically Gauian conditioned on X k and identically ditributed a L Z Xk Z = ηz 0 η 0 6 where Z = ηx k + N 7 and N N 0, i tandard Gauian. Proof: Let Z be defined according to 7. The likelihood L Z Xk Z = log ep [ Z η ] ep [ Z η 0 ] 8 i equal to the right hand ide of 6, the mean and variance of which are identical to thoe of L Y l Xk Y l. The ignificance of Propoition i that in term of etimating X k, having acce to the output of the companion calar channel 7 i a good a oberving the entire ubtree Y k. Indeed, there eit a conditionally Gauian variable Z = fy k, which i a ufficient tatitic of Y k for X k in the large-pare-ytem limit. The SNR of the equivalent channel η i given by. In the following, we briefly eplain why η can be obtained from the evolution formula 5. The problem of etimating X km given Y km i identical to that of etimating X k given Y k, ecept for two point: i the ubtree with X km a the root ha depth t ; ii X km ha Λ km children while X k ha Λ k. The impact of ii vanihe a Λ k. A a reult of i, can be regarded a an evolution of η t, which i dependent on the depth of the ubtree. It uffice to evaluate the variance of W l given Y km, which i the um of the variance of X km given Y km due to conditional independence. The individual variance of X km given Y km depend on Y km, which i equivalent to the ufficient tatitic Z = η t X k + N. A the linear combination of X km, W l i aymptotically Gauian whoe variance i β time the variance of individual X km given Y km. Indeed, if the left hand ide of i replaced by η t+, then W l in can be replaced by βν t W where W N 0,, and ν t i the average variance of X conditioned on it noiy obervation through a Gauian channel with SNR equal to η t. The iterative formula 5 6 are thu jutified. Furthermore, the function h in Theorem produce the ame Gauian tatitic a BP doe aymptotically. REFERENCES [] T. Tanaka, A tatitical mechanic approach to large-ytem analyi of CDMA multiuer detector, IEEE Tran. Inform. Theory, vol. 48, pp , Nov. 00. [] D. Guo and S. Verdú, Randomly pread CDMA: Aymptotic via tatitical phyic, IEEE Tran. Inform. Theory, vol. 5, pp , June 005. [3] T. Tanaka and M. Okada, Approimate belief propagation, denity evolution, and tatitical neurodynamic for CDMA multiuer detection, IEEE Tran. Inform. Theory, vol. 5, pp , Feb [4] Y. Kabahima, A CDMA multiuer detection algorithm on the bai of belief propagation, Journal of Phyic A: Mathematical and General, vol. 36, pp., 003. [5] J. P. Neirotti and D. Saad, Improved meage paing for inference in denely connected ytem, Europhy. Lett., vol. 7, no. 5, pp , 005. [6] A. Montanari and D. Te, Analyi of belief propagation for non-linear problem: The eample of CDMA or: How to prove Tanaka formula, in Proc. IEEE Information Theory Workhop, Punta del Ete, Uruguay, 006. [7] D. Guo and C.-C. Wang, Aymptotic mean-quare optimality of belief propagation for pare linear ytem, in Proc. IEEE Information Theory Workhop, Chengdu, China, 006. [8] C.-C. Wang and D. Guo, Belief propagation i aymptoticly equivalent to MAP detection for pare linear ytem, in Proc. 44th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, 006. [9] T. J. Richardon and R. L. Urbanke, The capacity of low-denity paritycheck code under meage-paing decoding, IEEE Tran. Inform. Theory, vol. 47, pp , Feb. 00. [0] C. C. Wang, S. R. Kulkarni, and H. V. Poor, Denity evolution for aymmetric memoryle channel, IEEE Tran. Inform. Theory, vol. 5, pp , Dec [] T. Tanaka, Replica analyi of performance lo due to eparation of detection and decoding in CDMA channel, in Proc. IEEE Int. Symp. Information Theory, Seattle, WA, USA, June 006. [] S. Lityn and V. Shevelev, On enemble of low-denity parity-check code: Aymptotic ditance ditribution, IEEE Tran. Inform. Theory, vol. 48, pp , Apr. 00. [3] J. Boutro and G. Caire, Iterative multiuer joint decoding: Unified framework and aymptotic analyi, IEEE Tran. Inform. Theory, vol. 48, pp , July