Concise Derivation of Complex Bayesian Approximate Message Passing via Expectation Propagation

Transcription

1 Concse Dervon of Complex Byesn Approxme Messge Pssng v Expecon Propgon Xngmng Meng, Sheng Wu, Lnlng Kung, Jnhu Lu Deprmen of Elecronc Engneerng, Tsnghu Unversy, Bejng, Chn Tsnghu Spce Cener, Tsnghu Unversy, Bejng, Chn Tsnghu Nonl Lborory for Informon Scence nd Technology, Bejng, Chn Eml: mengxm11@mlssnghueducn, {hury, kll, lhh-dee}@mlsnghueducn rxv: v [csit 5 Jn 016 Absrc In hs pper, we ddress he problem of recoverng complex-vlued sgnls from se of complex-vlued lner mesuremens Approxme messge pssng AMP s one se-ofhe-r lgorhm o recover rel-vlued sprse sgnls However, he exenson of AMP o complex-vlued cse s nonrvl nd no deled nd rgorous dervon hs been explcly presened To fll hs gp, we exend AMP o complex Byesn pproxme messge pssng CB-AMP usng expecon propgon EP Ths novel perspecve leds o concse dervon of CB- AMP whou sophsced rnsformons beween he complex domn nd he rel domn In ddon, we hve derved se evoluon equons o predc he reconsrucon performnce of CB-AMP Smulon resuls re presened o demonsre he effcency of CB-AMP nd se evoluon Index Terms Compressed sensng, complex-vlued pproxme messge pssng, expecon propgon, se evoluon I INTODUCTION Compressed sensng CS ms o undersmple hghdmensonl sgnls ye ccurely reconsruc hem by explong her srucure [1, [ To hs end, plehor of mehods hve been proposed n he ps yers [3 Among ohers, pproxme messge pssng AMP [4 proposed by Donoho e l s one se-of-he-r lgorhm o recover sprse sgnls As n effcen pplcon of belef propgon [5, [6, AMP hs found vrous pplcons n solvng lner nverse problems Moreover, AMP hs been exended o Byesn AMP B-AMP [7, [8 nd generl lner mxng problems [9 [11 However, mos of he exsng works focus on he cse of relvlued sgnls nd mesuremens, whle n mny pplcons, eg, communcon [10, mgnec resonnce mgng [1, nd rdr mgng [13, ec, s more convenen o represen sgnls n he complex-domn [14 Though cn be rnsformed nd processed n he rel domn, s benefcl o del wh complex-vlued sgnls n srghforwrd wy snce her rel nd mgnry componens re ofen eher boh zero or boh non-zero smulneously [14, [15 The exenson of AMP o del wh complex-vlued sgnls wh complex-vlued mesuremens hs lredy been consdered n [10, [15 [18 In [15, he uhors proposed one knd of complex pproxme messge pssng CAMP lgorhm However, he exenson of AMP o CAMP s sophsced A more compc form of CAMP s proposed n [10, [16 [18 To he bes of our knowledge, lhough such exensons hve been consdered, no deled nd rgorous dervon hs been explcly presened In [19, we derved he orgnl AMP lgorhm from he expecon propgon EP [0, [1 perspecve, whch unvels he nrnsc connecon beween AMP nd EP Neverheless, only dels wh rel-vlued sprse sgnls wh Lplce pror, whch lms s use n more generl problems In hs pper we furher exend o complex Byesn AMP CB-AMP, e, complex-vlued sgnl reconsrucon wh generl known pror dsrbuon Ths novel perspecve leds o concse nd nurl exenson from AMP o CB-AMP, whou sophsced rnsformons beween he complex domn nd he rel domn In ddon, we hve lso derved se evoluon equons o predc he reconsrucon performnce of CB-AMP The superory of CB-AMP s demonsred v smlon resuls, whch re conssen wh he predcon resuls of se evoluon equons A Sysem Model II DEIVATION OF CB-AMP VIA EP Consder complex-vlued lner sysem of he form y = Ax+w, 1 x C N s he unknown complex sgnl, A C M N s he mesuremen mrx, w C M s he ddve complex Gussn nose wh zero men nd covrnce mrx σ I M, I M s he deny mrx of sze M The complex Gussn dsrbuon of w s denoed by CN w;0,σ I M The pror dsrbuon of sgnl x s supposed o be known nd hs seprble form N p 0 x = p 0 x The gol s o esme x from he nosy observons y gven A nd he sscl nformon of x nd w usng he mnmum men squre error MMSE creron I s well

2 known h he MMSE esme of x s he poseror men, e, x = x p x y dx, p x y s he mrgnl dsrbuon of he jon poseror dsrbuon p x y = p y x p 0 x p y M p y x N p 0 x, 3 =1 denoes deny beween wo dsrbuons up o normlzon consn Under he sscl ssumpon of mesuremen nose w, he condonl dsrbuon of he - h elemen of y, y, gven x cn be explcly represened s p y x = 1 πσ exp 1 y σ A x 4 Messge pssng lgorhms [5, [6, [ provde fmly of effcen mehods o pproxmely compue he mrgnls The bsc prdgm s well llusred v fcor grph [6 whch represens he sscl dependences beween rndom vrbles The fcorzon n 3 cn be encoded n fcor grph G = V,F,E, V = {} s he se of vrble nodes, F = {} s he se of fcor nodes nd E denoes he se of edges In he sequel, we ssume h he mesuremen mrx A s homogenous mrx whose elemens dm d dsrbuon wh men zero nd vrnce γ B Approxme nference usng EP Gven he fcor grph represenon, he mrgnls cn be compued dsrbuvely v locl messge pssng [6, [0, [1 The projecon of prculr dsrbuon p no dsrbuon se Φ s defned s [1 Proj Φ [p = rgmndp q, 5 q Φ Dp q denoes he Kullbck-Lebler dvergence Denoe by m x nd m x he messge from vrble node o fcor node n he h eron nd he messge n he oppose drecon, respecvely Then, he messge pssng upde rules of EP red [0, [1 x m +1 m [ Proj Φ p0 x x b m b, 6 x m 1 x Proj m Φ [m x x m j xj p y x 7 j Afer projecon, ech messge m j xj from vrble node j o fcor node s pproxmed s complex Gussn densy funconcn x j ;x j,ν j, hus, under he produc mesure j m j xj, he rndom vrbles xj, j re ndependen complex Gussn rndom vrbles Defne Z = j A j x j, so h Z s complex Gussn rndom vrble wh men nd vrnce, respecvely, Z = j A jx j, 8 j V = A j νj 9 j Then, we obn m j xj p y x CN x ; y Z A, σ +V A, 10 whch mples h x lso dms complex Gussn dsrbuon In hs cse, he projecon operon n 7 reduces o deny operon, so h m x CN x ; y Z A, σ +V A 11 Nex we evlue he messge m +1 x The mrgnl poseror densy esme of x, e, he mrgnl belef esme m +1 x, s defned s m +1 x = ProjΦ [p 0 x m x 1 Accordng o he produc rule of Gussn funcons [3,we hve m x CN x ;,Σ, 13 Σ A 1, = 14 σ +V = A y Z Σ σ +V, 15 nd denoes conjuge operon For noonl brevy, we nroduce fmly of densy funcons px;,σ p 0 x [ z,σ exp x Σ, 16 z,σ = p 0 x exp x /Σ dx s he normlzon consn As n [8, [18, he correspondng men nd vrnce re denoed s f,σ = xpx;,σdx, 17 f c,σ = x f,σ px;,σdx 18 Combnng13,16,17 nd 18, we obn he enve pproxmon of he poseror men nd vrnce of x n he +1h eron, whch re denoed by f,σ nd f c,σ, respecvely Then, usng projecon operon1 nd momen mchng, we projec he poseror belef o he complex Gussn dsrbuon se, yeldng x CN x ;x +1,ν +1 m +1 x +1 = f,σ,, 19

3 ν +1 = f c,σ Accordng o 6 nd 1, he messge from vrble node o fcor node s evlued by 1 ν +1 m +1 x CN x ;x +1,ν+1, 0 = 1 ν +1 x +1 = ν+1 A σ +V +1 x ν +1 A, 1 y Z σ +V Now we hve closed he messge compuon However, bou OMN messges need o be compued In he sequel, we furher reduce he number of messges per eron o OM +N by neglecng he hgh order erms n lrge sysem lm C educng he number of messges Defne Z = V = A x, 3 A ν, 4 Then, cn be esly seen h 8 nd 9 cn be rewren s Z = Z A x, 5 V = V A ν 6 Neglecng he hgh order erm A ν n 6, we hve V V, 7 whch s ndependen of The smplfcon ofz s no h rvl snce we should be creful o keep he Onsger recon erm n pproxmng Z From 1, neglecng he hgh order erm A /σ + V, we obn ν +1 ν+1, 8 so h V A ν 9 Subsung 8 nd 7 no, we hve x +1 x+1 ν +1 A y Z σ +V 30 Combnng 5, 7, nd 30, we hve Z = Z A x +ν A y Z 1 σ +V 1, 31 whch leds o furher pproxmon x +1 x+1 ν+1 A [ σ +V y Z +A x ν A y Z 1 σ +V 1 x +1 y Z ν +1 σ +V A, 3 he ls sep s pproxmed by neglecng he hgh order erms Then, Z defned n 3 cn be pproxmed s Z Subsung 7 no 14 leds o A x y Z 1 σ +V 1 V 33 Σ A 1 34 σ +V Subsung 7 nd 31 no 15, we hve A y Z Σ σ +V +Σ A σ +V x Σ A σ +V ν A y Z 1 σ +V 1 x A y Z +Σ σ +V, 35 n he ls sep we hve negleced hgh order erm nd used he relonshp 34 A hs sep, we fnlly obn he complex Byesn pproxme messge pssng CB-AMP s shown n lgorhm1, whch s he sme s h n [18 nd [10 noe h some noonl modfcon s needed o mch [10 Algorhm 1 CB-AMP 1 Inlzon: = 1,x = 1 x p 0 x dx,ν = 1 x x 1 p 0x dx, = 1,,N,V 0 = 1,Z 0 = y, = 1,,M Fcor node upde: For = 1,,M V = Z = A ν, A x V σ +V 1 y Z 1 3 Vrble node upde: For = 1,N Σ = A 1, σ +V = A y Z x +Σ σ +V, x +1 = f,σ, ν +1 = f c,σ 4 Se +1 nd proceed o sep unl predefned number of erons or oher ermnon condons re ssfed

4 III STATE EVOLUTION ANALYSIS We re neresed n he men squre error MSE o chrcerze he reconsrucon performnce, whch s defned s MSE = 1 N x x 36 N As s shown n [8, he se evoluon or cvy mehod uses sscl nlyss of he messges eron, n he lrge sysem lm, o derve her dsrbuons eron +1 Defne V = 1 N N ν, E = 1 N N x x 37 We frs focus on he clculon of Subsung 8, 9, 14 s well s he sysem model 1 no he defnon 15, we hve A x +A w+a = σ + j Aj νj A σ + j Aj νj A x +A w +A b x + 1 j x Aj j x j A A w + j A j xj x j A j xj x j, A j 38 sep n 38 s due o he ssumpon h σ + j A j νj s ndependen of µ such h s cnceled ou n he denomnor nd numeror; sep b n 38 s rbued o he ssumpon h A dms d dsrbuon wh men zero nd vrnce γ so h A γ = Denoe by r = A w + A j A j xj x j, hen r s complex rndom vrble wh respec o he dsrbuon of he mesuremen mrx elemens nd he complex Gussn nose w CN w ;0,σ By cenrl lm heorem, cn be verfed h r s complex Gussn rndom vrble wh zero men nd vrnce σ + γne Thus, cn be reformuled s σ = x + +γne z, 39 z CN z;0,1 s complex Gussn rndom vrble wh zero men nd un vrnce Then, from 34, we obn h Σ σ +γnv 40 Thus, he MMSE esme of x he +1 -h eron s gven by f Σ,, nd he correspondng MSE reds E +1 = dx P 0 x Dz f Σ, x, 41 P 0 x s he pror dsrbuon defned n nd Dz s he un complex Gussn mesure Dz = e z dz/π Accordng o 37, he verge vrnce esme he + 1 -h eron s gven by V +1 = dx P 0 x Dzf c Σ, 4 So h 41 nd 4 consue he se evoluon equons for CB-AMP IV SIMULATION ESULTS We evlue he performnce of CB-AMP for reconsrucon of complex-vlued sprse sgnls The elemens of mesuremen mrx A re genered usng d complex Gussn dsrbuon wh men zero nd vrnce γ = 1/N The complex-vlued sprse sgnls re ssumed o follow N ρ-sprse Bernoull-Gussn dsrbuon, e, p 0 x = 1 ρ δ x +ρcn x ;µ,τ, 0 < ρ < 1, µ, τ re known In hs cse, fer some lgebr, he poseror men nd vrnce defned n 17 nd 18 cn be clculed s m f,σ = 1 ρ τ ρ V exp µ, 43 τ m V +1 f c,σ = ρv τ exp m V µ τ m Σ +V Z,Σ f,σ, 44 V = τσ Σ+τ, 45 m = τ+σµ, 46 Σ+τ Z,Σ = 1 ρ exp 47 Σ +ρ V τ exp m V µ, 48 τ Σ In he nosy cse, he MSE performnces of dfferen mehods re depced n Fg 1 when N = 10 3,α = M/N = 05,ρ = 01,µ = 0,τ = 1 Oher smulon scenros re omed due o lck of spce Compred wh he rel AMP mehod, whch convers he complex sgnl o he rel domn before processng, CB-AMP mproves he MSE evdenly nd converges more quckly In ddon, he heorecl se evoluon predcon mches closely wh he expermenl resul, mplyng h he performnce of CB-AMP cn be ccurely predced by se evoluon In he noseless cse, e, σ = 0, he phse rnson curves re shown n Fg For boh rel AMP nd CB- AMP, he sgnl lengh s N = 1000, nd number of erons s se o be T = 500 The phse rnson curves dsply he relonshp beween he mesuremen re α nd he sprsy re ρ success re of 50%, he success of recoverng he orgnl sgnl s sed f he men squre error MSE < 10 4 The lne α = ρ ndces he mxmum-poseror MAP hreshold As shown n Fg, CB-AMP

5 mproves he phse rnson curve of rel AMP sgnfcnly, whch s rbued o he srucured sprsy of he complex sgnl MSE σ = 10 3 σ = 10 5 Se Evoluon, σ = 10 3 AMP rel, σ = 10 3 CB AMP, σ = 10 3 Se Evoluon, σ = 10 5 AMP rel, σ = 10 5 CB AMP, σ = Ierons Fgure 1 MSE versus he number of erons N = 10 3,α = 05,ρ = 01, µ = 0,τ = 1 mesuremen re α CB AMP AMP rel α = ρ sprsy re ρ Fgure Phse rnson curve N = 10 3, µ = 0,τ = 1, Number of erons T = 500 Success s sed f MSE < 10 4 V CONCLUSION In hs pper, we consdered he problem of recoverng complex-vlued sgnls from se of complex-vlued lner mesuremens Usng EP, we hve exended he AMP lgorhm o complex-vlued Byesn AMP CB-AMP Ths novel perspecve leds o more concse nd nurl dervon of CB-AMP, whou resorng o sophsced rnsformons beween he complex domn nd he rel domn Se evoluon equons for CB-AMP re lso derved Smulon resuls demonsre h CB-AMP ouperforms rel AMP n he complex-vlued cse nd h se evoluon predcs he reconsrucon performnce of CB-AMP ccurely ACKNOWLEDGMENTS The frs uhor would lke o hnk C Schülke nd P Schner for vluble dscussons on complex AMP Ths work ws prlly suppored by he Nonl Nure Scence Foundon of Chn Grn Nos , , nd , he Nonl Bsc eserch Progrm of Chn Grn No 013CB39001 EFEENCES [1 D L Donoho, Compressed sensng, IEEE Trns Inf Theory, vol 5, no 4, pp , Apr 006 [ E J Cndès nd M B Wkn, An nroducon o compressve smplng, IEEE Sgnl Process Mg, vol 5, pp 1 30, Mr 008 [3 Y C Eldr nd G Kuynok, Eds, Compressed sensng: heory nd pplcons Cmbrdge Unv Press, 01 [4 D L Donoho, A Mlek, nd A Monnr, Messge-pssng lgorhms for compressed sensng, n Proc N Acd Sc, vol 106, no 45, Nov 009, pp [5 J Perl, Probblsc esonng n Inellgen Sysems: Neworks of Plusble Inference Morgn Kufmnn Pub, 1988 [6 F Kschschng, B J Frey, nd H-A Loelger, Fcor grphs nd he sum-produc lgorhm, IEEE Trns Inf Theory, vol 47, no, pp , Feb 001 [7 D L Donoho, A Mlek, nd A Monnr, Messge pssng lgorhms for compressed sensng: I movon nd consrucon, n IEEE Informon Theory Workshop ITW, Jn 010, pp 1 5 [8 F Krzkl, M Mézrd, F Susse, Y Sun, nd L Zdeborová, Probblsc reconsrucon n compressed sensng: lgorhms, phse dgrms, nd hreshold chevng mrces, Journl of Sscl Mechncs: Theory nd Expermen, vol 01, no 08, p P08009, 01 [9 S ngn, Generlzed pproxme messge pssng for esmon wh rndom lner mxng, n Proc IEEE In Symp Inf Theory, 011, pp [10 P Schner, A messge-pssng recever for BICM-OFDM over unknown clusered-sprse chnnels, IEEE J Sel Topcs Sgnl Process, vol 5, no 8, pp , Dec 011 [11 U S Kmlov, S ngn, A K Flecher, nd M Unser, Approxme messge pssng wh conssen prmeer esmon nd pplcons o sprse lernng, IEEE Trns Inf Theory, vol 605, pp , My 014 [1 M T Vlrdngerbroek nd J A Boer, Mgnec resonnce mgng: heory nd prcce Sprnger Scence & Busness Med, 013 [13 Brnuk nd P Seeghs, Compressve rdr mgng, n IEEE dr Conference IEEE, 007, pp [14 Z Yng, C Zhng, nd L Xe, On phse rnson of compressed sensng n he complex domn, IEEE Sgnl Processng Leers, vol 19, no 1, pp 47 50, 01 [15 A Mlek, L Anor, Z Yng, nd G Brnuk, Asympoc nlyss of complex LASSO v complex pproxme messge pssng CAMP, IEEE Trnscons on Informon Theory, vol 59, no 7, pp , 013 [16 S Som nd P Schner, Compressve mgng usng pproxme messge pssng nd mrkov-ree pror, IEEE Trns Sgnl Process, vol 60, no 7, pp , 01 [17 J P Vl nd P Schner, Expecon-mxmzon gussn-mxure pproxme messge pssng, IEEE Trns Sgnl Process, vol 61, no 19, pp , 013 [18 J Brber, C Schülke, nd F Krzkl, Approxme messge-pssng wh splly coupled srucured operors, wh pplcons o compressed sensng nd sprse superposon codes, Journl of Sscl Mechncs: Theory nd Expermen, vol 015, no 5, p P05013, 015 [19 X Meng, S Wu, L Kung, nd J Lu, An expecon propgon perspecve on pproxme messge pssng, IEEE Sgnl Processng Leers, vol, no 8, pp , 015 [0 T Mnk, A fmly of lgorhms for pproxme Byesn nference, PhD dsseron, Msschuses Insue of Technology, 001 [1, Dvergence mesures nd messge pssng, Mcrosof eserch Cmbrdge, Tech ep, 005 [ M J Wnwrgh nd M I Jordn, Grphcl models, exponenl fmles, nd vronl nference, Foundons nd Trends n Mchne Lernng, vol 1, no 1-, pp 1 305, 008 [3 C M Bshop, Pern recognon nd mchne lernng New York: Sprnger, 006