The sum-product algorithm on simple graphs

Transcription

1 Th sum-product agorithm on simp graphs Micha. E. O Suivan*, John Brvik, Shayn. M. Vargo, Dpartmnt of Mathmatics and Statistics San Digo Stat Univrsity San Digo, CA, USA, 98 Emai: mosuiv@math.sdsu.du Dpartmnt of Mathmatics and Statistics Caifornia Stat Univrsity, Long Bach, CA, 984 Emai: jbrvik@csub.du Dpartmnt of Mathmatics Univrsity of Southrn Caifornia Los Angs, CA, 989 Emai: svargo@usc.du Abstract This artic summarizs work in progrss on thortica anaysis of th sum-product agorithm. Two famiis of graphs with quit diffrnt charactristics ar studid: graphs in which a chcks hav dgr two and graphs with a sing cyc. Each famiy has a rativy simp structur that aows for prcis mathmatica rsuts about th convrgnc of th sum-product agorithm. I. INTRODUCTION On of th grat achivmnts in coding thory in th ast dcad was th discovry that itrativ dcoding mthods, such as th sum-product agorithm (SPA), can b usd to achiv Shannon capacity. This has bn shown xprimntay and provn for nsmbs of cods in, for xamp, [, 3.4,4.4]. Unfortunaty, athough thr ar provab asymptotic rsuts for th prformanc of th sum-product agorithm, thr is itt that can b said for spcific finit ngth cods. In this artic w rport on two simp cass for which w can driv thortica rsuts about convrgnc of th sum-product agorithm. By stabishing som simp, but provab, rsuts w hop to buid a foundation for furthr agbraic anaysis. Ths xamps may aso nhanc th intuitiv undrstanding of th agorithm and thrby yid improvd huristic mthods for cod construction. W wi ony considr binary cods, and sinc th SPA is most asiy dscribd via a bipartit graph, rathr than a chck matrix, w wi not mntion th chck matrix in our anaysis. It has provn usfu to us th foowing dfinition of a bipartit graph: A bipartit graph is a 5-tup, B (E, L, R, λ, ρ) consisting of an dg st E and two sts, th bit nods L and th chck nods R with two structura maps λ : E L and ρ : E R giving th nds of ach dg E. A codword is an association of or to ach L such that ach r R is connctd to an vn numbr of nonzro bits. Throughout th rst of th papr B (E, L, R, λ, ρ) is assumd to b a connctd bipartit graph. W xprss a of th probabiistic data in th agorithm using odds ratios. Th input data for bit is th odds that th actua intndd or transmittd vau for that bit was givn th signa rcivd, that is, u p ()/p (). Likwis, th mssags aong th dgs of th graph producd by th agorithm ar xprssd as th odds of. Th agorithm uss th transform from th odds of domain to th diffrnc domain in which a probabiity distribution p is rprsntd using p() p(). Th function s : R { } R { } dfind by s(x) x +x transforms from on domain to th othr. Notic that s(s(x)) x. Sum-Product Agorithm INPUT: For ach L, u (, ). DATA STRUCTURES: For ach E, x, y (, ). INITIALIZATION: St y for a E. ALGORITHM: BIT-TO-CHECK STEP: For ach E, st x u λ() f:λ(f)λ() f CHECK-TO-BIT STEP: For ach E, st y s f:ρ(f)ρ() f NEW ESTIMATE STEP: St û u λ () y f s(x f ) Whn ncssary w indicat th itration using a suprscript y

2 as foows. W initiaiz y () and for t, x (t) u λ() s f:λ(f)λ() f f:ρ(f)ρ() f y (t ) f s(x (t) f ) W sk conditions undr which ach û convrgs ithr to or to. Sction II trats graphs in which ach chck nod r R has dgr two. Th agorithm simpifis so that tchniqus from inar agbra may b appid. Th fina thorm dscribs xacty th rgion of convrgnc for th SPA. Sction III trats graphs in which thr is a sing cyc. W introduc a sight gnraization of th SPA, which aows input at th chck nods, and rduc th SPA on a sing cyc graph to this gnraizd agorithm appid to a simp cyc. Onc again w driv rsuts using inar agbra. II. CHECK NODES OF DEGREE TWO It is rativy straightforward to show that if a chck nods hav dgr, thn a dg mssags ar monomias in th u. Furthrmor, for ach dg, at any itration t, x (t) ē whr ē is th uniqu dg sharing a chck nod with. Lt us us a N L to dnot th row vctor of xponnts apparing in x, so x L ua,. W wi abbrviat this product as u a. Whn w want to spcify th tth itration w wi writ a (t). Lt N L b th zro row vctor and t δ N L b th row vctor which is in th th componnt and othrwis. Th updat in th SPA is x u a u λ() f:λ( f)λ() f u a f Kping track of th xponnts w hav th foowing agorithm. Loca Sum Agorithm DATA STRUCTURES: For ach E, a, N L. INITIALIZATION: St a for a E. ALGORITHM St a δ λ() + a f () f:λ(f)λ() f Lt A b th E L matrix whos th row is a and t Λ b th E L matrix with Λ, whn λ() and Λ, othrwis. So, th th row of Λ is δ λ(). Lt K b th E E matrix K,f whn λ(f) λ() and f, othrwis K,f. Th oca sum agorithm may thn b xprssd as A () and A (t) Λ + KA (t ) Th quation abov is asiy sovd, for t, ( ) A (t) K t + K t + + K + I Λ On can chck that K is th adjacncy matrix of a dirctd graph G, which w ca th fow graph of B, whos vrtx st is E. W now appy th thory of nonngativ matrics []. A nonngativ matrix K is cad primitiv whn som powr of th matrix is stricty positiv. In this cas thr is a uniqu ignvau of maximum moduus, this ignvau is positiv, and it has agbraic mutipicity on. A corrsponding ignvctor, cad th Prron vctor of K, is stricty positiv. Th foowing thorm shows that th sum-product agorithm convrgs away from a st of masur providd K is primitiv. Thorm.: Suppos K is primitiv. Lt y b th ft Prron vctor of K and t c y Λ. Th sum-product agorithm on B convrgs to zro whn u c < and convrgs to whn u c >. Whn B is rguar it foows that y is constant, and consqunty that c is aso constant. Thn th SPA convrgs if and ony if th product L u. It is possib for K to not b primitiv, which corrsponds to G bing mutipartit. Considr th simp xamp of two bit nods and n[ chck nods, ] ach connctd to both bit nods. Thn K K with K K and K primitiv. Th SPA convrgs if u u n and u n u hav th sam parity, that is both argr than or both smar than. Othrwis th agorithm osciats. Th cas n 3 appars in Figur II, aong with th associatd fow graph, which happns to b undirctd in this cas. For n 3 K K ē f f g ḡ ē f f g ḡ Fig.. A cas whr th SPA osciats on a rgion. At ft th bipartit graph B, at right th fow graph of B, G.

3 III. SINGLE CYCLE GRAPHS W now assum B is a bipartit graph which has a sing cyc. W may writ B as th union of th cyc C with svra trs, T,..., T r, ach disjoint from on anothr and ach mting th cyc in a sing nod. For ach tr, th mssags aong th dgs going toward th cyc wi vntuay stabiiz, aftr a numbr of itrations qua to th diamtr of th tr. Thus aftr a arg nough numbr of itrations th mssags into th cyc from th trs wi b constant. A. Rduction to a cyc W woud ik to compar th prformanc of th SPA on B, aftr this stting down priod, with th prformanc of th SPA on C, but thr ar compications. First, whn th mssags finay stabiiz, th dg mssags on th cyc ar no ongr, as thy whr at initiaization. Thus w considr mor gnra initiaizations. Scond, w must atr th input vaus at th nods of th cyc. If on of th T i is connctd to a bit nod thn w may simpy atr th vau for th bit nod by mutipying by this incoming mssag from T i. In ordr to hand th cas whr on of th T i is connctd to a chck nod, w aow initiaization of th chck nod. Thus w arriv at th foowing, sighty mor gnra, agorithm. Not that th cas v r for a r R givs th usua SPA. Gnraizd Sum-Product Agorithm INPUT : For ach L, u (, ). For ach r R, v r [, ). For ach E, y () (, ). DATA STRUCTURES: For ach E, x, y (, ). INITIALIZATION: St y y () for ach E. BIT-TO-CHECK STEP: For ach E, st x u λ() y f. f:λ(f)λ() f CHECK-TO-BIT STEP: For ach E, st y s s(v ρ()) NEW ESTIMATE STEP: For ach L, st For ach r R, st û u ˆv r s s(v r ) f:ρ(f)ρ() f λ () ρ (r) y. s(x f ). s(x ). T T r Fig.. ū û T v r ˆv r T Graphica rprsntation of th rduction construction. Again, w wi dcar bit to b if û <, to b if û >, and undcidd if û. W may now rduc an instanc of th GSPA on B to an instanc of th GSPA on th bipartit cyc C containd in B. Lt L b th bit nods of C and R th chck nods of C and t T for L b th trs attachd to th bit nods and S r for r R th trs attachd to th chck nods. Any on of ths may consist of a sing nod. Lt N b som sufficint numbr of itrations for th incoming mssags from th T and S r to stabiiz. For ach L t ū b th vau to which th GSPA convrgs on T at nod (i.. u (N) ). Simiary, t v r b th vau to which th GSPA convrgs on S r at nod r. For dgs of C, t ȳ () y (N) b th vau that th GSPA on B attains aftr N itrations. W rfr to th instanc ū for L, v r for r R and initiaization ȳ () as th rduction to C of th instanc u, v r, y () on B. Thorm 3.: For ach dg of C, x (t) y (t+n) ȳ (t). x (t+n) and As a consqunc of this thorm w concud that convrgncs (or ack throf), nw stimats, and dcisions for rduction to C coincid with thos obtaind on B. Thus w considr th GSPA on a simp cyc to s what may b dtrmind about convrgnc. B. Th GSPA on a simp cyc W now suppos that B is a r-cyc, numrating th bit nods, chck nods, and dgs as in Figur 3. To giv us an ida of th bhavior of th GSPA on th cyc, t us first considr th y updat aong dg. For itration t r, w

4 3 r- r- r- r- r- r-3 r- r-3 r Fig. 3. Bipartit graph for th simp r-cyc. obtain ( ( s s(v r )s ( ( s s(v r )s x (t) r )) u r y (t ) r Continuing th rcursiv opration w coud arriv at an xprssion for in trms of y (t r) and th input vaus. It is usfu to writ ach dg mssag as a fraction, y m n (suprscripting with t as ndd). Lmma 3.: For ach,..., r thr is a matrix M, whos ntris ar sums of distinct monomias in th u, v r, such that [ m (t) n (t) ] M [ ] m (t r) n (t r) In th numration in Figur 3, th dgs and + ar both incidnt on bit. Th foowing mma shows that th matrics M and for M + ar ratd. Not that ths matrics corrspond to cockwis vrsus countrcockwis fow of mssags. [ ] a b Lmma 3.3: Lt M. Thn a c d a +, d d +, and b c b + c +. For input vaus u > and v r, M is a stricty positiv matrix, providd that at ast on of th v r is positiv. Thorm 3.4: Considr th GSPA appid to th simp r-cyc with positiv bit nod inputs u, non-ngativ chck nod inputs v r and initiaization y (). If som v r is nonzro thn y ( ) b d a +. (d a ) + 4b c û ( ) 4b c (d a + (d a ) + 4b c ). )) Th th componnt of th dcision vctor is whn a k < d k, whn a k > d k, and undcidd othrwis. () C. Examps Fig. 4. Bipartit graph for th oipop cod. Th first xamp w wi anayz is th Loipop in Figur 4, a simp 4-cyc with on xtranous dg attachd to a chck nod. Th SPA wi convrg for a input. Lt (u u + )D + 6u ρ(u u ) u 4u u u u whr Du 9u u 4u u + 9. Th dcision vctor wi b [,,], if u u < and u < ρ; [,,], if u u < and u > ρ; [,,], if u u > and u < ρ; [,,], if u u > and u > ρ; undcidd if u u or u ρ. This impis that th SPA my produc non-codwords. Th codword dcisions occur whn to u < ρ. To bttr undrstand th bhavior of ρ, Tab I givs a ist of vaus for various products of u u, aong with th convrgnc vaus of th nw bit stimat. Athough w ony show products u u, not that ρ(/u u ) ρ(u u ). D. Examp : Th Pu-Toy Again, w can appy th rduction construction with N. W obtain th simp 4-cyc with inputs ū u, ū u, v u, and v u 3. Th initia updat matrics ar idntica to thos givn in th prvious xamp. Bow, w giv th convrgnc vau for y to dispay th growing compxity of th mssags. W hav whr y ( ) (u u 3 + u ) + u u u 3 u u u u u 3 + D, D (+u u u 3 u u u u u 3 ) +4(u u +u 3 )(u u u 3 +u u ).

5 TABLE I LIST OF VALUES PRODUCED BY THE SPA FOR THE LOLLIPOP CODE u u ρ u û ( ) û ( ) û ( ) Fig. 5. Bipartit graph for th pu-toy. Unik th oipop, th dcisions at th bits on th cyc now dpnd on th xtrior bit inputs. Thr ar two diffrnt cass. Th dcision vctor for th cyc bits wi b ; [,] or [,], if u u 3 < uu u u. [,] or [,], if u u 3 > uu u u Not that whi this sparats codwords and non-codwords on th simp cyc, any of th four vctors might ad to a codword (or non-codword) for th pu-toy. REFERENCES [] H. Minc, Nonngativ Matrics, Wiy, 988. [] Richardson, T., Urbank, R., Modrn Coding Thory, Cambridg Univrsity Prss, 8.