Abstract. The aim of this article is to present a random graph. for modeling sets of attributed graphs (AGs). We refer to these models as

Transcription

1 Second-order random graph for modelng et of attrbuted graph and ther applcaton to obect learnng and recognton Alberto Sanfelu 1, Francec Serratoa 2 and René Alquézar 3 1 Unertat Poltècnca de Catalunya, Int. de Robòtca Informàtca Ind., Span anfelu@r.upc.e 2 Unertat Rora Vrgl, Dept. d Engnyera Informàtca Matemàtque, Span Francec.Serratoa@ete.ur.e 3 Unertat Poltècnca de Catalunya, Dept. de Llenguatge St. Informàtc, Span alquezar@l.upc.e Abtract. The am of th artcle to preent a random graph repreentaton, that baed on 2 nd order relaton between graph element, for modelng et of attrbuted graph (AG). We refer to thee model a econd-order random graph (SORG). The bac feature of SORG that they nclude both margnal probablty functon of graph element and 2 nd - order ont probablty functon. Th allow a more prece decrpton of both the tructural and emantc nformaton content n a et of AG and, conequently, an expected mproement n graph matchng and obect recognton. The artcle preent a probabltc formulaton of SORG that nclude a partcular cae the two preouly propoed approache baed on random graph, namely the frt-order random graph (FORG) and the functon-decrbed graph (FDG). We then propoe a dtance meaure dered from the probablty of ntantatng a SORG nto an AG and an ncremental procedure to yntheze SORG from equence of AG. Fnally, SORG are hown to mproe the performance of FORG, FDG and drect AG-to-AG matchng n three expermental recognton tak: one - 1 -

2 n whch AG are randomly generated and the other two n whch AG repreent multple ew of 3D obect (ether ynthetc or real) that hae been extracted from color mage. In the lat cae, obect learnng acheed through the ynthe of SORG model. 1 Introducton Attrbuted Graph (AG) ha been ued to ole computer on problem for decade and n many applcaton. Some example nclude recognton of graphcal ymbol [13], character recognton [18], hape analy [5,17], 3D-obect recognton [29,25] and deo and mage databae ndexng [27]. In thee applcaton, AG repreent both unclafed obect (unknown nput pattern) and prototype. Moreoer, thee AG are typcally ued n the context of nearet-neghbour clafcaton. That, an unknown nput pattern compared wth a number of prototype tored n the databae. The unknown nput then agned to the ame cla a the mot mlar prototype. A number of mlarty meaure on AG and related computatonal procedure hae been propoed n th context [3,4,7,10,14,15,20,28]. Neerthele, the man drawback of repreentng the data and prototype by AG the computatonal complexty of comparng two AG. The tme requred by any of the optmal algorthm may n the wort cae become exponental n the ze of the AG. The approxmate algorthm, on the other hand, hae only polynomal tme complexty, but do not guarantee to - 2 -

3 fnd the optmal oluton [2,23]. For ome applcaton, th may not be acceptable. Moreoer, n ome applcaton, the clae of obect are repreented explctly by a et of prototype whch mean that a huge amount of model AG mut be matched wth the nput AG and o the conentonal error-tolerant graph matchng algorthm mut be appled to each model-nput par equentally. A a conequence, the total computatonal cot lnearly dependent on the ze of the databae of model graph and exponental (or polynomal n ubgraph method) wth the ze of the AG. For applcaton dealng wth large databae, th may be prohbte. To alleate thee problem, ome attempt hae been made to try to reduce the computatonal tme of matchng the unknown nput pattern to the whole et of model from the databae. Aumng that the AG that repreent a cluter or cla are not completely dmlar n the databae, only one tructural model defned from the AG that repreent the cluter, and thu, only one comparon needed for each cluter. We dtnguh two dfferent methodologe dependng on whether they keep probabltc nformaton n the tructure that repreent the cluter of AG (a) or not (b). (a) In the frt method, the model, whch are uually called Random Graph (RG), are decrbed n the mot general cae through a ont probablty pace of random arable rangng oer graph ertce and arc. They are - 3 -

4 the unon of the AG n the cluter, accordng to ome ynthe proce, together wth t aocated probablty dtrbuton. In th manner, a tructural pattern can be explctly repreented n the form of an AG and an enemble of uch repreentaton can be condered a a et of outcome of the RG. In th paper, we brefly recall the two mot mportant probabltc method, whch are Frt-Order Random Graph (FORG) [30] and Functon-Decrbed Graph (FDG) [25,26]. The approach preented n the paper by Sengupta et al. [21] can be regarded a mlar to the FORG approach. Fnally, we ntroduce the Second-Order Random Graph (SORG), whch can be een a a generalaton of both of them [24]. (b) In the non-probabltc method, we comment four dfferent approxmaton. The elf-organzng map (SOM) a ueful method to cluter et of obect. It cont of a layer of unt (neuron), that adapt themele to a populaton of nput pattern. SOM wa frt preented by Kohonen wth the lmtaton that pattern had to be repreented n term of feature ector only. Afterward, the ame author preented an extenon of th method to trng [10] and then Günter & Bunke preented n [9] a generalaton of the cluterng method appled to AG. Moreoer, Seong et al. [22], Cordella et al. [6] and Jang et al. [12] preented three dfferent method to cluter et of AG. In the frt one, a herarchcal model that ummare and organe the nput ntance ncrementally bult up wth a ucceon of AG. In the econd one, the et of AG repreented by the - 4 -

5 maxmally general prototype that can be een a the unon of the AG. And n the thrd one, the et of AG repreented by the generaled medan of the AG that belong to the et. In the followng ecton, we ntroduce the formal defnton ued throughout the paper. In ecton 3, we recall FORG and FDG, whch are the two man approxmaton of the general RG concept propoed n the lterature. In ecton 4, we preent SORG a a qute general formulaton for etmatng the ont probablty of the random element n a RG yntheed from a et of AG. In ecton 5 and 6, we how repectely that the FORG and FDG approache can be een a dfferent mplfcaton of SORG. In ecton 7 and 8, we propoe a dtance meaure between AG and SORG and a method to ynthee SORG, repectely. Fnally, we preent a comparate tudy between SORG and other probabltc model preented n the lterature. They are appled on AG randomly generated and on 3D-obect recognton. In the lat ecton we prode ome dcuon about our contrbuton. 2 Formal Defnton of Random-Graph Repreentaton Defnton 1: Attrbuted Graph (AG). Let and e denote the doman of poble alue for attrbuted ertce and arc, repectely. Thee doman are aumed to nclude a pecal alue that repreent a null alue of a ertex or arc. An AG G oer (, e ) defned to be a four-tuple - 5 -

6 G,,,, where k 1 n a et of ertce (or node), e e k,...,, 1,..., n a et of arc (or edge), and the mappng :, e e and : agn attrbute alue to ertce and arc, repectely. e e e Defnton 2: Random Graph (RG). Let and e be two et of random arable wth alue n (random ertce) and n e (random arc), repectely. A RG R oer (, e ) defned to be a tuple where k 1 n a et of ertce,, 1,..., n, k,..., e,,, P, e e, a et of arc, the mappng : aocate each ertex k wth a random arable wth alue n, and e :e aocate each arc e e k k wth a random arable k e wth alue n e. And, fnally, P a ont probablty dtrbuton P,,,, of all the random ertce 1, n 1 ( ), 1 nand random arc ( ), 1 m m. Defnton 3: Outcome graph. An Outcome graph any AG obtaned by ntantatng all random ertce and random arc of a RG n a way that atfe all the tructural relaton. Such ntantaton aocated wth a kl tructural omorphm : G' R, where G ' the extenon of G to the order of R (n whch null-attrbute ertce and arc hae been added to form a complete AG [26]). Hence, a RG repreent the et of all poble AG that can be outcome graph of t, accordng to an aocated probablty dtrbuton

7 Defnton 4: Probablty of an outcome graph. For each outcome graph G of a RG R, the ont probablty of random ertce and arc defned oer an ntantaton that produce G. Let G be orented wth repect to R by the tructurally coherent omorphm ; for each ertex n R, let 1 a be the correpondng attrbute alue n G, and mlarly, for each arc kl n R (aocated wth random arable 1 ) let b be the correpondng attrbute alue n G. Then the probablty of G accordng to (or gen by) the orentaton, denoted by P P G, defned a n m G a b pa,, a, b,, b Pr (1) 1 n 1 m 1 1 e kl 3 Approxmatng Probablty Dtrbuton n the Lterature If we want to repreent the cluter of AG by a probablty dtrbuton t mpractcal to conder the hgh order probablty dtrbuton,,, where all component and ther relaton n the P, 1, n 1 m tructural pattern are taken ontly (eq. 1). For th reaon, ome other more practcal approache hae been preented that propoe dfferent approxmaton [25,26,30]. All of them take nto account n ome manner the ncdence relaton between attrbuted ertce and arc,.e. aume ome ort of dependence of an arc on t connectng ertce. Alo, a common orderng (or labellng) cheme needed that relate ertce and arc of all the noled AG, whch obtaned through an optmal graph mappng proce called ynthe of the random graph repreentaton. In the - 7 -

8 followng ecton, we ummare the two man uch approache, FORG and FDG. 3.1 Frt-Order Random Graph (FORG) Wong and You [30] propoed the Frt-Order Random Graph (FORG), n whch trong mplfcaton are made o that RG can be ued n practce. They ntroduced three uppoton about the probabltc ndependence between ertce and arc: 1) The random ertce are mutually ndependent; 2) The random arc are ndependent gen alue for the random ertce; 3) The arc are ndependent of the ertce except for the ertce that they connect. Defnton 5: Frt-Order Random Graph (FORG). A FORG R a RG that atfe the aumpton 1, 2, 3 hown aboe. Baed on thee aumpton, for a FORG R, the probablty P G become n m p a q b a,a 1 P G (2) where p Pr a, 1 n, 1 1 a ˆ are the margnal probablty denty functon for ertce and q b a a ˆ Pr b a a 2, 1 m, 1, 2 1 1, 2 2 are the condtonal probablty functon for the arc, where 1, 2 refer to the random ertce for the endpont of the random arc

9 2 The torage pace of FORG nn mmn number of element of the doman and. e O where N and M are the 3.2 Functon-Decrbed Graph (FDG) Serratoa et al. [1,23,25,26] propoed the Functon-Decrbed Graph (FDG), whch lead to another approxmaton of the ont probablty P of the random element. On one hand, ome ndependence aumpton (a) are condered, but on the other hand, ome ueful 2 nd -order functon (b) are ncluded to contran the generalaton of the tructure. (a) Independence aumpton n the FDG 1) The attrbute n the ertce are ndependent of the other ertce and of the arc. 2) The attrbute n the arc are ndependent of the other arc and alo of the ertce. Howeer, t mandatory that all non-null arc be lnked to a nonnull ertex at each extreme n eery AG coered by an FDG. In other word, any outcome AG of the FDG ha to be tructurally content [26]. (b) 2 nd -order functon n the FDG In order to tackle the problem of the oer-generalaton of the ample, the antagonm, occurrence and extence relaton are ntroduced n FDG, whch apply to par of ertce or arc. In th way, random ertce and arc are not aumed to be mutually ndependent, at leat wth regard to the tructural nformaton, nce the aboe relaton repreent a qualtate - 9 -

10 nformaton of the 2 nd -order ont probablty functon of a par of ertce or arc. To llutrate the meanng of the FDG 2 nd -order relaton t conenent to plt the doman of the ont probablte n four regon (ee Fgure 1.a). P, P, P, P, Regon 2 Reg.3 Regon 4 (a) The four regon (b) Antagonm (c) Occurrence (d) Extence Fgure 1. 2 nd -order probablty of two FDG ertce The frt one compoed by the pont that belong to the Cartean product of the doman of actual attrbute of the two ertce,, correpondng to the cae where both element are not null. The econd and thrd regon are one-dmenonal (hown a traght lne) n whch only one of the ertce ha the null alue. Fnally, the fourth regon the ngle pont where both ertce are null. The 2 nd -order relaton are defned a follow: Antagonm relaton: w and w are antagontc f the probablte n the frt regon are all zero (fgure 1.b), A, 1Pr 0. In the 3D-obect modellng cae, two face are antagontc f t not poble to ee both n a ame ew. Occurrence relaton: There an occurrence relaton f the ont probablty functon equal zero n the econd regon (fgure 1.c), O, 1 Pr 0. The cae of the thrd regon

11 analogou to the econd one wth the only dfference of wappng the element. In the 3D-obect modellng cae, a face occurrent wth repect to another f alway that the former ble, the latter ble too. Extence relaton: Fnally, there an extence relaton between two ertce f the ont probablty functon equal zero n the fourth regon (fgure 1.d), E, 1 Pr 0. In the 3D-obect modellng cae, there an extence relaton between two face f one of them or both appear n all the ew ued to ynthee the model of the obect. Defnton 6: Functon-Decrbed Graph (FDG). An FDG F a RG that atfe the aumpton 1 and 2 hown aboe and contan the nformaton of the 2 nd -order relaton of antagonm, occurrence and extence between par of ertce or arc. Baed on thee aumpton, for an FDG F, P G become n m p a q b P G (3) 1 1 where p a defned a n FORG and q ˆ Pr b b. 1, 2 Howeer, the omorphm not only ha to be tructurally coherent but alo ha to fulfl the 2 nd -order contrant (antagonm, extence and occurrence) [25,26]. Otherwe, P G condered to be zero. The bac dea of thee contrant the atfacton by an AG to be matched of the antagonm, occurrence and extence relaton nferred from the et of AG ued to ynthee the FDG

12 2 2 The torage pace of FDG OnN mmn m where N and M are the number of element of the doman and, repectely. e 4 Second-order Random-Graph Repreentaton We how next that the ont probablty of an ntantaton of the random element n a RG can be approxmated a follow: P 1 G pd1,,, d p d r d, d (4) where p d are the margnal probablte of the random element (ertce or arc) and r are the Peleg compatblty coeffcent [16] that take nto account both the margnal and 2 nd -order ont probablte, r d Pr d d, d (5) p d p d The Peleg coeffcent, wth a non-negate range, related to the degree of dependence between two random arable. If they are ndependent, the ont probablty defned a the product of the margnal one, thu, r = 1 (or a alue cloe to 1 f the probablty functon are etmated). If one of the margnal probablte null, the ont probablty alo null. In th cae, the ndecene 0/0 oled a 1, nce th do not affect the global ont probablty, whch null. Eq. (4) obtaned by aumng ndependence n the condtonal probablte (ecton 4.1) and rearrangng the ont probablty expreon ung Baye rule (ecton 4.2),

13 4.1 Condtonal Probablte The condtonal denty probablty p / 1,..., of a random element ued to compute the ont denty probablty p 1,...,. Applyng the Baye rule to the condtonal probablty, the followng expreon hold, p / 1,..., p p 1,..., / (6) p,..., Due to the fact that th (+1-)-order probablty can not be tored n practce, we hae to uppoe at th pont that the condtonng random arable 1 to are ndependent to each other. In that cae, an etmate gen by p p / 1,..., p p (7) 1 p / 1 1 p p, p Thu, f we ue the Peleg compatblty coeffcent then the condtonal probablty, prob d / 1 d 1,..., d p d r d, d (8) Jont Probablty Ung the Baye theorem, the ont probablty denty functon p 1,,, can be plt nto the product of another ont probablty functon and a condtonal one, p,, p,,, p /,,, 1, (9)

14 and applyng n-1 tme the ame theorem on the remanng ont probablty, 1 1 1, 1,,, p p /,, p (10) If we ue eq. (8) to etmate the condtonal probablte, then the ont probablty p(d 1,,,d ) can be etmated a p*(d 1,,,d ) where, P (11) G p * d1,,, d pd p d r d, d and ntroducng the frt factor nto the product, we hae P 1 G p * d1,,, d p d r d, d (12) In the approxmaton of the ont probablty n the FDG and FORG approache, random ertce and random arc are treated eparately, for th reaon the aboe expreon can be plt conderng ertce and arc eparately a follow P n m n1 n n m m1 m G p* a1,, an, b1,, bm p a p b r a, a r a, b r b, b (13) Approxmaton of the ont probablty by FORG In the FORG approach, the Peleg coeffcent between ertce and between arc do not nfluence on the computaton of the ont probablty. That, by aumpton 1 and 2 (ecton 3.1), r a, a 1 and r b, b 1 for all the ertce and arc, repectely. On the contrary, aumpton 3 (ec 3.1) make that the probablty on the arc be condtoned on the alue of the ertce that the arc connect, q b a a 1, 2. In a mlar deducton to that of

15 ecton 4.3, and conderng aumpton 1, we arre at the equalence: q a a p r a, b r a, b b. Thu, 1, 2 b n m m p a p b r a,b P G , 2 (14) 6 Approxmaton of the ont probablty by FDG In the FDG approach, the 2 nd -order probablte between ertce can be etmated from the margnal probablte and the 2 nd -order relaton a follow (a mlar expreon obtaned for the arc, ee [25]), Pr a a 0 f Condton Q nd a a p a p a otherwe Pr 2 (15) where the Condton Q 2nd Q 2nd : A, a a O, O, a a E, a a a a (16) Note that, n the frt cae, t can be aured that the ont probablty null, but n the econd cae, we aume that the random element are ndependent and the probablty etmated a a product of the margnal one. Thu, the Peleg coeffcent are mplfed a r ' a 0 p r ', ung eq. (15), 0 f Q2 nd p a 0 a, a (17) 1 otherwe Moreoer, due to the ndependence aumpton 2 (ec 3.2), t not poble to hae a non-null arc and a null ertex a one of t endpont n an outcome graph. Thu, we hae p 0 and p In the other cae, by aumpton 1, they are aumed to be ndependent and

16 o computed a the product of the margnal one. The Peleg coeffcent between ertce and arc are mplfed a r " 0 f 2 a a, b b (18) 1 1 otherwe The fnal expreon of the ont probablty of an outcome AG wth repect to an FDG n m n n m m m p a p b r ' a, a r ' b, b r" a, b P G (19) , 2 7 Dtance meaure between AG and SORG The dtance meaure preented n th ecton prode a quanttate alue of the match between an AG G (data graph) and a SORG S (model graph) mlar to the one preented n [1]. It related to the probablty of G accordng to the labellng functon : G S, denoted P G n eq. (4). We may attempt to mnme a global cot meaure C of the morphm n the et H of allowable confguraton, by takng the cot a a monotonc decreang functon of the condtonal probablty of the data graph gen the labellng functon, C f PG. For ntance, C lnpg be a poble choce. Thu, conderng eq. (4), C 1 G ln p d, r d d would (20) Howeer, only that one graph element had a probablty of zero, the ont probablty would be zero and C would be nfnte. Snce th may happen

17 due to the noy preence of an unexpected element or the abence of a model element, only that one graph element were not properly mapped, the noled graph would be wrongly condered to be completely dfferent. We mut therefore admt the poblty of both extraneou and mng element n the data graph, nce the data extracted from the nformaton ource (e.g. mage) wll uually be noy, ncomplete or uncertan. A a conequence, the matche for whch G 0 P hould not be dcarded nce they could be the reult of a noy feature extracton and graph formaton. In addton, a model (SORG) hould match to a certan degree not only the obect (AG) n t learnng et but alo the one that are near. Hence, t more approprate for practcal purpoe to decompoe the global cot C nto the um of ome bounded nddual cot, one for each of the graph element matche (frt-order cot) and one for each Peleg compatblty coeffcent or par of element matche (econd-order cot) C 1 G Cp p d Crr, d, d (21) where frt- and econd-order cot are gen repectely by C p p Cot p d d (22) C r r, d, d Cot( p, 0 f p ( d ) K p ( d ) K ( d, d )) Cot( p ( d )) Cot( p ( d Pr Pr )) (23) otherwe

18 and the functon Cot Pr yeld a bounded normaled cot alue between 0 and 1 dependng on the negate logarthm of a gen probablty Pr and parametered by a pote contant K 0,1, whch a threhold on low Pr probablte that ntroduced to aod the cae ln 0, whch would ge negate nfnty. Th, ln(pr) f Pr KPr ln( KPr ) Cot Pr (24) 1 otherwe In the frt cae of equaton (23), both the ont probablty and at leat one of the margnal probablte are practcally zero, and a commented before, the ndecene 0/0 oled a 1 for the Peleg coeffcent, yeldng a null econd-order cot, nce ln(1)=0. Note that the global cot gen by equaton (21) not an edt operaton cot. Moreoer, econd-order cot may be pote or negate, thu correctng (f neceary) the um of frt-order cot and ung, to th end, the nformaton tored n the econd-order ont probablty functon. Once a cot meaure C defned, a dtance meaure between an AG and a SORG and the optmal labellng * are defned repectely a d mn CG and arg mn CG H * (25) H The algorthm we ue to calculate d and * a clacal recure tree earch procedure, where the earch pace reduced by a branch and bound technque (not decrbed here due to lack of pace)

19 8 Synthe of Second-Order Random Graph Below, we preent the Incremental-ynthe-of-SORG method (Algorthm 1) to ynthee an SORG from a equence of AG. The algorthm ue two procedure: SORG-ynthe-from-labelled-AG, to tranform an AG nto an equalent SORG, and SORG-ynthe-from-labelled-SORG to buld a SORG from two SORG wth a gen labellng. The ynthe method parametered by a matchng algorthm G F M, that uppoed to return an optmal (or a good uboptmal) labellng between an AG G and an SORG F, accordng to an approprate dtance meaure. In practce, we ue a algorthm G F M, the branch-and-bound method aforementoned that calculate the dtance meaure decrbed n the preou ecton. Algorthm 1: Incremental-ynthe-of-SORG Input: A equence of AG G 1,... G m, m 1, oer a common doman. A matchng algorthm G F optmal or ub-optmal labellng M, between an AG and an SORG that fnd an Output: An SORG F that repreent the gen et of AG. Begn F := SORG-ynthe-from-labelled-AG( G ) { buld the frt SORG from G 1 1 } for : 2 to m do let d : G, F and : G F be the dtance and labellng found by M G, F G', F', ' :=Extend-labellng-AG-SORG, F, G { It extend the AG and the SORG wth null element to make them tructurally omorphc and alo extend the gen labellng accordngly }

20 H := SORG-ynthe-from-labelled-AG( G ') { buld an auxlary SORG H from G '} let : G' H be a becte mappng ued n the preou ynthe let : H F' be the becte mappng determned by the compoton ' F := SORG-ynthe-from-labelled-SORG H, F', { buld F ung ynthe endfor end-algorthm from 2 SORG } The algorthm preented aboe mlar to the one decrbed n [26] for the ncremental ynthe of FDG. The only dfference wth the cae of yntheng FDG that, ntead of nferrng the FDG econd-order contrant, econd-order ont probablty denty functon mut be etmated now. To th end, t enough to modfy a follow the procedure that carry out the ynthe of a new model (now an SORG) from a et of AG (SORG-ynthe-from-labelled-AG) or from a et of preou model (SORG-ynthe-from-labelled-SORG) when a common labellng cheme gen [26]. g Let D G 1 g z be a et of AG defned oer a common attrbute doman. Aumng that a common labellng between all the AG n D gen, let g be the node labelled n the AG mnmum common order). g G' (the extenon of g G to a

21 The econd-order ont probablty denty functon of par of ertce P p ( a1, a2), 1 n, 1 n, can be etmated n the maxmum lkelhood ene ung frequence of attrbute and null alue n D a g g g g # g : 1 g z : ( ) a1 ( ) a2 p( a1,a2) Pr( a1 a2) (26) z If the SORG ynthe from a et of preou SORG F k 1 k h wth a gen common labellng, let where SORG t k z k h g 1 z g k t be a weght for each, 1 k h, k F gen by (27) k z the (tored) number of AG that wa ued to ynthee the k F. Then P p ( a1, a2), 1 n, 1 n, can be etmated from the correpondng probablty denty functon n the preou SORG a p h k k ( a1,a2) Pr( a1 a2) t p a1, a2 k 1 (28) The ont probablty functon of par of ertce and edge P ( a, b), 1 n, 1 m par of edge P q ( b, 2 ), 1 m, 1 m, and the ont probablty functon of 1 b can be etmated n both cae (et of AG or et of SORG) mlarly. 9 Reult We carred out three dfferent type of experment to ae the uefulne of our new repreentaton and to compare t wth ome other repreentaton preented n the lterature. In the frt experment, the AG were ynthetcally generated aryng ome parameter uch a the number of ertce or the dtance between the AG n ther cluter. In the econd

22 experment, we ued 3D-obect artfcally created by a CAD program. In the lat experment, we ued a real applcaton n whch AG repreent coloured 3D obect. They were extracted and recogned from ome 2D mage. We preent thee two applcaton on 3D-obect due to the fact that n the frt one, the 3D obect and the mage are le complex and there no egmentaton proce that dtort the obtaned AG and the run tme needed to compute the clafcaton. Thu, the frt experment are ueful to tudy our repreentaton from the theoretcal pont of ew, the econd one are ueful to apply our method on a 3D-obect non-noy repreentaton and the thrd one are ueful to apply the repreentaton on noy, real and complex mage. We preent the experment n the followng three ecton. In each experment, we compare SORG wth three other method: FDG, FORG and AG-to-AG matchng. Frt, we how ome nformaton of the AG and the tructure obtaned n the ynthe proce and then we how the run tme and rato of correctne of the clafcaton procee for each method. SORG, FDG and FORG were yntheed ung the dynamc cluterng n whch the model are ncrementally updated from a equence of AG that repreent the ame cluter or 3D-obect [26] (We ued the order of preentaton of AG that obtaned the bet reult). In the SORG method, AG were clafed ung the dtance meaure decrbed n th paper. In the FDG method, the AG were clafed applyng the dtance meaure

23 between AG and FDG relaxng econd-order contrant (moderate cot on the antagonm, extence and occurrence), wthout the effcent module, preented n [23,25]. FORG were compared ung the method preented n [25]. Fnally, n the drect AG-to-AG matchng method, we ued the edt-operaton dtance between AG preented n [20]. The algorthm preented here were mplemented n ual C++ and run on a Pentum IV (1.6Ghz). 9.1 Experment wth randomly generated AG The AG ued n th ecton were generated by the random graph generator proce hown n fgure 2 (th graph generator wa alo ued and explaned n depth n [26]). We frt generated 10 ntal AG randomly, one for each model, that had 15 ertce and 5 arc per ertex. From thee AG, the reference and tet et were dered n the followng way. For each ntal AG, a reference and a tet et of 10 AG wa bult by randomly deletng 3 ertce and replacng the attrbute of the other ertce by addng gauan noe wth arance V to the attrbute alue. Then, from each et of 10 reference AG, an FDG wa yntheed

24 Tet et: 100 element 1 2 AG1 AG AG1 AG10 10 AG10 Intal AG 1 Intal AG10 1 AG1 2 AG AG1 AG10 2 AG10 10 AG10 Reference et: 100 element FDG 1 FDG10 Fgure 2. Random generaton of reference and tet et and FDG ynthe. Fgure 3 how n (a) the rato of recognton correctne and n (b) the tme n econd pent to compute an AG clafcaton n aerage applyng 4 dfferent clafcaton method: SORG, FDG, FORG and drect AG-AG matchng. We hae een that the econd-order knowledge kept n the SORG hgher than n the FDG and than n the FORG. We ee, through the reult, that th knowledge ueful to repreent the cluter of AG and o to ncreae the recognton rato. The drect AG-AG matchng method hae mlar reult than SORG and FDGS only when there few noe n the tet et. When the arance of the noe ncreae, the AG n the tet et are ery dfferent from the AG n the reference et and then the rato of clafcaton decreae. Whle conderng the run tme, we ee that the hgher dfference appear when the arance of the noe large. FDG the fatet method nce the antagonm are ueful to prune the earch tree

25 (ee [25] for more detal). Neerthele, the Peleg coeffcent computed n the dtance between AG and SORG are alo ueful to prune the earch tree. For th reaon, SORG obtan better reult than FORG. Fnally, the drect AG to AG matchng the lowet method when the arance bgger than 0.6. Th due to the fact that the AG n the tet et are ery dfferent to thoe n the reference et and o the branch and bound algorthm can carcelly prune the earch tree. 1 0,9 0,8 0,7 0,6 Rato of Recognton Correctne 4 3,5 3 2,5 Recognton Run Tme n econd 0,5 0,4 0,3 0,2 0, ,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 2 Varance of Gauan Noe 2 1,5 1 0, ,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 2 Varance of Gauan Noe Fgure 3. (a) Rato of recognton correctne (b) run tme pent n the clafcaton. SORG: ; FDG: ; FORG: ; AG-AG: 9.2 Expermental aldaton ung ynthetc 3D obect In the econd expermental aldaton of our repreentaton, we degned fe obect by a CAD program (Fgure 4) and then, we took all the topologcally dfferent ew from thee obect (21 ew from the frt and econd obect and 12, 24 and 23 from the other three; n total, 101ew). Furthermore, we bult an AG from each ew n whch the ertce repreent the planar face (ther attrbute alue the actual face area) and the arc repreent the edge between face (ther attrbute alue the edge length)

26 Fgure 5 how the aerage number of ertce of the AG. From each et of AG that repreent one 3D-obect, we yntheed an FDG, thu, 5 FDG were bult. To bult the AG that compoed the tet et, we modfed the attrbute alue of the ertce and arc of the ntal 101 AG by addng ome Gauan noe wth arance V. The adantage of th controlled experment that the generated tructure repreent the 3D-obect wthout the uncertanty of the egmentaton proce. For ntance, n the FDG cae, an antagonm relaton between two ertce appear when thee element hae neer een together n the ame ew. And alo, an occurrence relaton appear when a ertex ble n all the ew n whch another one ble too. See [26] for more detal of the ynthetc data ued. Moreoer, there no tme pent on the egmentaton proce. Fgure dfferent ew extracted from the 5 obect created by a CAD program. Each obect repreented by the 2 mage of a column. The frt lne of ew are the more repreentate of the 3D-obect and the econd lne are the mplet ew

27 0,25 0,2 0,15 0,1 0, Dtrbuton of Number of Vertce extracted from obect Fgure 5. Rato of the number of ertce n aerage of the AG extracted from the 5 obect. AG hae from 1 to 9 ertce and the aerage 5. The reult obtaned on the rato of clafcaton and run tme are mlar than the one obtaned n the preou ecton (fgure 6). In th cae, we hae hown that SORG and FDG are ueful method to repreent 3Dobect although the extracted AG hae lot part of the three-dmenonal nformaton of the obect. Only when the arance hgher than 1.0, the clafcaton rato decreae dratcally. 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 Rato of Clafcaton Corrctne Varance of Guan Noe 2 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 Clafcaton Run Tme n Second 0 0,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 2 Varance of Gauan Noe Fgure 6. (a) Rato of recognton correctne (b) run tme pent n the clafcaton. SORG: ; FDG: ; FORG: ; AG-AG:

28 9.3 Applcaton of graph tructure to 3D obect recognton Fnally, we preent a real applcaton to recogne coloured obect ung 2D mage. Image were extracted from the databae COIL-100 from Columba Unerty ( oftlb/col-100.html). It compoed by 100 olated obect and for each obect there are 72 ew (one ew each 5 degree). AG are obtaned by the egmentaton proce preented n [8]. AG node repreent regon and ther attrbute alue ther aerage hue and arc repreent adacent regon and ther attrbute alue the dtance between aerage hue. Fgure 7 how the 20 obect at angle 100 and ther egmented mage wth the AG. Thee AG hae from 6 to 18 ertce and the aerage number 10 (fgure 8). The tet et wa compoed by 36 ew per obect (taken at the angle 0, 10, 20 and o on), wherea the reference et wa compoed by the 36 remanng ew (taken at the angle 5, 15, 25 and o on). We made 6 dfferent experment n whch the number of cluter that repreent each 3D-obect ared. If the 3D-obect wa repreented by only one cluter, the 36 AG from the reference et that repreent the 3D-obect were ued to ynthee the SORG, FORG or FDG. If t wa repreented by 2 cluter, the 18 frt and conecute AG from the reference et were ued to ynthee one of the SORG, FORG or FDG and the other 18 AG were ued to ynthee the other one. A mlar method wa ued for the other experment wth 3, 4, 6 and 9 cluter per 3D-obect

29 Fgure 7. The 20 elected obect at angle 100 and the egmented mage wth the AG. 0,3 0,25 0,2 0,15 0,1 0, Dtrb. of Number of Vertce extracted from ob Fgure 8. Rato of the number of ertce n aerage of the AG. Fgure 9.a how the rato of correctne of the four clafer aryng the number of cluter per each obect. When obect are repreented by only 1 or 2 cluter, there are too much purou regon (produced n the egmentaton proce) to keep the tructural and emantc knowledge of the obect. For th reaon, dfferent regon or face (or ertce n the AG) of dfferent ew (that, AG) are condered to be the ame face (or ertex n the AG). The bet reult appear when each obect repreented by 3 or

30 4 cluter, that, each cluter repreent 90 degree of the 3D-obect. When obect are repreented by 9 cluter, each cluter repreent 40 degree ew of the 3D-obect and 4 AG per cluter, there poor probabltc knowledge and therefore there a lack of dcrmnaton between obect. Fgure 9.b how the aerage run tme pent to compute the clafcaton. When the number of cluter per obect decreae, the number of total comparon alo decreae but the tme pent to compute the dtance ncreae nce the tructure that repreent the cluter (SORG, FORG or FDG) are bgger. 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 Rato of Clafcaton Corrctne Number of cluter for 3D-obect 5 4,5 4 3,5 3 2,5 2 1,5 1 0,5 0 Clafcaton Run Tme n Second Number of cluter for 3D-obect Fgure 9. (a) Rato of recognton correctne (b) run tme pent n the clafcaton. SORG: ; FDG: ; FORG: ; AG-AG: 10 Concluon and future work We hae preented SORG a a general formulaton of an approxmaton of the ont probablty of random element n a RG, that decrbe a et of AG, baed on 2 nd -order probablte and margnal one. We hae een that the FORG and FDG approache are two pecfc cae of SORG. In both cae, the margnal probablte of the random ertce and arc are

31 condered, but the dfference between them n how are the 2 nd -order relaton between ertce or arc etmated. FORG keep only the 2 nd -order probablty between arc and ther extreme ertce, nce the other ont probablte are etmated a a product of the margnal one. On the contrary, FDG keep only a qualtate and tructural nformaton of the 2 nd - order probablte between all the ertce and arc. If we compare both method, FORG hae local (arc and endpont ertex) 2 nd -order emantc knowledge of the et of AG but do not ue any 2 nd -order tructural nformaton of the et of the AG. FDG do not keep any 2 nd -order emantc nformaton but nclude the 2 nd -order tructural nformaton of all the et of AG. For th reaon, the torage pace of FORG ncreae to the quare on the ze of the random-element doman but the FDG ncreae to the quare on the number of ertce and arc. Howeer, the mot mportant mplcaton of the gen general formulaton of the 2 nd -order random graph repreentaton that t open the door to the deelopment of other probabltc graph approache, ether full 2 nd -order or not. In addton, t nteretng to tudy emprcally the relaton between the amount of data to be kept n the model and the recognton rato and run tme n eeral applcaton. That, to know n whch applcaton worthwhle to ue explctly the 2 nd -order probablte or enough to etmate them by other way le cotly n pace requrement, uch a FORG and FDG

32 Reference 1. R. Alquézar, F. Serratoa, A. Sanfelu, Dtance between Attrbuted Graph and Functon-Decrbed Graph relaxng 2 nd order retrcton. Proc. SSPR 2000 and SPR 2000, Alcante, Span, Sprnger LNCS-1876, pp , S.Berrett, A.Del Bmbo & E.Vcaro, Effcent matchng and ndexng of graph model n content-baed retreal, IEEE Tran. on PAMI, Vol. 23, No. 10, pp: , H. Bunke, Error-tolerant graph matchng: a formal framework and algorthm. Proc. Workhop SSPR 98 & SPR 98, Sydney, Autrala, Sprnger LNCS-1451, pp.1-14, H.Bunke & G.Allerman, Inexact graph matchng for tructural pattern recognton, Pattern Recognton Letter, 1 (4), pp: , V.Canton et al., 2D obect recognton by multcale tree matchng, Pattern Recognton, 31, pp: , L.P.Cordella, P.Fogga, C.Sanone & M.Vento, Learnng tructural hape decrpton from example, Pattern Recognton Letter, Vol. 23, pp: , W.J. Chrtma, J. Kttler and M. Petrou, Structural matchng n computer on ung probabltc relaxaton, IEEE Tranacton on PAMI, ol. 17, pp ,

33 8. P.F. Felzenwalb and D.P. Huttenlocher, Image Segmentaton Ung Local Varaton, Proc. of the IEEE Computer Soc. Conf. on Computer Von and Pattern Recognton, pp , S.Günter & H.Bunke, Self-organzng map for cluterng n the graph doman, Pattern Recognton Letter, Vol. 23, pp: , B.Huet & E.R.Hancock, Relatonal obect recognton from large tructural lbrare, Pattern Recognton, Vol. 35, pp: , T. Kohonen & P.Someruo, Self-Organng Map on ymbol trng, Neurocomputng 21, pp: 19-30, X.Jang, A.Münger and H. Bunke, On medan graph: Properte, algorthm and applcaton, IEEE Tran. on Pattern Analy and Machne Intellgence, Vol. 23, No. 10, pp: , J. Lladó, E. Martí, J.J. Vllanuea, Symbol Recognton by Error- Tolerant Subgraph Matchng between Regon Adacency Graph, IEEE Tran. on Pattern Analy and Machne Intellgence, Vol. 23, No. 10, pp: , B.T. Memer & H. Bunke, A new algorthm for error-tolerant ubgraph omorphm detecton, IEEE Tran. on Pattern Analy and Machne Intellgence, Vol. 20, pp: , R.Myer, R. Wlon & E.Hancock, Bayean graph edt dtance, IEEE Tran. on PAMI, Vol. 22, pp: ,

34 16. S. Peleg and A. Roenfeld, Determnng compatblty coeffcent for cure enchancement relaxaton procee, IEEE Tranacton on Sytem, Man and Cybernetc, ol. 8, pp , M.Pelllo, K.Sddq & S.Zucker, Matchng herarchcal tructure ung aocated graph, IEEE Tran. on Pattern Analy and Machne Intellgence, Vol. 21, pp: , J.Rocha & T.Pald, A hape analy model wth applcaton to a character recognton ytem, IEEE Tran. on Pattern Analy and Machne Intellgence, pp: , A. Sanfelu, R. Alquézar, J. Andrade, J.Clment, F. Serratoa & J.Vergé, Graph-baed Repreentaton and Technque for Image Proceng and Image Analy, Pattern Recognton, ol. 35, pp: , A. Sanfelu and K. Fu, A dtance meaure between attrbuted relatonal graph for pattern recognton, IEEE Tranacton on Sytem, Man and Cybernetc, ol. 13, pp , K.Sengupta & K.L.Boyer, Organzng large tructural model bae, IEEE Tran. on PAMI, Vol. 17, No. 4, pp: , D.S. Seong, H.S. Km & K.H. Park, Incremental Cluterng of Attrbuted Graph, IEEE Tranacton on Sytem, Man and Cybernetc, ol. 23, pp ,

35 23. F. Serratoa, R. Alquezar & A. Sanfelu, Effcent algorthm for matchng attrbuted graph and functon-decrbed graph, n Proceedng ICPR 2000, 15 th Int. Conf. on Pattern Recognton, Barcelona, Span, Vol.2, pp , F. Serratoa, R. Alquézar & A. Sanfelu, Etmatng the Jont Probablty Dtrbuton of Random Vertce and Arc by mean of Second-order Random Graph, Proc. Syntactc and Structural Pattern Recognton, SSPR 2002, LNCS 2396, pp: , F. Serratoa, R. Alquézar & A. Sanfelu, Functon-decrbed graph for modelng obect repreented by attrbuted graph, Pattern Recognton, 36 (3), pp , F. Serratoa, R. Alquézar & A. Sanfelu, Synthe of Functon- Decrbed Graph and cluterng of Attrbuted Graph, Internatonal Journal of Pattern Recognton and Artfcal Intellgence, Vol. 16, No. 6, pp , K.Shearer, S.Venkateh & H.Bunke, Vdeo ndexng and mlarty retreal by larget common ubgraph detecton ung decon tree, Pattern Recognton, 34 (5), pp: , W.D.Wall, P.Shoubrdge, M.Kraetz & D.Ray, Graph dtance ung graph unon, Pattern Recognton Letter, Vol. 22, pp: , E.K. Wong, Model matchng n robot on by ubgraph omorphm Pattern Recognton, Vol. 25, pp: ,

36 30. A.K.C. Wong & M. You, Entropy and dtance of random graph wth applcaton to tructural pattern recognton, IEEE Tran. on PAMI, ol. 7, pp ,