T-norms and t-conorms in multilayer perceptrons

Transcription

1 T-orms ad t-coorms multlayer perceptros C.J. Matas Dept. o Computer Scece ad Artcal Itellgece Uversty o Graada cmatas@decsa.ugr.es Abstract The deto o t-orms ad t-coorms o the class o Hamacher wth multlayer eedorward artcal eural etworks s acheved ths work. Ths act lets to sert uzzy kowledge to eural etwork beore ts trag. Keywords: Neural Networks, Fuzzy Logc, Lgustc Hedges. Itroducto Artcal Neural Networks (ANN s) [5, 9, ] are computatoal models that have exhbted excellet behavor classcato ad approxmato problems. They are terestg due to the trag startg rom put-output examples ad ts geeralzato capacty. However, ANN s have the shortcomg o beg black boxes. Ths mples two restrctos whe they are used or solvg a problem: It s ot possble to sert kowledge to ANN s. It s ot possble to uderstad how a traed ANN solves a problem. I ths paper, we are gog to sert uzzy kowledge to multlayer eedorward ANN s. I order to acheve ths am, we eed: To buld membershp ucto wth ANN s. To dee a t-orm (uzzy tersecto) ad a t-coorm (uzzy uo) [7] wth euros. To use lgustc hedges our kowledge. These tasks wll be solved ths paper. Beore t, a troducto to multlayer eedorward artcal eural etwork s preseted. 2 Artcal Neural Networks Multlayer eedorward ANN s are the most commo model o eural ets. Let us cosder a ANN wth put euros (x,..., x ), h hdde euros (z,..., z h ), ad m output euros (y,..., y m ). The ucto the et calculates s: y k = g F :R R m ; F(x,..., x )= (y,..., y m ) A k j= ( ( z β ) + ϕ ), z = ( ( x w ) + τ ) j jk k j A = where g A ad A are actvato uctos: g A s usually mplemeted as g A ( x) = x. A s usually a o-lear, atsymmetrc, mootoc, ad lmted ucto [5]. Ote, t s a geeralzed sgmod [0]: A ( x) x ρ Fm e + Fmax = x + x ρ ρ e + e e + x ρ whch cocdes wth ether the sgmod, or F m =0, F max =, ρ=2, or wth the hyperbolc taget, or F m =, F max =, ρ=, or wth the logc threshold, or ρ 0. The parameter ρ s versely proportoal to the steepess o the actvato ucto the org. 3 Costructo o membershp uctos rom euros I ths secto, t s preseted how to costruct membershp uctos or represetg three kds o lgustc propostos: a) x s about greater tha j j 33

2 K, b) x s about lower tha K ad c) x s approxmately K. The kd a) ad b) are drectly represeted usg a euro that uses the sgmod actvato ucto: a) µ s about greater tha K (x) = sgm( (x-k) w + + 8) (Fg. ), ad b) µ s about lower tha K (x) = sgm( (-x+k) w + + 8) (Fg. 2), where the parameter w + >0.0 determes the magtude o the slope o the membershp ucto (the hgher w + s, the hgher magtude s). The magtude o slope o the membershp ucto s practcally (w + /6). where: The parameter w + >0.0 determes the wdth o the membershp ucto (as the w + creases, the wdth decreases). The wdth s practcally equal to (2 (6/w + )). Fgure 3: x s approxmately K wth K=2 ad w + =8. 4 Deto o t-orms ad t-coorms rom euros Fgure : x s about greater tha K wth K=2 ad w + =8. T-orms ad t-coorms are bary operators appled o membershp degrees belogg to [0,]. These operators mplemet the tersecto o uzzy sets (torms) ad the uo o uzzy sets (t-coorm). I ths secto, t s preseted the costructo o t- orms ad t-coorms by meas o ANN s. Frst, we eed to dee a operator Θ accordg to the ollowg property: A( x j A A j + x ) = ( x ) Θ ( x ) () where x, x j R. Ths mples that: Fgure 2: x s about lower tha K wth K=2 ad w + =8. The basc dea or costructg membershp uctos that represet the lgustc proposto x s approxmately K startg rom sgmod uctos was rst suggested by Lapedes ad Farber [8]. It cossts o takg the derece betwee two parallel-dsplaced sgmod uctos [3]. That s, µ s approxmately K (x) = sgm( (x-k) w + +8) - sgm( (x- K) w + -8) (Fg. 3), Deto F aθb= m a Fmax) b Fmax) + Fmax Fm a) F ( F a) F b) + ( a F ) b F ) m wth a,b (F m,f max ). m max max m b) It ca be oted that the deto o the operator Θ oly depeds o the parameters F m ad F max. Ths operator s depedet o the parameter ρ ucto A. So, t wll be wrtte as Θ (Fm, Fmax) the rest o ths paper. 34

3 T-orms ad t-coorms o the class o Hamacher [4, 6] ca be deed rom ths geerc operator Θ (Fm, Fmax): a) T-orm o the class o Hamacher I F m =0 ad F max =2 the operator Θ (Fm,Fmax) s equal to: a b a Θ (0,2) b = wth a,b (0,2). + ( a) b) Ths expresso ca be geerated usg the ollowg actvato ucto () to mplemet A : A ( x) = ( x) = 2 sgm(2 x). Besdes, a,b (0,] the operator Θ (0,2) s a t- orm o the class o Hamacher [4, 6], that s, a,b (0,] a Θ (0,2) b a AND b Ths property s acheved whe the puts x, x j equato () are egatve. I order to acheve a uzzy tersecto wth euros, we must dee the ucto where: orm ( u ) = (4 ( u )), = = u [0,], =,,. orm ca be mplemeted wth the sgmod ucto, so: 2 sgm(2 4 ( u ) = (4 orm = = = ( u )) = 2 sgm( ( u )) = = (8 u ) 8 ). orm (0) = (4-)) = 2 sgm(-8) = 0.0, ad orm () = (4 0) = 2 sgm(0) =.0 (these two codtos wll allow to cosder ths ucto lke a lgustc hedge). Startg rom ths ucto orm ad the membershp uctos o the uzzy sets A ad B (µ A (x) ad µ B (x)), we ca obta: orm (µ A (x) + µ B (x)) = ( 4 (µ A (x)-) + (µ B (x)-) ) ) = (4µ A (x)-) + 4µ B (x)-) ). As 4µ A (x)-) ad 4µ B (x)-) are egatve, we have (4µ A (x)-) + 4µ B (x)-)) = (4µ A (x)-)) AND (4µ B (x)-)) = orm (µ A (x)) AND orm (µ B (x)). Hece, the uzzy sets A ad B are moded by the ucto orm, a uzzy tersecto o these uzzy sets s acheved. I the ext secto, t s explaed as the ucto orm ca be cosdered a lgustc hedge. b) T-coorm o the class o Hamacher I F m = ad F max = the operator Θ (Fm,Fmax) s equal to: a + b a Θ(,) b = wth a,b (-,). + a b Ths expresso ca be geerated usg the hyperbolc taget ucto (tah) to mplemet A ( A (x)=tah(x)). Besdes, a,b [0,) the operator Θ (-,) s a t- coorm o the class o Hamacher [4, 6], that s, a,b [0,) a Θ (-,) b a OR b. Ths property s acheved whe the puts x, x j equato () are postve. I order to acheve a uzzy uo wth euros, we must dee the ucto where: u [0,], coorm (u) = tah (4 u), coorm ca be mplemeted wth the sgmod ucto so: coorm (u) = tah (4 u) = 2 sgm(8 u), coorm (0) = tah(0) = 0 ad coorm () = tah(4).0 35

4 (these two codtos wll allow to cosder ths ucto lke a lgustc hedge). Startg rom ths ucto coorm ad the membershp uctos o the uzzy sets A ad B (µ A (x) ad µ B (x)), we ca obta: coorm (µ A (x) + µ B (x)) = tah(4µ A (x) + µ B (x))) = tah(4 µ A (x) + 4 µ B (x)). As 4 µ A (x) ad 4 µ B (x) are postve, we have tah(4 µ A (x) + 4 µ B (x)) = tah(4 µ A (x)) OR tah(4 µ B (x)) = coorm (µ A (x)) OR coorm (µ B (x)). Hece, the uzzy sets A ad B are moded by the ucto coorm, a uzzy uo o these uzzy sets s acheved. I the ext secto, t s explaed as the ucto coorm ca be cosdered a lgustc hedge. 5 Lgustc hedges or sertg kowledge to a ANN I ths secto, the uctos orm ad coorm are preseted as lgustc hedges. a) orm as lgustc hedge I Fg. 4, the lgustc hedge o type cocetrato (µ A (x)) 5 ad the ucto orm are llustrated [0,]. We ca see that the ucto orm [0,] s very smlar to the lgustc hedge (µ A (x)) 5. Thereore, the ucto orm appled o the uzzy set A ca be cosdered a lgustc hedge o type cocetrato. It ca be deomated as x s extremely A. Fgure 4: a) Lgustc hedge (µ A (x)) 5. b) Fucto orm. b) coorm as lgustc hedge O the other had, Fg. 5, the lgustc hedge o type dlato (µ A (x)) 0.25, the ucto coorm ad the expresso -(-x) 5 are llustrated [0,]. We ca see that the ucto coorm s a lgustc hedge o type dlato ad t s very smlar to the expresso -(-x) 5. The ma derece wth the stadard lgustc hedges o type dlato les that the ucto coorm appled o a uzzy set A creases the magtude o the grade o membershp o x A, whch s relatvely small or those x wth a low grade o membershp A (hgh grade stadard hedges), ad relatvely large or those x wth hgh membershp (low membershp stadard hedges). Thereore, the ucto coorm appled o the uzzy set A ca be cosdered a lgustc hedge o type dlato. It ca be deomated x s somewhat A. Fgure 5: a) Lgustc hedge (µ A (x)) b) Fucto coorm. c) Expresso -(-x) 5. 6 Isertg uzzy kowledge to ANN s Multlayer eedorward ANN s are addtve uzzy systems (see [, 2]). Thereore, we wat to sert uzzy kowledge to a ANN, ths kowledge must be represeted wth the ormat o a addtve uzzy system. The prevously preseted t-orm, t-coorm ad membershp uctos ca be used ths addtve uzzy system. 7 Examples I ths secto, we preset a example where uzzy kowledge s serted to a multlayer eedorward ANN. Ths kowledge s expressed usg two put varables (servce ad ood) ad oe output varable (tp). These varables have the ollowg lgustc values: a) : poor, average ad good (Fg. 6). b) : racd, ormal ad delcous (Fg. 7). 36

5 c) Tp: cheap (=5), average (=5) ad geerous (=25) Poor Average Good Fgure 6: Lgustc values poor, average ad good o the varable servce. R * 3 : I ( s somewhat good) OR ( s somewhat delcous) the (Tp s geerous (=25)). The membershp degree o the proposto s poor OR s racd (atecedet o the rule R ) s llustrated Fg. 8.a together wth the degree o the proposto s somewhat poor OR s somewhat racd (atecedet o the rule R * ) (Fg. 8.b). We ca see that the ma derece s motvated by the dlato produced by the lgustc hedge somewhat. A smlar reasog ca be carred out wth the atecedets o the rules R 3 ad R * 3. Racd Normal Delcous 0.5 a) Fgure 7: Lgustc values racd, ormal ad delcous o the varable ood. The addtve uzzy rules that cota the kowledge are: R : I ( s poor) OR ( s racd) the (Tp s cheap (=5)). R 2 : I ( s average) AND ( s ormal) the (Tp s average (=5)). R 3 : I ( s good) OR ( s delcous) the (Tp s geerous (=25)). These uzzy rules use t-orms ad t-coorms the atecedets. These operators are mplemeted wth the stadard operatos algebrac product (a b=a b) ad algebrac sum (a b=a+b-a b) [7], respectvely. I order to sert ths kowledge to a ANN we have to alter the uzzy propostos wth lgustc hedges ad to use t-orms ad t-coorms o the class o Hamacher or mplemetg the logcal coectves the atecedets. The uzzy rules are trasormed to: R * : I ( s somewhat poor) OR ( s somewhat racd) the (Tp s cheap (=5)). R * 2 : I ( s extremely average) AND ( s extremely ormal) the (Tp s average (=5)). Fgure 8: a) s poor OR s racd, b) s somewhat poor OR s somewhat racd. The membershp degree o the proposto s average AND s ormal (atecedet o the rule R 2 ) s llustrated Fg. 9.a together wth the degree o the proposto s extremely average AND s extremely ormal (atecedet o the rule R * 2 ) (Fg. 9.b). We ca observe that the ma derece s motvated by the cocetrato produced by the lgustc hedge extremely. a) Fgure 9: a) s average AND s ormal, b) s extremely average AND s extremely ormal. Fally, t s terestg to compare the output space produced by the addtve rules R, R 2 ad R 3 (Fg. 0) wth the output space produced by the ANN that cotas the rules R *, R * 2 ad R * 3 (Fg. ). We ca ote that these output spaces are eough smlar. I ths way, we have attaed the mplemetato o a 37

6 uzzy rule based system usg ANN s, that s, t has bee acheved the serto o uzzy kowledge to multlayer eedorwar ANN s. Fgure 0: Output space o the addtve uzzy system composed by the rules R, R 2 ad R 3. Fgure : Output space produced by the ANN that cotas the rules R *, R * 2 ad R * 3. 8 Coclusos A method or sertg uzzy kowledge to ANN s has bee preseted. I order to dee t, t was ecessary to expla the costructo o membershp uctos, t-orms ad t-coorms by meas o ANN s. Ths costructo was possble thaks to the use o lgustc hedges. I ths maer, t s possble to sert kowledge provded by a expert to a ANN beore ts trag. Trasactos o Neural Networks, vol.8, Nº5, pp , 997. [2] J.L.Castro, C.J. Matas & J.M. Beítez, Iterpretato o artcal eural etworks by meas o uzzy rules, IEEE Trasactos o Neural Networks, vol. 3,, pp. 0-6, [3] S. Geva, K. Malmstrom & J. Stte, Local cluster eural et: archtecture, trag ad applcatos, Neurocomputg, 20, pp , 998. [4] H. Hamacher, Über logsche Aggregatoe cht-bär explzerter Etschedugs-krtere, Rta G. Fscher Verlag, Frakurt, 978. [5] S. Hayk, Neural Networks: A Comprehesve Foudato, Mc Mlla College Publshg Compay, New York, 994. [6] P. Klemet, R. Mesar, E. Pap, O the relatoshp o assocatve compesatory operators to tragular orms ad coorms, Iteratoal Joural o Ucertaty, Fuzzess ad Kowledgebased Systems, 4, (2), pp , 996. [7] G.J. Klr & B. Yua, Fuzzy sets ad uzzy logc: Theory ad Applcatos, Pretce Hall, 995. [8] A. Lapedes & R. Farber, How eural ets work, : D.Z. Aderso (Ed.), Neural ormato processg systems, Amerca Physcal Socety, New York, 988. [9] R.P. Lppma, "A Itroducto to Computg wth Neural Nets", IEEE ASSP Magaze, pp.4-22, Aprl, 987. [0] L.M. Reyer, Ucato o Neural ad Wavelet Networks ad Fuzzy Systems, IEEE Trasactos o Neural Networks, vo. 0, 4, 999. [] P.D. Wasserma, Advaced Methods Neural Computg, Va Nostrad Rehold, 5, Fth Aveue, New York, NY, 0003, 993. Ackowledgmets Ths work has bee supported by the CICYT Project TIC Reereces [] J.M. Beítez, J.L. Castro & I. Requea, Are Artcal Neural Networks Black Boxes?, IEEE 38