Modular dynamic RBF neural network for face recognition

Transcription

1 Edih Cowan Univrsiy Rsarch Onlin ECU ublicaions 0 0 odular dynaic RBF nural nwork for fac rcogniion Su Inn Ch'Ng Kah hooi Sng Li-inn Ang Edih Cowan Univrsiy 009/ICOS his aricl was originally publishd as: Ch'Ng, S, Sng, K, & Ang, L K (0 odular dynaic RBF nural nwork for fac rcogniion rocdings of IEEE Confrnc on Opn Syss (pp -6 Kuala Lupur, alaysia IEEE Original aricl availabl hr 0 IEEE rsonal us of his arial is prid rission fro IEEE us b obaind for all ohr uss, in any currn or fuur dia, including rprining/rpublishing his arial for advrising or prooional purposs, craing nw collciv works, for rsal or rdisribuion o srvrs or liss, or rus of any copyrighd coponn of his work in ohr works his Confrnc rocding is posd a Rsarch Onlin hp://rocuduau/cuworks0/63

2 odular Dynaic RBF Nural Nwork for Fac Rcogniion Su Inn Ch ng, Kah hooi Sng Dparn of Copur Scinc & Nworkd Syss Sunway Univrsiy Slangor, alaysia Li-inn Ang School of Enginring Edih Cowan Univrsiy Ausralia Absrac Ovr h yars, w hav sn an incras in h us of RBF nural nworks for h ask of fac rcogniion Howvr, h us of scond orr algorihs as h larning algorih for all h adjusabl parars in such nworks ar rar du o h high copuaional coplxiy of h calculaion of h acobian and Hssian arix Hnc, in his papr, w propos a odular srucural raining archicur o adap h Lvnbrg- arquard basd RBF nural nwork for h applicaion of fac rcogniion In addiion o h proposal of h odular srucural raining archicur, w hav also invsigad h us of diffrn fron-nd procssors o rduc h dinsion siz of h faur vcors prior o is applicaion o h L-basd RBF nural nwork h invsigaiv sudy was don on hr sandard fac daabass; ORL, Yal and AR daabass Kywords- odular srucur, RBF nural nworks, Lvnbrg-arquard algorih, fac rcogniion I INRODUCION h us of radial basis funcion (RBF nural nwork for fac rcogniion has bn wily xplord ovr h pas ca wih proising rsuls [-8] os of hs works ar oivad by h findings of h work by oo al in [] ha sudy h us and adapaion of h RBF nural nwork for h sall sapl high dinsional probl of fac rcogniion h auhors in [] usd rincipal Coponn Analysis (CA[9] and Fishr Linar Discriinan (FLD [0] o rduc h dinsion of h fac iags and xrac h discriinan faurs Subsqunly, h parns ar classifid using an RBF nural nwork larnd using a hybrid algorih; Linar Las Squars (LLS and Gradin Dscn (GD LLS is usd o larn h wighs of h nwork whil GD is usd o larn h adjusabl parars; widh and cnrs, of h RBF unis oda, h hod of srucur rinaion ployd by hs auhors is sill wily pracicd in h RBF sign of so of h rcnly vlopd fac rcogniion syss [, 3, 6-8] For xapl, h work by [7], uss h srucur rinaion hod by [], wih rgularizd orhogonal las squar (ROLS o cra an RBF nural nwork wih incrnal larning capabiliis for h ask of fac rcogniion On h ohr hand, h work by [, 3, 6, 8] uss diffrn faur xracion/pr-procssing chniqus prior o applying h daa o h RBF nural nwork signd using h sa concp spcifid in [] Copard o firs orr algorihs, scond orr algorihs ar advanagous as hy ar uch fasr [, ] and can b an ffciv soluion for probls wih up o hundrds of parars [3] hus, insad of using wo spara algorihs g LLS for wighs and GD for widh and cnrs, h scond-orr algorih can b horically usd o larn all h adjusabl parars in h RBF nural nwork his is vin in h rcn work by [, 4] which uss diffrn iprovd hods of copuaion o rin h acobian arics in h Lvnbrg-arquard (L algorih and subsqunly us h iprovd L algorih o rin all h adjusabl parars in a singl fd-forward nural nwork Alhough h abov-niond approachs wr sd o b ffciv on h wo coonly usd bnchark ss; funcion approxiaion and wo-spiral classificaion, h applicaion of such hods o high dinsional uli-class larning has no bn xplord o h bs of h auhors knowldg, i is nod ha h us L algorih for h larning of h RBF nural nworks signd for h ask of fac rcogniion is scarc his could b du o h fac ha h high copuaional coplxiy incurrd by h larning algorih islf vnually far ouwighs any advanags offrd by h algorih rsuling in os rsarchrs oping for a lss copuaionally anding algorih In his papr, h odifid L algorih o upda all h adjusabl parars in h RBF nural nwork is firs prsnd his is followd by h proposal of h odular dynaic L-basd RBF nural nwork, which is basd on h nwork consrucion hod prsnd in [] hn, h proposd odular dynaic L-basd RBF (D-RBF is coupld wih svral fron-nd procssing hods o rin which hod coplns h prforanc of h proposd D-RBF nural nwork h conribuion of his papr is wo-fold; h proposal of h D-RBF nural nwork for high dinsional uli-class larning and prforanc analysis of a cobinaion of diffrn fron-nd procssing hods wih h D-RBF nural nwork

3 X, X, X,N Y Y Y W W W 3 Figur Archicur of RBF nural nwork h srucur of h papr is as follows: In Scion, a brif rviw of h RBF nural nwork archicur is prsnd his is followd by h prsnaion of h us of L algorih o upda all h adjusabl parars for RBF nural nworks In Scion 4, a brif analysis of h issus facd by high dinsional uli-class larning for L-basd RBF nural nwork is prsnd A scripion of h proposd odular srucural raining archicur o adap h L-basd RBF nural nwork for high dinsional uli-class larning is prsnd in Scion 5 Scion 6 scribs h gnral hod of applicaion of h proposd D-RBF nural nwork for fac rcogniion h siulaion rsuls obaind ar prsnd and discussd in Scion 7 Finally, h papr is conclud in Scion 8 II RBF NEURAL NEWORK ARCHIECURE h RBF nural nwork consiss of hr layrs; inpu layr, hidn layr and oupu layr Fig shows h archicur of a ypical RBF nural nwork wih N inpus, RBF unis and S oupus h daa s is nod by X whr nos h oal nubr of parns h oupu of h RBF nwork wih Gaussian funcion can b calculad as follows: Z Y ( X W, w0 N ps Y p s + ( p X p ( X C xp ( whr W, s no h wighs bwn h -h RBF uni and S-h oupu, w 0 h bias, C h cnr and h widh No ha rprsns h Euclian nor Z Z Z III ODIFIED L ALGORIH FOR RBF NEURAL NEWORK h upda rul of h L algorih coonly usd o upda h wighs of arificial nural nworks is provid in (3 [] w + w ( w + δ + μi whr δ ( + μi whr is h acobian arix, I is h iniy arix, µ is h larning ra and is h rror vcor conaining h oupu rrors for ach inpu vcor usd on raining h nwork In h cas of updaing h wighs, h acobian arix is calculad basd on h firs-orr parial rivaivs of h nwork rror wih rspc o h wighs h acobian arix in such cass for nubr of parns, S nubr of oupus and NW oal nubr of wighs rsuls in a acobian arix of siz ( S NW and is ahaically rprsnd as: w wnw w wnw S S S w wnw w wnw w w NW w w wnw o odify h upda rul in (3 o upda all h parars, w incorpora h cnrs and widh ino h upda rul by concanaing all h parars ino a singl vcor, R, and subsiuing i ino h original quaion h foraion of vcor R can b rprsnd by (5 [] (4 (3

4 R W C ] (5 [, Now, h nw upda rul is as follows: R + R ( + μi Whr R + rprsns h updad parars, rprsns h acobian arix, μ h larning parar and h rror vcor find in (8 For h original algorih, sinc h upda rul is usd for solly rining h wighs, hnc h acobian arix is ford by diffrniaing h rror wih rspc o wighs, rfr o (4 In h currn cas, sinc h cnrs and widh ar incorporad ino h upda rul, h acobian arix is ford by diffrniaing h rror wih rspc o wighs, s, cnrs N, and widh rspcivly and concanaing h rsuls oghr nw d d S s Z N,, S d Y ( X (6 (7 (8, (9 N,, S ( X ( X, C, S N, w, SY N w, S Y, S, S N, ( X ( X, C N N, d (0 ( whr is h rror vcor and d is h sird oupu a nwork oupu S for raining parn h calculaion of h acobian lns can b calculad using h diffrnial chain rul which rsuls in (9-( Wih h addiion of h diffrnial of h cnr and widh o h original acobian arix, h siz of h nw acobian arix is now largr h siz of h nw acobian arix is rprsnd by (, whr rprsns h nubr of parns, S h nubr of oupus, h nubr of nurons and N h nubr of inpus o rov h nd of soring h nir acobian arix during ach upda, h concp of calculaing h Hssian arix and gradin in [5] is usd whrby only h rows of h nw acobian arix nds o b calculad and sord h Hssian arix, Q, is hn ford by suing h rsul of uliplicaion bwn h acobian rows, j, wih is ransposd, j ; rfr o (3 On h ohr hand, h gradin, g, is obaind by suing and uliplying h acobian row wih h rror vcor,, rfr o (4 Siz( nw ( S [( S + N + ] Q g ( j j j (3 (4 IV L ALGORIH FOR HIGH DIENSIONAL ULI- CLASS LEARNING For ach iraion in h L algorih, h invrsion of h Hssian arix, H, is rquird his bcos a probl for high dinsional uli-class larning using L-basd RBF nural nworks bcaus wih h us of h L algorih o upda all h adjusabl parars, h forulaion of h Hssian arix is now pnn on h siz of h inpus and oupus Dnoing h nubr of inpus as NI, oal nubr of oupus as NO and h nubr of nurons as ; h siz of h Hssian arix can b rprsnd as follows Siz (Η (ΝΙ + + ΝΟ Μ (5 Hnc, for h high dinsional uli-class larning probl, NI and NO which pnds on h siz of faur vcor and nubr of classs o b classifid rspcivly, will inviably incras rsuling in h invrsion of a largr arix during ach upda of h larning procss o rduc h copuaional coplxiy rlad o h invrsion of h Hssian arix during ach iraion, sps can b akn o rduc boh h siz of faur vcors and h nubr of oupus h lar is a uch rickir probl bcaus unlik h siz of h faur vcor which can asily b rducd hrough h us of pr-procssing or dinsion rducion chniqus, h nubr of classs o b classifid canno b arbirarily rducd jus o rduc h nubr of oupus In his papr, w propos h us of a odular raining srucur o iiga h probl of rducing h nubr of oupus usd by h nwork srucur h proposd odular srucural raining archicur is prsnd in h following scion

5 X X X N Inpu Daa RBF odul RBF odul RBF odul C Z Z Z C Figur D-RBF nwork archicur Cobining uni Z ou V ROOSED ODULAR DYNAIC RBF NEURAL NEWORK (D-RBF h archicur of h proposd odular raining srucur for h dynaic L-basd RBF nural nwork (D-RBF is picd in Fig h srucur of h proposd D-RBF nural nwork is siilar o ha of h convnional RBF nural nwork shown in Fig xcp ha h raining of ach class is don in oduls rsuling in h classificaion of only on class a a i hn, h individual oupus of ach odul ar concanad oghr o for h final oupu of h D-RBF nural nwork, Z ou hus, for h raining of an arbirary nubr of classs, h nubr of inpus for ach RBF odul is quals o ha of h nubr of faurs (i h dinsion of h inpu spac bu ach odul only has on oupu and h oal nubr of oduls in h nwork is quals o h oal nubr of classs o b classifid For xapl, noing h inpu daa as X N for nubr of parns and N nubr of inpus, h classificaion of C classs using D-RBF nural nwork will rquir C nubr of oduls h oupu of h C-h odul, Z C, is givn as follows: Z Y C Y + ( X ( X W w0 X, N C (6 xp (7 Z ou [Z Z Z 3 Z C ] (8 whr h oal nubr of nurons, W h wighs bwn h -h nuron and h singl oupu, w 0 h bias, C h cnr and h widh No ha rprsns h Euclian nor h growh procss of h dynaic RBF nural nwork prsnd in [] is usd for h nwork consrucion in ach RBF odul whil h larning algorih prsnd in Scion III is usd for h raining of ach of h RBF odul h ails of h growh procss of h nwork will no b scribd in his papr hrough h us of h proposd odular srucural raining, h siz of h Hssian o b invrd during ach iraion for ach RBF odul will now b only pnn on h siz of h faur vcors, which subsqunly ranslas o h nubr of inpus rquird (NI, rfr o quaion (9 Siz (H (NI + (9 VI ALICAION O FACE RECOGNIION h us of a odular srucur is abl o rduc h siz of h Hssian arix and hnc h ovrall copuaion of h raining of such L-basd RBF nworks o b inpnn of h nubr of oupus Nonhlss, h copuaion of h Hssian arix of h odular dynaic RBF nural nwork is sill pnn on h siz of faur vcors (nubr of inpus his aspc can b rsolvd by h applicaion of dinsion rducion hods or faur slcion chniqus o nsur ha only h os significan faurs ar applid o h D-RBF nural nwork for classificaion h gnral block diagra of a fac rcogniion xpr using h proposd D-RBF nural nwork as a classifir is picd in Fig 3 Figur 3 Gnral block diagra of a fac rcogniion xpr using h proposd D-RBF nural nwork h fron-nd procssor block s can b rplacd wih any faur xracor or pr-procssing chniqus VII RESULS AND DISCUSSION In his scion, h us of uliband curvl chniqu [6], CA+LDA[], p faurs[7] and block-basd prprocssing [8] as h fron-nd procssor is invsigad o rin h yp of fron-nd procssor ha coplns h prforanc of h proposd D-RBF nural nwork h rcogniion prforanc for ach of h fac rcogniion xpr using h diffrn yp of fron-nd procssor and h faur vcor dinsion is usd as h asurn inx Our invsigaiv sudy is don on hr sandard fac daabass naly h ORL daabas, Yal daabas and AR daabas h Yal daabas consiss 5 individuals whr for ach individual, iags conaining varying illuinaion, facial xprssions and glasss wr akn h illuinaion variaion conaind in h Yal daabas is liid o ihr fro h lf, fro h righ or cnr Fro hs fac iags, hr iags wih nural xprssion and vn lighing wr usd for raining whil h raining iags wr for sing

6 h ORL daabas conains a s of fac iags of als and fals akn bwn April 99 and April 994 a Olivi Rsarch Laboraory in Cabridg, UK hr ar a oal of n diffrn iags of 40 subjcs in h daabas hs iags conain facial xprssion variaions and prspciv variaions In our xprins, fiv iags wr randoly slcd for raining and h raining fiv iags wr usd for sing h AR daabas consiss of fac iags of 6 popl akn in wo sssions on wo diffrn days For ach subjc, h fronal viw facs fauring diffrn facial xprssions, ild illuinaion ffc and occlusions wr akn h sa picurs wr akn in boh sssions Howvr, in our xprins, a subs of 00 popl (50 als and 50 fals ou of h nir daabas was usd For h valuaion of xprssion variaion using his daabas, wo iags showcasing nural xprssion for ach subjc wr usd for raining whras hr iags wih varying facial xprssions wr usd for sing On h ohr hand, for h valuaion of illuinaion variaion using his daabas, hr iags wih vn illuinaion pr subjc wr usd for raining whras four iags wih varying illuinaion wr usd for sing In h following x, h s s ha conains xprssion variaion is nod as AR-Exprssion whras h s s ha conains illuinaion variaion is nod as AR-Illuinaion All h iags usd in h siulaion wr scald o 3 3 pixls bfor raining A suary of h oal nubr of subjcs, nubr of raining iags and nubr of sing iags ha wr usd for ach daabas in h following xprins is abulad in abl I For h siulaion of h fac xpr sys using p faurs, h layr sizs of h p blif nworks wr s o: ( (no of subjcs h wighd su fusion rul is usd o fus h individual rsuls of all blocks in boh h block-basd D- RBF fac xpr and p faurs fac xpr h prforanc valuaion of h proposd fac xprs on all hr daabass ar shown in abl II A coparison of h faur sizs of h diffrn fron-nd procssor is abulad in abl III ABLE I: Suary of daabas sings usd in h xprins AR AR Daabas Yal ORL Exprssion Illuinaion Nubr of subjcs Nubr of raining iags Nubr of sing iags ABLE II: Rcogniion prforanc (% for ach of h fron-nd procssor wih h proposd D-RBF nural nwork Fron-End rocssor Daabas Yal ORL AR_Exprssion AR_Illuinaion Dp faurs Block-basd uliband Curvl CA+LDA ABLE III: Coparison of dinsions of faur vcor for diffrn fronnd procssors Faur Vcor Fron-End rocssor Dinsion Dp faurs 60 Block-basd 64 uliband Curvl CA+LDA no of subjcs- Fro h rsuls obaind in abl II, h fron-nd procssor using h block-basd chniqu yilds h bs prforanc for all hr daabass xcp for h ORL daabas his could b du o h fac ha h ORL daabas conains sligh pos variaions which ar no prsn in h ohr wo daabass In addiion o h suprior prforanc xhibid by h block-basd D-RBF fac xpr, by dividing h fac iags ino sallr block prior o raining, h faur siz inpu o h proposd D-RBF nural nwork is graly rducd copard o h uliband curvl chniqu and CA+LDA wih h lar s faur siz highly pnn on h siz of oal subjcs, rfr o abl III his can b disadvanagous whn h nubr of subjcs o b raind is larg g larg-scal populaion rcogniion, as h siz of h faurs will incras xponnially and his will affc h prforanc and h raining duraion of h proposd D-RBF nural nwork VIII CONCULSIONS In his papr, w hav prsnd a odular srucural raining archicur o adap h L-basd RBF nural nwork, o h applicaion of fac rcogniion his rsuls in h proposal of h D-RBF nural nwork which uss h growh procss prsnd by [] o cra a copac nural nwork in ach of h RBF oduls For h applicaion of fac rcogniion, h copuaional coplxiy of h Lbasd RBF nural nwork raining is furhr rducd hrough h us of fron-nd procssors o xrac and rduc h dinsion of h faur vcors o b applid o h D-RBF nural nwork hus, w hav invsigad h us of diffrn pr-procssing and dinsion rducion hods in his papr Our siulaion rsuls shows ha h fron-nd procssor using block-basd procssing yilds h bs rcogniion prforanc ajoriy of h s ss and has h scond salls faur vcor dinsion copard o h ohr fron-nd procssors REFERENCES [] B- Oh, "Fac Rcogniion using Radial Basis Funcion Nwork basd on LDA," World Acay of Scinc Enginring and chnology, vol 7, pp 55-59, 005 [] E ng oo, al, "Fac rcogniion wih radial basis funcion (RBF nural nworks," Nural Nworks, IEEE ransacions on, vol 3, pp , 00 [3] Haddadnia, al, "A hybrid larning RBF nural nwork for huan fac rcogniion wih psudo Zrnik on invarian," in Nural

7 Nworks, 00 ICNN '0 rocdings of h 00 Inrnaional oin Confrnc on, 00, pp -6 [4] S hakur, al, "Fac Rcogniion by Cobinaion of RBF Nural Nworks Using Dpsr-Shafr hory," prsnd a h rocdings of h Inrnaional Confrnc on Copuing: hory and Applicaions, 007 [5] V Radha and N Nallaal, "Nural Nwork Basd Fac Rcogniion Using RBFN Classifir," prsnd a h World Congrss on Enginring and Copur Scinc (WCECS 0, San Francisco, USA, 0 [6] W Wang, "Fac Rcogniion Basd on Radial Basis Funcion Nural Nworks," prsnd a h rocdings of h 008 Inrnaional Sinar on Fuur Inforaion chnology and anagn Enginring, 008 [7] Y W Wong, al, "Radial Basis Funcion Nural Nwork Wih Incrnal Larning for Fac Rcogniion," IEEE ransacions on Syss, an, and Cybrnics-ar B Cybrnics, vol 4, pp , 0 [8] Y Wu, al, "Rsarch of Fac Rcogniion Basd on LLE and RBF Nural Nwork," in Elcrical and Conrol Enginring (ICECE, 00 Inrnaional Confrnc on, 00, pp [9] urk and A nland, "Eignfacs for fac rcogniion," ournal of Cogniiv Nuroscinc, vol 3, pp 7-86, 99 [0] N Blhuur, al, "Eignfacs vs Fishrfacs: rcogniion using class spcific linar projcion," arn Analysis and achin Inllignc, IEEE ransacions on, vol 9, pp 7-70, 997 [] Y Hao, al, "Advanags of Radial Basis Funcion Nworks for Dynaic Sys Dsign," Indusrial Elcronics, IEEE ransacions on, vol 58, pp , 0 [] Hagan and B nhaj, "raining fdforward nworks wih h arquard algorih," Nural Nworks, IEEE ransacions on, vol 5, pp , 994 [3] N Apazis and S ranonis, "wo highly fficin scond-orr algorihs for raining fdforward nworks," Nural Nworks, IEEE ransacions on, vol 3, pp , 00 [4] ian-xun, al, "A Nw acobian arix for Opial Larning of Singl-Layr Nural Nworks," Nural Nworks, IEEE ransacions on, vol 9, pp 9-9, 008 [5] B Wilaowski and Y Hao, "Iprovd Copuaion for Lvnbrg-arquard raining," Nural Nworks, IEEE ransacions on, vol, pp , 00 [6] S I Ch'ng, al, "uliband Curvl-basd chniqu for Audiovisual Rcogniion ovr Inrn roocol," Lcur Nos of h Insiu for Copur Scincs, Social Inforaics and lcounicaions Enginring (LNICS, vol, pp 53-60, 0 [7] Liu, "A novl x classificaion approach basd on p blif nwork," prsnd a h rocdings of h 7h inrnaional confrnc on Nural inforaion procssing: hory and algorihs - Volu ar I, Sydny, Ausralia, 00 [8] S I Ch'ng, al, "Block-basd Dp Blif Nworks for Fac Rcogniion," Inrnaional ournal of Biorics, vol 4, pp 30-43, 0