A New Development on ANN in China Biomimetic Pattern Recognition and Multi Weight Vector Neurons

Transcription

1 A New Developmet o ANN Cha Bommetc atter Recogto ad Mult Weght Vector Neuros houue Wag Lab of Artfcal Neural Networks. Ist. of emcoductors. CA. Beg Cha wsue@red.sem.ac.c Abstract. A ew model of patter recogto prcples Bommetc atter Recogto whch s based o matter cogto stead of matter classfcato has bee proposed. As a mportat meas realzg Bommetc atter Recogto the mathematcal model ad aalyzg method of ANN get breakthrough: a ovel all-purpose mathematcal model has bee advaced whch ca smulate all kds of euro archtecture cludg RBF ad B models. As the same tme ths model has bee realzed usg hardware; the hgh-dmeso space geometry method a ew meas to aalyzg ANN has bee researched. Keywords: Neural etwork hgh-dmesoal space geometry patter recogto Itroducto Artfcal eural etwork s a mportat method of patter recogto all alog accordg to ANNs powerful ablty to appromate fuctos ad ecellet ablty of self-learg parttog ad robustess. But o oe had ANN lacks a all-purpose mathematcal model whch ca smulate all kds of euro archtecture. Furthermore after the etworks have bee bult huge mult-varable olear equatos are very dffcult to aalyze mathematcally because the mathematcal descrpto of each euro s a mult-varable olear equato. The comple multvarable olear equatos are always qute hard to aalyze ad solve although lots of methods of mathematcal aalyss have bee advaced by may scholars []. Ths codto wll lmt the -depth developmet of the ANN from the aalytc pot of vew. O the other had the tradtoal patter recogto s based o matter classfcato whch uses optmal separatg as ma prcple. It s dfferet from fucto of huma cogze. I order to solve these problems a ovel all-purpose mathematcal model of euro ad a ew aalyss method of ANN have bee proposed. At the same tme a ovel mode of patter recogto Bommetc atter Recogto has bee advaced ad appled practcally. The superortes of ths method have bee prelmarly showed by the epermetal results. G. Wag et al. Eds.: RFDGrC 2003 LNAI 2639 pp prger-verlag Berl Hedelberg 2003

2 36. Wag 2 The Basc All-urpose Mathematcal Model of Neuros As we kow the basc ut of ANN s euro o matter how t s smulated wth hardware or software geeral-purpose computer. I accordace the ANNs performace s prmarly decded by the basc computato method ad fucto of euros. Accordg to the bra cells actvato ad restrat mechasm a hyperplae euro model ad a hypersphere euro model have bee proposed respectvely the earler tme [2]. The mathematcs model of hyperplae eural show as followg: Y f W X 0 θ Where Y deotes the euro output. f s a actvato fucto olear fucto of euro whch may be a step fucto. X s the put vector of euro. W s the euros weght vector ad θ s the euros actvato threshold. The mathematcal model of hypersphere euro RBF ca be epressed as followg: Y f 0 W X 2 θ It was testfed by epermets that applcatos of patter recogto or fucto fttg the eural etworks descrbed by 2 has the better performace tha the eural etwork descrbed by. Ths paper dscusses a basc mathematcal model wth better commoalty uversal fuctoalty ad easy mplemetato by geeral-purpose eural etworks hardware. The hardware based o ths model ca have better adaptablty ad hgher performace. The latter tet wll dscuss a ovel basc algorthm for hgh-order hypersurface eural etworks whch has bee appled the desg ad practce of CAANDAR-II eurocomputer The Dscusso of the Commoalty of Hypersurface Neuro Basc Mathematcal Model The basc mathematcal model of hypersurface euro for the eurocomputer must satsfy the followg codtos: The model ca cover both the tradtoal hyperplae euro ad RBF eural etworks. 2 The model had the possblty to mplemet may varous hypersurface. 3 The model ca mplemet character modfcato by adustg morty parameters. 4 The model ca easly mplemet hgh-speed calculato wth hardware methods. Accordg to the codto the basc mathematcal calculato of geeral euro must have basc calculatos of both formula ad 2. Thus the sgal for each put ode must have two weghts: oe s drecto weght ad the other s core weght. The followg formula llustrates the structure of ths model:

3 A New Developmet o ANN Cha 37 W W Y f [ W θ ] W 3 W W Where Y deotes the output of euro. f deotes the actvato fucto of euro. θ s the threshold of euro. W ad W are two weghts from the -th put ode to euro. X s a postve put value from the -th put ode. s the dmeso of put space. s a parameter for determg the sg of sgle etry. If 0 the sg of sgle etry s always postve ad f the sg of sgle etry s the same as the sg of W X W. s a de parameter. Obvously the formula ad formula 2 are oly two specal cases of formula 4. Whe parameter has dfferet value hypersurface of euro has dfferet fgure lke the followg eamples. p/2w0.5 pw0.5 p2w0.5 w20.5w30.5 w20.5w30.5 w20.5w30.5 Fg.. Fg. 2. Fg. 3. p3w0.5 p2w.0 p2w/3 w20.5w30.5 w2.0w3/3 w2.0w3.0 Fg. 4. Fg. 5. Fg. 6.

4 38. Wag 2.2 The Geeral Calculato of Neural Networks Hardware Based o Mult Weght Vector Neuros The author created geeral CAANDRA II eurocomputer hardware based o formula 5 whch s sutable for tradtoal B etworks RBF etworks ad varous hgh-order hypersurface eural etworks. Its geeral calculato s as follows: Om t + Fk λ[ C R θ] 4 Where W Im W W g Omg W g R [ W ] Im W + W g Omg Wg 5 W I W g W O W m Accordg to the formula 4 CAANDRA II eurocomputer ca smulate radom eural etworks archtectures wth varous euro features cludg hyperplae hypersphere varous hyper-sausage hypercube ad so o. The detal of above formula s provded represet [3]. g mg g 3 Aalyss ad Theory of Hgh-Dmesoal pace Geometry for ANN Although the hgh-dmesoal space geometry have bee developed for may years systemc ad appled geometrc aalytcal methods have ot bee foud yet. We try to do some prmary supplemets ths fled. We gave some aoms ad theorems of the hgh-dmesoal space geometry whch ca be used to aalyze the behavor of the ANN. 3. The Correspodg Represetato of the Neuro the Hgh-Dmesoal pace Geometry Geerally a euro ca be uderstood as a smple calculato as followg: Y f [ Φ 2 L θ ] 6 For the euro of B etworks Φ L w 7 2 Ad for the euro of RBF etworks [ w ] Φ L 8 o a euro ca be correspodgly represeted as a hyperplae or a hypersurface the hgh-dmesoal space. The equato s: Φ L θ The bass of the euro output fucto f s the dstace betwee the put pot put space ad the hyperplae or the hypersurface. Obvously the cocepto of geometry about the hyperplae or the hypersurface s qute effectve to aalyze the behavor of ANN.

5 A New Developmet o ANN Cha The Bascally Aalytcal Methods of Theorems about the Hgh-Dmesoal pace Geometry Ths part s omtted because of the lmt of aper legth. It s dscussed detal aother aper [6]. 4 The Theory ad Applcato of Bommetc Topologcal atter Recogto Bommetc atter Recogto BR s a ew model of atter Recogto based o matter cogto stead of matter classfcato. Ths ew model s rather closer to the fucto of huma beg tha tradtoal statstcal atter Recogto usg optmal separatg as ts ma prcple. The basc prcple of Bommetc atter Recogto s: I the feature space R suppose that set A be a pot set cludg all samples class A. For f y A ad ε >0 are gve there must be et B: B 2 L y m m+ < ε m [ ] m N} A That s to say there s o solated pot set A ad A s coected. Tradtoal patter recogto takes the optmal demarcato of dfferet kds of samples feature space as the target. Whereas the bommetc patter recogto takes the optmal cover of dstrbuto of oe kd of samples feature space as the target. Take the case of two-dmeso space as followg chart: I ths chart tragles are the samples to be detfed crcles ad crosses are two kds of samples that dffer from tragles fold les are the partto modes of tradtoal B etworks patter recogto great crcles are the partto modes of RBF etworks equal to the recogto mode wth template matchg curves cosstg of hyper sausage stad for recogto modes of the bommetc patter recogto. The dstrbuto of the samples the feature space s cotuous so the basc pot of bommetc patter recogto s to aalyze the relatoshp of the trag samples. The dffereces betwee the Bommetc atter Recogto ad tradtoal atter Recogto are showed Table.

6 40. Wag Table. Compareato of Tradtoal atter Recogto ad BR Tradtoal atter Recogto The optmal classfcato of may kds of samples The dstcto betwee oe kd of samples ad lmted kds of kow samples Based o the dstct of dfferet samples To fd the optmal classfcato surface Bommetc atter Recogto Cogzg dfferet kds of samples oe by oe The dstcto betwee oe kd of samples ad ukow ulmted dfferet kds of samples Based o the coecto of homologous samples To fd the optmal coverg of homologous samples The method used by bommetc patter recogto s Hgh-Dmesoal pace Comple Geometrcal Body Cover Recogto Method. It studes some kds of samples dstrbuto feature space ad gves reasoable cover so the samples ca be recogzed. That s to say actual Bommetc atter Recogto to udge whether the pots belogs to set a a -dmesoal geometrcal shape the feature space coverg set a eed be costructed wth the software ad hardware methods. Ths geometrcal shape s a uo of fte -dmesoal hypersphere wth costat k as radus ad fte pots as ceter of sphere. More precsely t s the topologcal product of set A ad -dmesoal hypersphere. Accordg to dmeso theory [4] to dvde -dmesoal space to two parts the terface must be a - dmesoal hyperplae or hypersurface. A euro Artfcal Neural Networks ANN ca costruct a - dmesoal hyperplae or hypersurface -dmesoal space. Moreover from the foregog theory a euro ca costruct may kds of comple closed hypersurface ad a mult-weght euro ca be a covered rage of complcated shape feature space based o ts multweght vectors.. Therefore ANN usg the aalyss method of hgh-dmesoal space geometry s a very approprate method to mplemet the Bommetc atter Recogto. The applcato of Bommetc atter Recogto usg ANN s very broad oe of whch cogzg obects wll be troduced the followg cotet. I ths appled eample cogzed obects such as aval shp tak bus bow horse sheep o horzotal plae or sea level are tested to be cogzed from dfferet drectos. amplg process s to collect the bmp mages sampled from dfferet drectos cotuous mappg ad the compress to 256-dmesoal samples the feature space. ce observatoal drectos are horzotal the drecto ca oly vary oe dmeso. o the samples dstrbuto the feature space s appromate oe-dmesoal mafold. Cosderg some small chages o other drectos the coverg shape of certa kd of obect the feature space ca be regarded as a topologcal product of oe-dmesoal mafold homeomorphc wth a aulus ad 256-dmesoal hypersphere. It ca be descrbed that there has a pots

7 A New Developmet o ANN Cha 4 set a the 256-dmetoal feature space ad the dstaces betwee ay pot a ad a certa closed space curve s less the a certa costat k. The space curve covers set cludg all samples. The we got A R A N m m N A where R A y k y samples of umber total the m m a > < < + } 0 2 } } 2 ε ε ε L L To cover a appromately wth several euros ANN space curve A s replaced by several space le segmets. The the coverg of every euro s a topologcal product of a le segmet ad a -dmesoal hypersphere. Cosder the orgal samples set let be a subset of wth elemets as followg: t cos selected s d where d } ta 02 L Let euros cover a appromately the the coverg of th euro s: [0]} } + + α α α B R B y k y The coverg of all euros s: U 0 a whch represets hypersurfaces lookg as f sausages. o t s called Hyper ausage Neuro HN. I the epermets the followg eght obects were used: lo tger tak car etc. as show fgure 7. Those obects were rotated to sample ad each obect has bee sampled 400 tmes. o the frst samples set has eght kds of samples ad totally 3200 samples. Ad the aother tme samplg was doe repeatedly we got the secod samples set wth 3200 samples. Fg. 7. The we use s obects cat dog zebra etc as show fgure 8. Each obect was sampled 400 tmes by usg the above method. o the thrd samples set wth 2400 samples was created. The ma steps ad results of the epermets are as follows:

8 42. Wag Fg. 8. The trag sets are chose from the Frst amples et wth dfferet dstaces of every two adog samples They have 338 samples eght classes altogether. 2 Neural etworks correspodg to eght kds of obects a b etc. were costructed wth ufed dstace parameter k. Ad the umber of euros s as equal as that of samples the trag set. 3 All 6400 samples the frst ad secod sample sets were regarded as testg samples for calculatg correct recogto rate. The result of epermet s that the correct recogto rate s 99.87%. No sample has bee recogzed correctly oly 0.3% s reected. 4 All 8800 samples three sample sets are regarded as testg samples for calculatg error recogto rate. Ad the error recogto rate s 0. The cotract of the epermetal results of RBF-VM [5] ad HN-BR wth reducg umber of trag samples s showed Table 2. Amout of Trag amples Table 2. The Results of RBF-VM ad BR V RBF-VM correct rate HN BR correct rate % % % % % % % % % % % %

9 A New Developmet o ANN Cha 43 5 Cocluso A ew model of patter recogto prcples Bommetc atter Recogto usg hyper sausage euro s proposed ths aper. It s a fre-ew theoretcs of atter Recogto. As the base we have advaced a ew model of hypersurface euro mult weght vector euro ad mplemeted t the ew CAANDRAeurocomputer. Ths method of patter recogto s more effectve ad effcet tha tradtoal patter recogto. Ad ts applcato s very broad we apply Bommetc atter Recogto to detfyg obects such as aval shp tak bus bow horse sheep from dfferet drectos o horzotal plae or sea level etc. the result s much better tha VM. Refereces [] J.J. Hopfeld Neural etworks ad physcal systems wth emerget collectve computatoal abltes roc. Natl. Acad. c. U..A [2] W.. McCulloch ad W.tts. A logc Calculus of the Ideas Immet Nerves Actvty.Bullet of Mathematcal Bophyscs 943 5:5 33. [3] Wag hou-ue Dscusso o the Basc Mathematcal Models of Neuros Geeral urpose Neurocomputer ACTA ELECTRONICA INICA Vol.29 No.5 pp May.200 [4] Ryszard Egelkg Dmeso Theory WN-olsh cetfc ublshers-warszawa 978. [5] Boser B Guyo I ad Vapk V.N. A Trag Algorthm for Optmal Marg Classfers Ffth Aual Workshop o Computatoal Learg Theory ttsburgh ACM [6] Wag houue Wag Baa Aalyss ad Theory of Hgh-Dmeso pace Geometry for Artfcal Neural Networks Acta Electroca ca Vol. 30 No. Ja 2002