A handwritten signature recognition system based on LSVM. Chen jie ping

Size: px

Start display at page:

Download "A handwritten signature recognition system based on LSVM. Chen jie ping"

Meagan French
5 years ago
Views:

1 Iteratoal Coferece o Computatoal Scece ad Egeerg (ICCSE 05) A hadrtte sgature recogto sstem based o LSVM Che je pg Guagx Vocatoal ad echcal College, departmet of computer ad electroc formato egeerg, ag, guagx, Cha 5306 Keords: VSM, LSVM, Hadrtte sgature recogto, Recogto rate. Abstract. Hadrtte sgature recogto has a de applcato prospect the feld of dett detfcato. hs paper presets a shape feature, damc feature extracto ad recogto techolog of LSVM state based o the combato of hadrtte sgature recogto sstem. Objectve fucto chages the expermetal model of SVM, to LSVM, to avod decomposto he solvg the multple soluto algorthm of the problems exstg the SVM, for the purpose of mprovg the operatg tme of the sstem. he expermetal results sho that t has a hgh recogto rate ad faster recogto rate. Itroducto As a behavor characterstc, hadrtte sgature has ts o bologcal features as relatve stablt ad mtatve dffcult. I ma cases such as sgg a cotract, agreemet or makg a certfcate, hadrtte sgature s usuall dspesable ad has rreplaceable legal status ad mportace. Hece, hadrtte sgature recogto has a de applcato prospect the feld of dett detfcato []. Hoever, as the damc feature of hadrtte sgature loses durg the rtg process, factors that ca represet the sger s feature become less, hch leads to a lo recogto rate of hadrtte sgature. hs paper troduces a LSVM based hadrtte sgature recogto sstem hch has a hgh recogto rate ad fast recogto speed accordg to the expermetal result of mtato. LSVM based hadrtte sgature recogto sstem he desg dea of the hadrtte sgature recogto sstem ths paper s: coduct a pre-treatmet of the mage of the hadrtte sgature, cludg the smoothg, barzato, ormalzato ad refemet of the sgature mage; extract the features of the pre-treated mage from the aspects of shape features ad pseudo damc characterstcs [], ad select them th Bhattachara algorthm; at last, use LSVM as the algorthm for the recogto ad get the hadrtte sgature recogto sstem [3]. he process of LSVM based hadrtte sgature recogto sstem s sho Fgure. Fgure Process of LSVM based hadrtte sgature recogto sstem 05. he authors - Publshed b Atlats Press 48

2 LSVM (Lagraga Support Vector Mache) LSVM s a e mache learg method [4] proposed b Vapk et al. Due to ts outstadg learg performace, LSVM has a de applcato varous felds such as detectg ad dscrmatg huma faces, recogzg hadrtg letters ad categorzg artcles. he dea that supports the LSVM s: to make the sample formato compromse betee the model complext ad learg capablt so as to obta the best geeralzato ablt that s the ablt of predctg future output accuratel. he dea s based o the VC theor statstcal theor ad the prcple of the mmum structure rsk, ad LSVM s to fd out the optmal classfcato plae betee to dfferet categores of samples o the bass of mmum structure rsk. Fgure s a sketch of optmal classfcato plae. H stads for classfcato plae; H ad H refer to hperplae; the classfcato terval of the to categores s the dstace betee H ad H. he optmal plae meas a plae that ca ot ol separate the to categores accuratel but also has a maxmum classfcato terval betee the to categores. Fgure Optmal classfcato plae We assume the sample set as (x, ), d x R =,,,, {, } as the categor label, the sample sets of C ad C are learl separable sets, that s (,b ) s exsted, the ( x ) b 0, x C ( x ) b 0, x C () he lear dscrmato fucto s g( x) ( x ) b. he purpose of classfcato s to fd out (,b ) to separatel C ad C optmall. g( x) Normalze the dscrmato fucto ad assure all the samples satsf, that s to costra the (,b) as, m g( x) he terval of classfcato s. Uder such crcumstaces, e have to satsf the follog fucto to make assure all the classfcatos of the samples are correct: [( x ) b],,... () Accordg to Fgure, the classfcato terval of the to categores s. o crease the terval meas to reduce (or ). Optmal classfcato plae meas the plae that ca satsf 48

3 Formula () th mmum value. Support vector mache refers to the testg samples of the to categores. he optmal classfcato ca be expressed as: m, b (costrat codto) s.t. [( x ) b],,... (3) he quadratc programmg problem th lear costrats Formula (3) ca be solved b usg Lagrage multpler method ad s defed as follog Lagrage fucto: I L(, b, a) a{ [( x ) b] } (4) s.t. a 0 If the to sdes of Formula (4) take the partal dervatves 0 of ad b, the optmal questo should satsf the follog formula accordg to Kuhu-ucker codto: a [ ( x ) b] 0},,..., (5) here ll be ver fe a (that s Lagrage multpler) that s ot 0. he relatve sample x of a s the Support vector mache, hch s the pots o hperplaes of H ad H. he maxmum objectve fucto s: Q( a) a jaa j ( x ) j (6) a x he the optmal soluto s: After above solutos, the optmal classfcato fucto s: f ( x) sg[( x) b ] (7) b x here,sg()refers to sg fucto, ' If e use er product fucto K( x, x )to replace the dot product the optmal classfcato plae, the optmal fucto ll be: Q( a) a aa j j K( x, ) j (8) Related classfcato fucto ll be: f ( x) sg( a K( x, x) b ) (9) Other codtos the algorthm ll keep the same. hs s the olear SVM. Hoever, as stadard support vector mache solves dual problems mal b quadratc programmg, the trag speed s ver lo. Besdes, there ll be plet of matrx operatos th ths algorthm. As a result, recetl ears, ma scetsts troduced the LSVM (Lagraga Support Vector Mache), hch s proposed b OlvL Magasara 00[6]. LSVM made a small chage o the objectve fucto of SVM model so that to chage the dual problem to a mmum quadratc fucto hch has o upper lmts, the solve th terato. I ths a, LSVM ca avod the problem of decomposto for ma tmes ad mprove the algorthm trag speed greatl. If there are m sample pots the dmesoal space that eed to be categorzed ad are represeted b mx matrx A; the elemet d the dagoal matrx D s + or -; the categor that 483

4 represets the sample A s A or A. After mmze the error varace the stadard SVM, e ould get the follog classfcato plae: x (0) Expresso of the classfer decded b the classfcato plae s: 0, the x A x 0, the x A 0, the x A or x A () he orgal problem of SVM stadard model ca be expressed as: me D( A e ) e he dual problem s: 0 a e s.t. 0 mw ( a) a DAA Da e a () (3) s.t. e Da 0 At frst, LSVM ll make follog chages o the objectve fucto Formula (3): '. Use NF stead of NF for the error term, that s to replace e th. So the costrat codto 0 ca be omtted.. Add to the objectve fucto ad the objectve fucto ll be. he purpose of the chage s to remove the upper costrat problem of the related dual problem. After the chage, the orgal problem ll be: m (4) s.t. D( A e ) e Costruct the Lagrage fucto for the dual problem L(,, a) a[ A e ) e] (5) ake the dervatves respectvel of,, the above formula ad set them 0, e ga: a A Da,, -e Da (6) Substtute them to Formula (5), e get the dual problem: m a ( D( AA ee ) D) a e a m 0aR (7) As sho, the revsed problem as chaged to a optmzato problem thout upper costrat codto ad ca be smplfed as: 484

5 H D[ A e], Q HH So the dual problem Formula (6) becomes: m a Qa e a m 0 ar (8) Qa e, the the KK(Karush Kuhu ucher) codto of the optmzato We assume problem for the Formula (6) ll be: a ( a 0, 0) a 0 ad 0 meas both a ad should be o-egatve vectors A to real umbers (or vectors) a ad b ca satsf the follog theorem: 0 a b 0 a ( a b), 0 (9) (0) Formula (9) (that s KK codto) ca be chaged to follog equato form ( ca be a postve umber): Qa e (( Qa e) a) () Accordg to the KK codto above, e ca get Formula (), hch s a teratve formula that ll la a mportat bass for LSVM algorthm. he formula s sho as: a Q ( e (( Qa e) a ) ), 0,,... () hs teratve formula ol eeds to satsf: 0 (3) We ca get a optmal soluto of lear covergece Formula (8) terated from a tal.9 pot. Here e take. I order to get Q, e eed to calculate the verse matrx of m x matrx.it ll a large amout he the umber of sample pot s large. Here e use to Sberma Morrso Woodbur (SMW)to solve the verse matrx: I I Q ( HH ) ( H H ) H ) (4) Where,H s a m (+l) matrx. So after the chage, a verse matrx problem of a large m m matrx as tured to that of a small (+l) ( 十 ) matrx. It makes the soluto of Q possble, ad eve accelerates the soluto, hch eables LSVM algorthm to solve the lear classfcato problem ver fast. Istructo of the samples used the sstem ad expermetal results I the expermet, e collected 400 hadrtte sgatures from 0 dfferet age groups. Frst e asked the 0 subjects to sg 0 sgatures dfferet tme ad evromet so that e obtaed 00 real sgatures; the e asked them to mtate 0 sgatures from others, ad e got 00 false sgatures. So e have 400 sgatures total, 50% of real sgatures ad 50% of false sgatures. Durg the expermet, e dvded the real sgatures to to groups, 00 of them ere combed th 40 false sgatures ad ere treated as the trag samples. Both of the trag samples ad the rest samples ere regarded as the sample of expermet recogto. Here e take 林建书 as a example to aalze the expermetal results. Fgure 3 ad Fgure 4 are some of the real ad false sgatures of 林建书. 485

Fgure 3 Real sgatures of L Jashu Fgure 4 False sgatures of L Jashu Frst, e coduct a pre-treatmet o the orgal mage ad extract 9 features of

We ould use Bhattachara to select the features ad get 7 of the features for the dstcto ad recogto, hch clude: depth-dth rato of the

hgh gra level stablt feature, grascale dstrbuto hstogram of the hadrtte sgature.

he expermetal results are sho able : able Expermetal results obtaed th LSVM algorthm Cocluso rag accurac estg accurac Correct recogto rate

83% hs paper proposed a hadrtte sgature recogto sstem based o LSVM recogto techolog combed th extracto of shape feature ad damc feature.

features as the level for dstcto ad recogto so that to obta a better recogto rate th mmum features.

problem exsted the SVM ad mproved the operato tme of the sstem. Refereces: [] Wag Sh, Wag Yuasheg. he Arrval of Bometrc me [J].

6 Fgure 3 Real sgatures of L Jashu Fgure 4 False sgatures of L Jashu Frst, e coduct a pre-treatmet o the orgal mage ad extract 9 features of the treated mage from the aspects of shape ad pseudo damc characterstcs. But ot all the features ll be used. We ould use Bhattachara to select the features ad get 7 of the features for the dstcto ad recogto, hch clude: depth-dth rato of the level-compressed sgature, black dot area ad total area rato, relatve gravt the horzotal ad vertcal drectos, gra feature of skeleto drecto, hgh gra level stablt feature, grascale dstrbuto hstogram of the hadrtte sgature. he expermet as coducted th the LSVM Hadrtte sgature recogto algorthm. he expermetal results are sho able : able Expermetal results obtaed th LSVM algorthm Cocluso rag accurac estg accurac Correct recogto rate of real sgature Correct recogto rate of false sgature 87.9% 80.% 84% Averaged correct recogto rate 86.8% 8.86% 84.83% hs paper proposed a hadrtte sgature recogto sstem based o LSVM recogto techolog combed th extracto of shape feature ad damc feature. he sstem extracted 9 dfferet features of the hadrtte sgature mage from the aspects of stll ad damc characterstcs ad evetuall selected 7 features as the level for dstcto ad recogto so that to obta a better recogto rate th mmum features. Based o the off-le LSVM algorthm recogzer, the expermet chaged the objectve fucto of SVM model to LSVM, hch avoded the multple soluto problem exsted the SVM ad mproved the operato tme of the sstem. Refereces: [] Wag Sh, Wag Yuasheg. he Arrval of Bometrc me [J].Guagmg Dal,00(58):0-. [] Xe Ya, Idetfcato Research of Off-le Chese Sgature Based o PseudoDamc Feature Extracto [D].Shadog Uverst,007. [3] Fe Zogzhe, Cheg Xagju.Model Recogto Prcple[M].X a:scece ad echolog Press of X a Uterst,003. [4] Vapk VN. he Nature of Statstca of Learg heor [M]. Ne York Sprg [5] Zhag Xzhog. Chese Characters Recogto echolog [M].Bejg : sghua Uverst Press,

7 [6] Wag Peg,Zhu Xaoa.Selecto ad Applcato of Model Based o RBF Kerel SVM [J].Computer Egeerg ad Applcatos, 003(4):

Support vector machines II

Support vector machines II CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error