Maximum Likelihood Directed Enumeration Method in Piecewise-Regular Object Recognition

Transcription

1 Maxmum Lkelhood Dected Enumeaton Method n Pecewse-Regula Obect Recognton Andey Savchenko Abstact We exploe the poblems of classfcaton of composte obect (mages, speech sgnals wth low numbe of models pe class. We study the queston of mpovng ecognton pefomance fo medum-szed database (thousands of classes. The key ssue of fast appoxmate neaest-neghbo methods wdely appled n ths task s the heustc natue. It s possble to stongly pove the effcency by usng the theoy of algothms only fo smple smlaty measues and atfcally geneated tasks. On the contay, n ths pape we popose an altenatve, statstcally optmal geedy algothm. At each step of ths algothm ont densty (lkelhood of dstances to pevously checked models s estmated fo each class. The next model to check s selected fom the class wth the maxmal lkelhood. The latte s estmated based on the asymptotc popetes of the Kullback-Leble nfomaton dscmnaton and mathematcal model of pecewse-egula obect wth dstbuton of each egula segment of exponental type. Expemental esults n face ecognton fo FERET dataset pove that the poposed method s much moe effectve than not only bute foce and the baselne (dected enumeaton method but also appoxmate neaest neghbo methods fom FLANN and NonMetcSpaceLb lbaes (andomzed kd-tee, composte ndex, pem-sot.. Intoducton Conventonal machne leanng technques (suppot vecto machnes, multlayeed feed-fowad neual netwoks, deep neual netwoks, etc [] eque lage epesentatve tanng sample to estmate the class bode. These methods ae known to be chaactezed wth low accuacy f only few models ae avalable fo each class []. Ths ssue s qute acute n, e.g., face ecognton task n whch t s sometmes dffcult to gathe vaous photos of the nteestng peson [, 3]. The poblem of nsuffcent accuacy becomes moe complcated f the numbe of classes s lage (hundeds o even thousands of classes. As a esult, thee s pactcally no altenatve to the neaest neghbo (NN methods n ths task []. Howeve, f the complex obects should be ecognzed n eal-tme (e.g., vdeo-based face ecognton [3] and only standad hadwae s avalable, the pefomance of bute-foce mplementaton of the NN seach s not enough. It seems that conventonal fast appoxmate NN methods fo mage ecognton, e.g. tangle tee [4], composte kd-tee [5], andomzed kd-tee [6], Best- Bn Fst [7], etc. can be appled. Unfotunately, t s known that these technques show good pefomance only f the fst NN s qute dffeent fom othe models [7]. Such estcton has much n common wth many eal-wold applcatons, fo nstance, faces have smla shape and common featues. The othe lmtaton s the applcaton wth smlaty measues whch satsfy metc popetes (sometmes, tangle nequalty and, usually, symmety [4, 7, 8]. Moeove, these methods ae usually developed to appoxmately match vey-lage numbe ( of mage

2 descptos of extacted keyponts [9]. Hence, the pefomance s compaable wth bute-foce method fo medum-szed vocabulaes (thousands of classes. To decease the ecognton speed fo such tanng sets, odeng pemutatons (pemsot method has ecently been poposed [0]. Anothe nteestng appoach, namely, the dected enumeaton method (DEM outpefoms the known appoxmate NN methods n face ecognton []. Fnal ssue s the heustc natue of most popula appoxmate NN methods. It s pactcally mpossble to pove that patcula algothm s optmal (n some sense and nothng can be done to mpove t. In ths pape we popose an altenatve soluton on the bass of the statstcal appoach - whle lookng fo the NN fo patcula quey obect, conventonal pobablty of belongng of pevously checked models to each class s estmated. The next model fom the database s selected fom the class wth maxmal pobablty. Thus, ou task s to estmate ths pobablty and to clafy the mentoned geedy-seach algothm. The est of the pape s oganzed as follows. In Secton we exploe the task of ecognton of pecewse-egula obects and pesent the statstcal paametc cteon based on the Kullback-Leble mnmum dscmnaton pncple []. In Secton 3 we befly evew the baselne method (DEM, emnd the asymptotc popetes of the Kullback-Leble dscmnaton and popose the novel Maxmum-Lkelhood DEM (ML-DEM. In Secton 4 we demonstate expemental esults of compason of ou method wth seveal appoxmate NN algothms n face ecognton wth FERET dataset. Fnally, concludng comments ae gven n Secton 5.. Statstcal ecognton of pecewse-egula obect In the classfcaton task t s equed to assgn the quey obect (facal photo, speech sgnal, mage of natual scenes, text to one of R> classes. Most pat of contempoay eseach assumes that each class s specfed by the gven database { }, {,..., R} of R cases (models. Let the quey obect be epesented as a sequence of K egula (homogeneous pats [3] extacted by any segmentaton pocedue: { ( k=, K} =. Evey k-th segment ( { ( =, n( } a sequence of (pmtve featue vectos ( { x,..., ( } = x s put n coespondence wth x = ; ( x ; p wth fxed dmenson p=const, whee n( s the numbe of featues n the k-th segment. Smlaly, evey -th model s epesented as a sequence { ( k=, K } K segments and the k-th segment s defned as the numbe of featues n the k-th segment of the -th model. To apply statstcal appoach, let's assume that:. Vectos x (, x ( k ae andom. = of ( = x ( =, n ( of featue vectos x (. Hee n ( s. Segments (, k=, K and (, k =, K ae goups - andom samples of..d. featue vectos x ( and x (, espectvely.

3 3. Featue vectos of patcula segment of one class ae dentcally dstbuted. As the pocedue of automatc segmentaton s naccuate, evey segment ( should be compaed wth a set ( N of numbes of closed to k segments of the -th model. Ths neghbohood s detemned fo a specfc task ndvdually. If t s assumed that segmentaton pocedue s always coect, we may put N ( { k}, =, K = K. K K Thee ae two possble appoaches to estmate unknown class denstes, namely, paametc and nonpaametc []. Let's dscove paametc appoach n detal. It s assumed that dstbutons of vectos x ( and x ( ae of multvaate exponental type f θ ; n [] geneated by the fxed (fo all classes functon f 0( wth p-dmensonal paamete vecto θ: ( exp( τ( θ θˆ( f ( / M( fθ; n = 0 τ ( whee ˆ( θ s an estmaton of paamete θ usng avalable data (andom sample of sze n, ( τ( θ θˆ( f ( M( τ = exp 0 d ( and τ (θ s a nomalzng functon (p-dmensonal paamete vecto defned by the followng equaton f the paamete estmaton ˆ( θ s unbased (see [] fo detals d θˆ( fθ ( d ln ( τ = θ ; n M (3 dτ Each -th class of each k-th segment s detemned by paamete vecto θ (. Ths assumpton about exponental famly f θ ˆ( ( ; n( n whch paamete θ (k s estmated by usng the obseved (gven sample (, coves wde ange of known dstbutons (polynomal, nomal, etc. []. Hence, the ecognton task s educed to a poblem of statstcal testng of R smple hypothess about paamete vecto θ (. In ths pape we focus on the case of full po uncetanty and assume that the po pobabltes of each class ae equal. In such case, Bayesan appoach wll be equvalent to the maxmum lkelhood cteon. Fo ou task, evey segment s ecognzed wth the followng ule max k N max,..., f ( ( θˆ ( k ; n( k { R}. (4 It can be shown that eq. (4 s equvalent to the Kullback-Leble mnmum nfomaton dscmnaton pncple [] whee and (, ρ = K = mn nk k = k N (, ρ mn, (5 =, R I ˆ (*: f ; ( θˆ ( ( k ; k n( (6 3

4 Iˆ *: fˆ θ ( ( k ; = f ( θˆ ( ( ; n( ; ( = n( f θˆ ln f θˆ ( ( ; ( ( k ( n( d ( ; n( (7 s the Kullback-Leble dvegence between segments ( and ( k ; and n= K n(. k= Thus, cteon (5-(7 s an obvous mplementaton of Bayesan appoach to composte obect ecognton f the pobablstc mathematcal model of pecewse-egula obect [3] s used. 3. Maxmum-lkelhood dected enumeaton method Let's use an appoach known fom atfcal ntellgence to ceate an appoxmate NN algothm fo measue of smlaty (6, (7. Namely, we pmaly focus on geedy algothms: on each step t exploes the model whch s the NN of the quey obect wth the hghest pobablty. Such choose of the geedy class of algothms s explaned not only by ts smplcty, but by the fact that pactcally all known appoxmate NN methods ae geedy n some sense. 3.. Baselne: dected enumeaton method As a baselne method we use the DEM [] whch was based on the metc popetes of the Kullback-Leble dvegence and egads the models' smlaty ρ, = ρ(, as an aveage nfomaton fom an obsevaton to dstnct class fom an altenatve class. Hence, at the pelmnaly step of the DEM, the model dstance matx Ρ = [ ρ, ] s calculated as t s done n the AESA (Appoxmatng and Elmnatng Seach Algothm method [4]. Ths tme-consumng pocedue should be epeated only once fo a patcula task and tanng set. Ognal DEM used the followng heustc: f thee exsts a model * fo whch ρ *, < ρ0 <<, then fo an abtay -th model the followng condton holds (, ρ,. << (5 can be smplfed ρ * wth hgh pobablty. Hence, the ctea ρ, * < ρ = const 0. (8 Eq. (8 defnes the temnaton condton of the appoxmate NN method. If false-accept ate (FAR s fxed then 0 β = const, ρ s evaluated as a β -quantle of the dstances between mages fom dstnct classes { ρ =, R, =, R, } a matte of fact, the optmzaton task (5 s eplaced to an exhaustve seach whch temnates f condton (8 holds fo the cuently checked model. Accodng to the DEM [], at fst, the model {,..., R}, s andomly chosen so that {,..., R} ρ, ρ,,, (9,. As 4

5 and the dstance ρ, s calculated. If the dstance s lowe than a theshold ρ 0 (8, the seach s temnated. Othewse, t s put nto the poty queue of models soted by the dstance to. Next, the hghest poty tem s pulled fom the queue and the set of models whee ρ( = ρ (, the set (M s detemned fom ( M M k, (0 ρ( ρ(, ρ s the devaton of ρ, elatve to the dstance between and. Fo all models fom (M the dstance to the quey obect s calculated and the condton (9 s vefed. Afte that, evey pevously unchecked model fom ths set s put nto the poty queue. The method s temnated f fo one model obect condton (9 holds o afte checkng fo E max = const models. As we stated eale, ths method s heustc as most popula appoxmate NN algothms. Howeve, the pobablty that the model s the NN of can be dectly calculated fo the Kullback-Leble dscmnaton by usng ts asymptotc popetes. Let's descbe them befly. 3.. Asymptotc popetes θ ˆ ν ν, It s known [] that f the segment ( has dstbuton of exponental type wth paamete ( (, {,..., R} then the -tmes Kullback-Leble dvegence (6 I *: fˆ ; ( ; k θ k n( ˆ s asymptotcally dstbuted as a noncental χ wth p degees of feedom and noncentalty paamete I *: fˆ ; ( k θ k ; n( ν ˆ. By assumng the ndependence of all K segments (, we can conclude that f the quey obect coesponds to class ν, then the dstance ( nk ρ, ν s asymptotcally dstbuted as a has asymptotc non-cental χ dstbuton wth χ wth K p degees of feedom. Smlaly, nk ρ(,, ν K p degees of feedom and noncentalty paamete K p s hgh, then, by usng the cental lmt theoem we obtan the nomal dstbuton nk ρν,. If of the dstance (, ρ. p 8nK ρν, + K p N ρ ν, + ; n nk. ( 5

6 3.3. Poposed method Based on the asymptotc dstbuton ( we eplace the step (0 of the ognal DEM to the pocedue of choosng the maxmum lkelhood model. Let's assume that the models dstances,..., l have been checked befoe the l-th step,.e. the ρ,,..., ρ, have been calculated. By assumng the equal po pobablty of each class and l ndependence of the models fom dffeent classes, let's choose the next most pobable model lkelhood method []: l l+ = ν l wth the maxmum l+ ag max f ρ (, W ν, ( {,..., R} {,..., } = whee f (, s the condtonal densty (lkelhood of the dstance (, ρ W ν ρ f the hypothess W ν s tue (the class label of the quey obect s ν. To estmate ths lkelhood, asymptotc dstbuton ( s used. Hence, the lkelhood n ( can be wtten n the followng fom ( ρ(, W ( nk ( ρ(, ρ K p nk ν, f = = ν exp π (8nK ρ ν, + K p 8nK ρν, + K p (3. ( nk = + nk ( ρ(, ρ K p ν, exp ln8nk ρν, K p exp π 8nK ρν, + K p By dvdng ( by a constant ( nk / π l, takng a natual logathm, dvdng by nk / and addng l, expesson ( can be fnally tansfomed to whee l ag mn ϕ µ. (4 µ {,..., R} {,..., } = + = l l p ρ(, ρ µ, = n p ϕ µ + ln4ρµ, +. (5 p 4nK n 4ρµ, + n As the aveage segment's sze s usually much hghe the numbe of paametes smplfed ϕ µ ( ( ρ(, ρ n>> N, then the functon n (5 can be µ, (6 4ρµ, Ths equaton s n good ageement wth the heustc fom the ognal DEM [] - the close ae the dstances ρ (, and ρ µ, and the hghe s the dstance between models µ and, the lowe s ϕ. µ 6

7 Next, the temnaton condton (8 s checked fo the model. If the dstance ρ, l+ s lowe than a l + theshold ρ 0, then the seach pocedue s stopped on the L checks = l+ step. Othewse the model set of pevously checked models and the pocedue (4, (6 s epeated. Let's etun to the ntalzaton of ou method. We would lke to choose the fst model s put nto the l+ to obtan the decson (8 n a shotest (n tems of numbe of calculatons L checks way. Let's maxmze an aveage pobablty to obtan the decson on the second step To estmate the condtonal pobablty Pϕ ( (: Pϕ ν = ( µ {,..., Rρ ρ } Fnally, based on ( one can wte R = ag max P ϕ ν µ W { } = { },..., R R mn ϕ µ ν. (7 µ ν,... R ν mn ϕ Wν n (7 we use agan the asymptotc dstbuton {,... R} ( ρ(, µ ρ, µ ρν, µ Wν ( ρ(, ρ ( ρ(, ρ R µ ν, µ µ mn ϕ µ Wν = P { },... R = ρν, µ ρ, µ { Rρ < } P P, µ ν, µ,...,, µ ρν, µ, µ Wν ( ρ(, µ ρ, µ ρν, µ Wν R { } nk P ϕ = +Φ ν µ mn ϕ µ Wν ρ, µ ρν, µ, (0,... R = whee Φ ( s the cumulatve densty functon of the nomal dstbuton. As a esult, the fst model to check obtaned fom the followng expesson { } R R = ag max +Φ µ,..., R ν= = nk (9 s ρ, µ ρν, µ. ( Thus, the poposed ML-DEM (9, (4, (6, ( s an optmal (maxmal lkelhood geedy algothm fo an appoxmate NN seach wth temnaton condton (8 fo the Kullback-Leble dscmnaton (6, (7. As a matte of fact, ths method can be appled wth an abtay complex smlaty measue. The next secton povdes expemental evdence to suppot ths clam. 4. Expemental study In ths secton we pesent expemental study of poposed ML-DEM n the poblem of face ecognton. 4.. Face ecognton It s equed to assgn a quey mage to one of R classes (pesons specfed by efeence mages { }, {,..., R}. 7

8 We assume that the obect of nteest (face s pelmnaly detected by an abtay algothm (e.g., Vola-Jones method [4]. To mplement hee statstcal appoach descbed n Secton, the mage s dvded nto a egula gd of S S blocks, S ows and S columns (n ou notaton, S S = K = K =... = KR. Next, the hstogam H ( s, s = [ h ( s, s,..., hn ( s, s] of appopate smple featues s sepaately evaluated fo each block ( s, s of the efeence mage. Hee N s the numbe of bns n the hstogam, s { S }, s { S },...,,...,. The most popula mage pont's smple featue s the gadent oentaton (pobably, weghted wth the gadent magntude,.e., H ( s, s s the hstogam of oented gadents (HOG [5]. In ths pape we assume, that each hstogam H ( s, s nomalzed, so that t may be teated as a pobablty dstbuton [5]. The unted vecto [ H (,,..., H (, S, H (,,..., H ( S, S] s amounted the desed descpto of the whole efeence mage. The same pocedue s epeated to evaluate the hstogams H( s, s = [ h( s, s,..., hn( s, s] coespondng to the quey mage. The neghbohood ( ( N of block (, s s contans the cells (, s s fo whch s s <, s s <, whee = const s chosen based on the concete task (usually =0 o = [6]. In such case, the dstance n the neaest neghbo ule (5, (6 wll be calculated wth the mutual algnment of the hstogams n the - neghbohood as follows ρ S S (, = mn ρ ( H H( s s, H ( s + s + whee SS s = s =,,, s, ( ρ ( H H s,, +, + s H s s s an appopate dstance between HOGs. If the Kullback-Leble dscmnaton (7 s appled, the dstance between HOGs can be wtten n the followng fom N hk; ρ KLH, H = hk; ln. (3 = hk; Hee we mssed ndces ( s, s fo smplcty and use the convoluton of the HOGs wth any kenel K N N hk; = Kh, hk; = Kh (4 = = to pevent dvson by zeo n (3 f the hstogam value fo seveal bns s equal to zeo. In the expement we use the conventonal Gaussan Pazen wndow [7]. In ths study we also exploe the homogenety-testng pobablstc neual netwok (HT-PNN showed good accuacy wth HOGs n face ecognton and known to be equvalent to the statstcal appoach f the patten ecognton poblem s efeed as a task of testng fo homogenety of segments [8]: HT PNN ( H, H = N hk; h K; h ln + h ln = h + + K; hk; hk; hk; ρ. (5 8

9 4.. Expemental esults In ths expement FERET dataset was used ( R=43 fontal mages of 994 pesons populate the database (.e. a tanng set, othe 88 fontal photos of the same pesons fomed a test set. The faces wee detected wth the OpenCV lbay. The medan flte wth wndow sze (3x3 was appled to emove nose n detected faces. The faces wee dvded nto 00 fagments ( S = S = 0. The numbe of bns n the HOG N=8. To obtan theshold ρ 0, the FAR s fxed to be β =%. These paametes povde the best accuacy n ou expements. The eo ate obtaned by coss-valdaton wth the NN ule and smlaty measue ( wth Eucldean and the PNNH ( dstances s shown n Table n the fomat aveage eo ate ± ts standad devaton. Table. Eo ate (n % of the NN method ( =0 = Kullback-Leble (3 8.9±.3 7.0±.3 HT-PNN (5 7.8±. 6.6±.3 Fom ths table one could notce that, fst, algnment of HOGs ( wth = mpoves the ecognton accuacy. And, second, we expementally suppots the fact [8] that the eo ate fo the Kullback-Leble dstance (3 exceeds the eo fo the HT-PNN (5. In the next expement we compae the pefomance of the poposed ML-DEM wth an ognal DEM [], bute foce and seveal appoxmate NN methods fom FLANN [5] and NonMetcSpaceLb [9] lbaes showed the best speed, namely. Randomzed kd-tee fom FLANN wth 4 tees [6]. Composte ndex fom FLANN whch combnes the andomzed kd-tees (wth 4 tees and the heachcal k-means tee [5]. 3. Odeng pemutatons (pem-sot fom NonMetcSpaceLb whch s known to decease the ecognton speed fo medum-szed databases (thousands of models [0]. We evaluate the eo ate (n % and the aveage tme (n ms to ecognze one test mage wth a moden laptop (4 coe 7, 6 Gb RAM and Vsual C++ 03 Expess comple (x64 envonment and optmzaton by speed. We exploe an obvous way to mpove pefomance by usng paallel computng [6]. Namely, the whole tanng set was dvded nto T=const non-ovelapped pats and each pat s pocessed n ts own task. All tasks wok n paallel and temnate ght afte any task fnds the soluton. Each task s mplemented as a sepaate thead by usng the Wndows TheadPool API. We analyze both conventonal nonpaallel case (T= and the paallel one (T=8. Afte seveal expements the best (n tems of ecognton speed value of paamete M of ognal DEM (0 was chosen M=64 fo nonpaallel case and M=6 fo paallel one. Paamete E max was chosen to acheve the ecognton 9

10 accuacy whch s not 0.5% less than the accuacy of bute foce (Table. If such accuacy could not be acheved, E max was set to be equal to the count of models assgned to each task. The aveage ecognton tme pe one test mage (n ms fo the Kullback-Leble dscmnaton (3 fo =0 and = s shown n Fg. and Fg., espectvely. 4 Aveage ecognton tme, ms Numbe of paallel theads T Bute foce Randomzed KD tee Composte Pem-sot DEM ML-DEM Fgue : Aveage ecognton tme, Kullback-Leble dscmnaton, =0 Aveage ecognton tme, ms Numbe of paallel theads T Bute foce Randomzed KD tee Composte Pem-sot DEM ML-DEM Fgue : Aveage ecognton tme, Kullback-Leble dscmnaton, = Hee one can notce that modfcatons of kd-tee fom FLANN (andomzed and composte ndces do not show supeo pefomance even ove bute foce as the numbe of models n the database s not vey hgh. Howeve, as t was expected, pem-sot method s chaactezed wth -3.5 tmes lowe ecognton speed n compason wth an exhaustve seach. Moeove, pem-sot seems to be bette than the ognal DEM fo nonpaallel case (T=, though the DEM's paallel mplementaton s a bt bette. The most mpotant concluson hee s that the poposed ML-DEM shows the hghest speed n all expements. To clafy the dffeence n pefomance of the ognal DEM and the poposed ML-DEM, we show the dependence of the eo ate and the numbe of checked models L checks / R 00% on the maxmum numbe of models to be checked E max n Fg. 3 and Fg. 4, espectvely. We descbe hee the case = fo whch the DEM s the best among all othe methods. Fg. 3 demonstates that the speed of convegence to an optmal soluton fo the ML-DEM s much hghe than the same 0

11 ndcato of the DEM. Even when Emax = 0. R we can get an appopate soluton. Eo ate, % DEM ML-DEM 0, 0,3 0,5 0,7 0,9 Emax/R Fgue 3: Dependence of eo ate on E max, Kullback-Leble dscmnaton, = Count of model checks pe database sze, % , 0,3 0,5 0,7 0,9 DEM ML-DEM Emax/R Fgue 4: Count of models checks pe database sze L checks / R 00%, Kullback-Leble dscmnaton, = Fg. 4 poves that the poposed ML-DEM s an optmal geedy algothm n tems of the numbe of calculated dstances L checks. Howeve, addtonal computatons of the ML-DEM (4, (6 whch nclude the calculatons fo evey non pevously checked model, ae qute complex. Hence, the dffeence n pefomance wth the DEM and othe appoxmate NN methods s hgh only fo vey complex smlaty measues (e.g., fo the case of HOG's algnment, =. The aveage ecognton tme fo the HT-PNN (5 fo =0 and = s shown n Fg. 5 and Fg. 6, espectvely. 0 Aveage ecognton tme, ms Numbe of paallel theads T Bute foce Randomzed KD tee Composte Pem-sot DEM ML-DEM Fgue 5: Aveage ecognton tme, HT-PNN, =0

12 Aveage ecognton tme, ms Numbe of paallel theads T Bute foce Randomzed KD tee Composte Pem-sot DEM ML-DEM Fgue 6: Aveage ecognton tme, HT-PNN, = The esults of ths expement ae vey smla to the Kullback-Leble esults (Fg., though the eo ate hee s 0.5- % lowe (see also Table. Howeve, the ognal DEM s hee a bt faste than the pem-sot fo conventonal dstance ( =0, Fg. 5 but s not so effectve fo algnment ( =, Fg. 6. FLANN's kd-tees ae 0-5% faste than the bute foce. And agan, the poposed ML-DEM s the best choce hee especally fo most complex case (T=8, = fo whch only 6 ms (n aveage s necessay to ecognze a quey face wth 93% accuacy. 5. Concluson and futue wok We have shown that usng the asymptotc popetes ( of the Kullback-Leble dscmnaton fo ecognton pecewse-egula obects (5-(7 n the DEM [] gves vey good esults n face ecognton wth medum-szed database, educng the ecognton speed by moe than tmes n compason wth bute foce and by.-.5 tmes n compason wth othe appoxmate NN methods fom FLANN and NonMetcSpaceLb lbaes. We studed the nfluence of vaous dstance paametes (dstance type, neghbohood sze and the maxmal numbe E max of dstances to calculate. In contast to the most popula fast algothms, ou method s not heustc (except the temnaton condton (8. Moeove, t does not buld data stuctue based on an algothmc popetes of appled smlaty measue (e.g., tangle nequalty of Mnkowsk metc n the AESA [4], Begman ball fo Begman dvegences [8]. The poposed ML-DEM s an optmal (maxmum lkelhood geedy method n tems of the numbe of dstance calculatons (see Fg. 3 fo NN ule (5 wth the sum of Kullback-Leble dscmnatons (5, (6. Moeove, as we showed n the last pat of ou expemental study, the ML-DEM can be successfully appled (Fg. 5, 6 wth othe dstances, e.g., not popula but vey accuate HT- PNN (5 [8]. The man decton fo futhe eseach n the ML-DEM can be elated to mpovng the pefomance of each step (4, (6 by ts smplfcaton o the usage of deas of pvot-based appoxmate NN methods [0]. As a matte of fact, t s the man obstacle to use ou method wth vaous smlaty measues. We ae also wokng on exploaton the nfluence of the popula dstances (Eucldean, ch-squaed, Jensen-Shannon, etc. on the pefomance of ou method. Refeences [] S. Theodods, and K. Koutoumbas (eds.. Patten ecognton. Boston: Academc Pess, p.

13 []. Tan, S. Chen, Z. H. Zhou, and F. Zhang. Face ecognton fom a sngle mage pe peson: a suvey. Patten Recognton, 39(9:75 745, 006 [3] C. Shan. Face ecognton and eteval n vdeo // Vdeo Seach and Mnng, Studes n Computatonal Intellgence, 87:35-60, 00 [4] E. Vdal. An algothm fo fndng neaest neghbous n (appoxmately constant aveage tme. Patten ecognton Lettes, 4(3:45 57, 986. [5] M. Mua, and D. G. Lowe. Fast appoxmate neaest neghbos wth automatc algothm confguaton. 4th Intenatonal Confeence on Compute Vson Theoy & Applcatons (VISAPP, : , 009. [6] C. Slpa-Anan, and R. Hatley. Optmsed KD-tees fo fast mage descpto matchng. CVPR, Anchoage, Alaska, USA, pages -8, 008 [7] J. Bes, and D. G. Lowe. Shape ndexng usng appoxmate neaest neghbou seach n hgh dmensonal spaces. CVPR, San Juan, Pueto Rco, pages , 997. [8] L. Cayton. Effcent Begman Range Seach. Advances n Neual Infomaton Pocessng Systems, Y. Bengo, D. Schuumans, J.D. Laffety, C.K.I. Wllams, and A. Culotta (Eds., :43-5, 009 [9] D. Lowe. Dstnctve mage featues fom scale nvaant keyponts. Intenatonal Jounal of Compute Vson, 60(:9 0, 004 [0] E. C. Gonzalez, K. Fgueoa, and G. Navao. Effectve Poxmty Reteval by Odeng Pemutatons. IEEE Tansactons on Patten Analyss and Machne Intellgence, 30(9: , 008. [] A. V. Savchenko, Dected enumeaton method n mage ecognton. Patten Recognton, 45(8:95 96, 0. [] S. Kullback. Infomaton Theoy and Statstcs. Mneola, N.Y: Dove Publcatons, 997. [3] P. Pandon, and M.Vettel. Appoxmaton and compesson of pecewse smooth functons. Phlosophcal Tansactons of the Royal Socety, 357(760:573-59, 999. [4] P. Vola, and M. Jones. Rapd obect detecton usng a boosted cascade of smple featues. CVPR, Kaua, HI, USA, pages 5 58, 00. [5] N. Dalal, and B. Tggs. Hstogams of oented gadents fo human detecton. CVPR, San Dego, CA, USA, pages , 005. [6] A. V. Savchenko. Real-Tme Image Recognton wth the Paallel Dected Enumeaton Method. The 9th ICVS (Intenatonal Confeence on Vson Systems, M. Chen, B. Lebe, and B. Neumann (Eds., LNCS, 7963:3-3, 03. [7] D. F. Specht. Pobablstc neual netwoks. Neual netwoks, 3(:09 8, 990. [8] A. V. Savchenko. Pobablstc neual netwok wth homogenety testng n ecognton of dscete pattens set. Neual Netwoks, 46:7 4, 03. [9] L. Boytsov, and N. Blegsakhan. Engneeng Effcent and Effectve Non-Metc Space Lbay. The 6th SISAP (Smlaty Seach and Applcatons, N. Bsaboa, O. Pedea, P. Zezula (Eds., LNCS 899: 80-93, 03 [0] B. Bustos, E. Ch avez, and G. Navao. Pvot selecton technques fo poxmty seachng n metc spaces. Patten Recognton Lettes, 4-4: , 003 3