Saliency and Active Contour based Traffic Sign Detection

Transcription

1 ISSN , England, UK Journal of Informaton and Computng Scence Vol. 7, No. 3, 01, pp Salency and ctve Contour based Traffc Sgn Detecton Shangbng Gao 1 and Yunyang Yan 1 1 The School Computer Engneerng, Huayn Insttute of Technology, Hua an, 3003, P.R.Chna (Receved January 5, 01, accepted May 0, 01) bstract. In ths paper, we propose a new approach to detect salent traffc sgns, whch s based on vsual salency and auto-generated strokes for mage segmentaton. The proposed algorthm deals wth two tasks on detectng traffc sgns: auto-locaton and auto extracton. Frstly, nspred by recent work of vsual salency detecton, we obtan the locaton of traffc sgns n a natural mage by mult-scale prncple component analyss (MPC). Secondly, n order to extract traffc sgns, auto-generated strokes are used nstead of drawng the strokes by the users, the sgn board area s extracted usng localzng Regon-Based ctve Contour. Extensve experments on publc datasets show that our approach outperforms state-of-the-art methods remarkably n salent traffc sgn detecton. Moreover, the proposed detecton method has hgher accurate rate and robustness to dfferent natural scenes. Keywords: Multscale; Image segmentaton; actve contour; traffc sgn 1. Introducton utomatc extracton of traffc sgn s a hot topc of ntellgent transportaton systems [1]. It has been wdely appled n many applcatons such as: drvng safety and automatc vehcle gudance etc. Traffc sgn detecton has two key goals: locaton and extracton. Because of complex traffc sgn mages, color-based and shape-based methods can not rapdly fnd the obect and deal wth llumnaton, vewpont change. For example, Loy et al. [] appled the radal symmetry algorthm [3] to detect regular polygons, whch needs to set parameters n advance and unft for all the sgns. Moreover, the fully automatc segmentaton of traffc sgn from the background s very dffcult; and there s not a mature and ntegrated method for traffc sgn detecton system untl now. Therefore, to develop a real-tme and accurate road traffc sgn detecton system has been a challengng task n computer vson. Humans can dentfy salent areas n ther vsual felds wth surprsng speed and accuracy before performng actual recognton. Computatonally detectng such salent mage regons remans a sgnfcant goal, as t allows preferental allocaton of computatonal resources n subsequent mage analyss and synthess. number of very nsprng and mature salency models have been recently ntroduced n the lterature. Itt et al. [4] ntroduced a salency model whch was bologcally nspred. Specfcally, they proposed the use of a set of feature maps from three complementary channels as ntensty, color, and orentaton. The normalzed feature maps from each channel were then lnearly combned to generate the overall salency map. Based on Itt s algorthm, many salency models have appeared, such as, SR [5], Duan s method[6]. Our work s nspred by [6] whch defned the salency n three elements: dssmlarty, spatal dstance and central bas. Motvated by Duan s model, a new salency model s proposed based on mult-scale prncple component analyss (MPC). We use frst two nformaton to measurng the patch s salency value. The central bas, whch based on the prncple that domnant obects often rase to the center of the mage, s proposed by [7]. Ths underlyng hypothess brngs two problems. Frst, background near the center of mage maybe more salent than the foreground whch s far away from the center. Second, for a salent obect, the part near the center s more salent than that far away from the center. To dmnsh ths effect, we gve up the central bas and use the multple scales to decrease the salency of background patches, mprovng the contrast between salent and non-salent. We get traffc sgn s locaton usng the proposed algorthm. Based on an nteractve segmentaton method proposed by Shawn et al. [8], whch s sem-automatc mage segmentaton. In order to extract traffc Correspondng author. Tel.: E-mal address: luxaofen_00@16.com Publshed by World cademc Press, World cademc Unon

2 36 Shangbng Gao, et al: Salency and ctve Contour based Traffc Sgn Detecton sgn regon, we acheve automatc segmentaton by auto-generatng strokes after locaton. The proposed algorthm can make salent regon more sgnfcant than Duan s algorthm, as shown Fg.1, and has good robustness to adverse envronment. (a) (b) (c) (d) (e) (f) Fgure 1. n example of traffc sgn detecton. (a) Orgnal mage; (b)salency map by Duan s method; (c) Salency map by MPC; (d) Bnary map; (e) uto-generated strokes; (f) Extracton result. Ths paper s organzed as follows: secton descrbes the proposed traffc sgn detecton method; secton 3 performs extensve experments to verfy the proposed method; and secton 4 shows the concluson of ths paper.. THE PROPOSED TRFFIC SIGNS DETECTION METHOD Vsual salency often appears n all knds of vsual scales, dfferent regons also have dfferent vsual attenton. In order to comprehensvely consder localty and ntegrty for the mage salency regon, n ths paper, we present an mproved algorthm (MPC) on the bass of Duan s model to compute traffc sgn salency map. nd then we extract traffc sgn part from orgnal mage usng automatc segmentaton method obtaned by loadng auto-generated strokes..1. Mult-scale PC method Gven an mage I wth dmenson H W, non-overlappng patches wth the sze of n n pxels are L H / n W / n. Denote the patch as p, 1,,, L. drawn from t. The total number of patches s Then each patch s represented as a column vector x of pxel values. The length of the vector s 3k snce the color space has three components. Fnally, we get a sample matrx X [ x1, x,, xl], L s the total number of patches as stated above. To effectvely descrbe patches n a relatvely low dmensonal space, we used an equvalent method to PC to reduce data dmenson. Each column n the matrx X subtracts the average along the columns. Then, we T calculated the co-smlarty matrx ( X X ) / L, so the sze of the matrx s L L.The egenvalues and egenvectors were calculated based on the matrx selected wth ther T egenvector U [ u1, u,, u d ] accordng to the bggest d egenvalues, where u s an egenvector. The sze of the matrxu s d L. New algorthm consders two factors for evaluatng the salency: the dssmlartes of color between mage patches n a reduced dmensonal space, and ther spatal dstance. patch s salent f the color of ts pxels s unque. We should not, however, look at an solated patch, but rather at ts surroundng patches, whch lead to a center-surroundng contrast. Thus, a patch p s consdered salent f the appearance of the patch p s dstnctve wth respect to all other mage patches. Specfcally, let dstcolor ( p, p ) be the dstance between the patches p and p n the reduced dmensonal space. Patch p s consdered salent when dst p, p ) s hgh for. dst color color ( d ( p, p ) u u (1) n1 n The postonal dstance between patches s also an mportant factor. Generally speakng, background patches are lkely to have many smlar patches both near and far-away n the mage. It s n contrast to salent n JIC emal for contrbuton: edtor@c.org.uk

3 Journal of Informaton and Computng Scence, Vol. 7 (01) No.3, pp p s salent when the patches patches that the latter tend to be grouped together. Ths mples that a patch smlar to t are nearby, and t s less salent when the resemblng patches are far away. Let dst ( p, p ) be the Eucldean dstance between the postons of patches p and p, whch s represented by the two centers of patches p and p n the mage, normalzed by the larger mage dmenson. Based on the observatons above we defne a dssmlarty measure between a par of patches p and p as: dssmlarty( p, p dstcolor( p, p ) 1 dst( p, p ) ) Ths dssmlarty measure s proportonal to the dfference n appearance and nverse proportonal to the postonal dstance. To evaluate a patch s unqueness, we can compute the dssmlarty between the patch and all of other patches and take the sum of these dssmlartes as the salency of related patch. In practce, there s no need to ncorporate ts dssmlarty to all other mage patches. It suffces to consder the K most smlar patches that f the most smlar patches are hghly dfferent from p, then clearly all mage patches are hghly dfferent from p. Hence, for every patch p, we search for the K most smlar patches { q }, 1,,, K n the mage, accordng to (). Under ths defnton, our algorthm that measures the salency value from the perspectve of global nformaton and local nformaton s dfferent from global salency detecton [0] and Duan s method [6]. patch p s salent when dssmlarty( p, qk ) s hgh for k [ 1, K]. The salency of patch p s defned as (we choose K 100 n our experments): K 1 S 1 exp{ dssmlarty( p, qk )} (3) K k 1 Based on the observaton that patches n background are lkely to have smlar patches at multple scales, whch s n contrast to more salent patches that could have smlar patches at a few scales but not at all of them(it s equal to the prncple proposed by [10] that salent obect always smaller than the background). Therefore, we wsh to ncorporate multple scales to further decrease the salency of background patches, mprovng the contrast between salent and non-salent regons. In addton, the patch wth large scale can not descrbe the boundary of small salent obect. So we hope to use dfferent scales that large scale to detect the whole nformaton and the small scale to descrbe the salent obect n detals. Last, we comple all salency value nto fnal salency. The results of dfferent scales and the fnal result llustrated n Fg.1. The number of PCs set to 4. a b c d e Fgure.Our method usng dfferent scales. a.nput, b.scale=30, c. sacle=0, d.scale=10, e. fnal result For a patch p of scale r, the salency value accordng to (3) s defned as K r 1 r r S 1 exp{ dssmlarty( p, qk )} K k 1 We consder the scales R r, r,, r }, usng (4) to calculate the salency of patch as { c { 1 M r 1 r r S, S,, S M }. The fnal salency s computng as S 1 r S M r R c (5) JIC emal for subscrpton: publshng@wu.org.uk

4 38 Shangbng Gao, et al: Salency and ctve Contour based Traffc Sgn Detecton.. Traffc sgns extracton Salency map only provdes coarse nformaton about where traffc sgns locate n the orgnal mage. Traffc sgn extracton s also an mportant task and necessary for road traffc sgn recognton work. Image segmentaton can be appled to extract the promnent regons. In very complex mages the fully automatc segmentaton of the obect from the background s very dffcult. In [4], Shawn et.al proposed a natural framework that allows any regon-based segmentaton energy to be reformulated n a local way. For example, we choose the global regon-based energy that uses mean ntenstes s the one proposed by Yezz et al. [9] whch we refer to as mean separaton energy: E MS ( u - v) (6) y where u and v represents the global mean ntenstes of the nteror and exteror regons, respectvely. Optmzng ths energy causes that the nteror and exteror means have the largest dfference possble. B( x, y) s ntroduced to mask local regons. Ths functon B ( x, y) wll be 1 when the pont y s wthn a ball of radus r centered at x, and 0 otherwse. ccordngly, the correspondng F s formed by localzng the global energy wth local mean equvalents as shown: F ( ) MS ux vx (7) We can get the followng local regon-based flow: ( I( y) ux)) ( I ( y) v ( x) B( x, y) ( y) ( x)) ( x) t ) dy ( x) dv( ) (8) y ( x) where u and u v v are the areas of the local nteror and local exteror regons respectvely gven by u B( x, y) H( y) dy (9) y v B( x, y) H (1 ( y)) dy (10) y Two closed curve lnes are marked n an orgnal mage as the obect marker and the background marker respectvely. s shown Fg.1 (e), the green lne s the obect marker and the blue lne s the background marker. 3. Expermental results Our algorthm s mplemented n Matlab 7.0 on a.8-ghz Intel Pentum IV PC. Extensve experments are performed to valdate the proposed method. Frst, we compare the proposed MPC wth Itt s model, SR and Duan s models n salency evaluaton, as shown Fg.3. Fnally, we test the proposed algorthm under dfferent surroundngs, as shown Fg.4. all the mages for test are ganed from webste and publc datasets n our experments, ncludng dfferent scene condtons, dfferent colors and shapes Evaluaton and comparson wth other salency models To compare our results wth [6], we chose 11 as reduced dmenson whch s the best value to maxmze salency predctons. For the patch sze, we choose { 30,0,10 } because better results are easy to obtanng n these values [6]. We obtaned an overall salency map by usng YCbCr color space n all experments. Some vsual results of our algorthm are compared wth state-of-art methods n Fg. 3. Note that our method s much less senstve to background texture, whch s dfferent from Itt s method, Duan s method and SR. However, our vsual salency model can accurately fnd traffc sgn area and keep complete area nformaton, makng the road traffc regon more promnent than other models. 3.. Results by usng the proposed method n dfferent scenes We apply our method to test ts performance on some adverse condtons, such as some traffc sgns affected by llumnaton changes, occluson, rotaton, and shadow, dfferent szes. Fg.4 (a) shows the detecton results usng the proposed model on some unfavorable condtons. Expermental results show the proposed method can make traffc sgn regon popup n an mage of even bad envronment and robust to vewpont change and nsuffcent llumnaton. Fg.4 (b) shows the expermental results of the proposed traffc sgn detecton model n varous suburban and hghway scenes. The smple background on hghway scene can JIC emal for contrbuton: edtor@c.org.uk

5 Journal of Informaton and Computng Scence, Vol. 7 (01) No.3, pp make the detecton more convenent. However, n urban areas and campus as shown Fg.4 (c), there may be some complex vsual obects, such as trees and buldngs, pedestrans and motor vehcles. Traffc sgn detecton becomes more dffcult f only usng color-based and shape-based detecton method. From expermental results, we can see the proposed model s also nvarant to traffc sgn scale change, and our model does not rely on the sze and shape of traffc sgn, and can stll accurately detect traffc sgn area n complex scene. Fgure 3. The comparson of traffc sgns salency maps. Second row: salency maps by Itt s model [4]; Thrd row: salency maps by SR; Fourth row: salency maps by Duan s method; Ffth row: salency maps by MPC model. (a) dverse road condtons (b) Hghway scene (c) Urban area and campus road scenes Fgure 4. Results of traffc sgn detecton usng the proposed model n dfferent condtons. (a) Expermental results of traffc sgns affected by adverse external factors contanng tlted, nsuffcent llumnaton and dustsandstorm weather; (b) Expermental results of hghway sgns n dfferent szes and smple background; (c) Expermental results of road sgns wth complex background n urban area and campus. 4. Concluson JIC emal for subscrpton: publshng@wu.org.uk

6 40 Shangbng Gao, et al: Salency and ctve Contour based Traffc Sgn Detecton In ths paper we ntroduce a multscale detecton method of vsual percepton based on Duan s model to obtan more salent area and lessen redundant nformaton. Frstly, we obtan multscale mages usng PC, then compute salency maps and label n orgnal mages, fnally extract traffc sgns area by automatc mage segmentaton. Through computng auto-generated strokes mage n advance, we can realze automatc segmentaton for traffc sgns. The proposed algorthm could deal wth traffc sgns n the presence of lght, scale and vewpont change, expermental results test our model can acheve a hgh detecton rate. But the proposed algorthm has some lmtatons for qute a few complcated pctures, n the future work, we wll consder mprovng the algorthm, and make t more unversal for all the traffc sgns. cknowledgements Ths work was supported by the grant from Natural Scence Research Proect of Jangsu Provncal Colleges and Unverstes, No. 11KJD50003, Natural Scence Research Key Proect of Jangsu Provncal Colleges and Unverstes, No. 11KJ460001, the Jang Su Qng Lan proect and Hua an produce-learnresearch proects, No. HC01113, HSZ01104 HG HG011041, fund of Hua an 533 proect 5. References [1] Kastrnak, V. and Zervaks, M. and Kalatzaks, K, survey of vdeo processng technques for traffc applcatons, Journal Image and Vson Computng, 1(4)(004), pp [] G. Loy and N. Barnes, Fast shape-based road sgn detecton for a drver assstance system, n Pro. IEEE/RSJ Intellgent Robots and Systems, 1(004),pp [3] G. Loy and. Zelnsky, Fast radal symmetry for detectng ponts of nterest, IEEE Transactons on Pattern nalyss and Machne Intellgence, 5(8) (003), pp [4] L. Itt, C. Koch, and E. Nebur, model of salency-based vsual attenton for rapd scene analyss, IEEE Transactons on Pattern nalyss and Machne Intellgence, 0(11)(1998), pp [5] Hou X.D, Zhang L.Q, Salency detecton: spectral resdual approach, In IEEE Conf Computer Vson and Pattern Recognton, (007),pp.1-8. [6] L. Duan, C.Wu, J. Mao, L. Qng and Y. Fu, Vsual Salency Detecton by Spatally Weghted Dssmlarty, IEEE Conference on Computer Vson and Pattern Recognton (CVPR),(011),pages 1-3. [7] B. W. Tatler, The central fxaton bas n scene vewng: Selectng an optmal vewng poston ndependently of motor bases and mage feature dstrbutons, Journal of Vson, 7(14)(007), pages [8] Shawn Lankton,llen Tannenbaum, Localzng Regon-Based ctve Contours, IEEE Transactons on Image processng, 17(11)(008),pp [9] J.. Yezz,. Tsa, and.wllsky, fully global approach to mage segmentaton va coupled curve evoluton equatons, Journal of Vsual Communcaton and Image Representaton, 13(1) (00),pp JIC emal for contrbuton: edtor@c.org.uk