Probabilistic Lane Tracking in Difficult Road Scenarios Using Stereovision

Transcription

1 Probablsc Lane Trackng n Dffcul Road Scenaros Usng Sereovson Radu Danescu, Sergu Nedevsch Absrac Accurae and robus lane resuls are of grea sgnfcance n any drvng asssance sysem. In order o acheve robusness and accuracy n dffcul scenaros, robablsc esmaon echnques are needed o comensae for he errors n deecon of lane delmng feaures. The aer resens a soluon for lane esmaon n dffcul scenaros based on he arcle flerng framework. The soluon emloys a novel echnque for ch deecon based on fuson of wo sereovson-based cues, a novel mehod for arcle measuremen and weghng usng mulle lane delmng cues exraced by grayscale and sereo daa rocessng, and a novel mehod for decdng uon he valdy of he lane esmaon resuls. Inalzaon samles are used for unform handlng of he road dsconnues, elmnang he need for exlc rack nalzaon. The resuled soluon has roven o be a relable and fas lane deecor for dffcul scenaros. Index erms-lane deecon, rackng, arcle flerng, cue fuson, sereovson L I. INTRODUCTION ane/road deecon has been a ferle research feld for decades, due o he grea sgnfcance of accurae and robus road descron resuls n any drvng asssance sysem. The algorhms have become ncreasngly comlex, as he argeed scenaros became ncreasngly dffcul. From he hghway scenaro, he lane deecon sysems moved o cy and counry roads. Wh hs move, he nal emhass on lane delmng feaures such as lane markngs was relaced by he emhass on model arameers esmaon echnques, whch use sac and dynamc knowledge-based robablsc consrans o counerac ossble nosy feaures and smooh he resul. These consrans lead o robablsc reasonng n he form of rackng, radonally acheved by he use of he Kalman fler. The use of Kalman fler rackng has he advanage of reducng he search sace, elmnang he deecon oulers, and smoohng of he resul. The feaures ha make he Kalman fler soluons smooh and effcen are he very feaures ha cause roblems when he road s no connuous. Shar urns, lane changes, aycal road geomeres ose roblems o a racker ha reresens he lane robably densy as a Gaussan R. Danescu and S. Nedevsch are wh he Techncal Unversy of Cluj Naoca, Romana, e-mal: Radu.Danescu@cs.ucluj.ro, Sergu.Nedevsch@cs.ucluj.ro, Dearmen address: Comuer Scence Dearmen, Sr. C. Dacovcu nr. 15, 4 2 Cluj Naoca, Romana, Phone: funcons, and he reducon of he search sace around he as resuls makes dffcul o handle new hyoheses, and causes deecon errors o accumulae, f he search regons are drawn owards false delmers. Parcle flerng s a novel echnology for robablybased rackng, allowng mulle hyoheses rackng, smle measuremen, and faser handlng of road dsconnues. Ths aer descrbes a lane deecon sysem ha combnes he advanage of arcle flerng, sereovson and grayscale mage rocessng n order o acheve robus lane esmaon resuls n dffcul scenaros of cy, hghway and counry roads. II. PROBABILISTIC FOUNDATIONS OF LANE TRACKING Whle here s no unversal defnon of rackng, we can regard as he rocess of reasonng abou he sae of a me evolvng eny gven a sequence of observaons. In arcular, lane rackng can be defned as he rocess of reasonng abou he oson and geomery of he lane gven a sequence of mage-derved feaure ses. The goal of rackng as robablsc nference s o evaluae P Y y,..., Y y ), ha s, o comue he condonal robably densy of he sae gven he sequence of measuremens from he as and curren frame. Due o he fac ha he rackng rocess mus delver resul a each frame, and o he fac ha a racker should be able o funcon n mosly he same way for an ndefne erod of me, he rocess of esmaon of P Y y,..., Y y ) has o be wren n a recursve manner, such ha he resuls of he as frames can be reused n he esmaon for he curren frame. In order o acheve hs, he followng conces are used: Dynamc model: P ), he robably of reachng 1 some value of he random varable gven he as sae - 1, under he assumon ha only he mmedae as maers. Predcon: comuaon of he condonal robably densy of he curren sae gven he as sequence of measuremens, P Y y,..., Y y ). Gven he 1 1 smlfcaon assumon ha only he mmedae as maers, he redcon robably values can be comued recursvely, gven he as resuls and he dynamc model:

2 P P y,..., y 1 ) 1 ) P 1 y,..., y 1 ) d Daa assocaon: A each frame here may be several measuremens avalable, and no all of hem are equally useful. Denong by y r he r-h measuremen of he frame, he robably of hs measuremen beng useful s exressed r 1 as P Y y y,... y ). If each measuremen s condonally ndeenden of he ohers he measuremen ndeendence assumon s aken), he usefulness of each measuremen can be comued as: P Y P Y y r y y,... y r 1 ) ) P y,..., y 1 1 ) d Sae udae: he sae robably densy P Y y,..., Y y ), he end resul of he rackng rocess, s comued usng Bayes rule. P y,..., y ) P y ) P P y ) P y,..., y y,..., y 1 1 ) ) d The equaons of rackng as robablsc nference are comlex o aly n he general case. Even more, he robably denses nvolved mossble o reresen analycally mos of he me, and herefore are aroxmaed. Aroxmang means eher coercng hem o a known robably densy funcon, such as a Gaussan, or by mananng a dscree numercal reresenaon hroughou he whole rocess. The Gaussan reresenaon leads o he well-known Kalman fler soluons, and he reresenaon as dscree samles leads o he arcle flerng soluons. III. PARTICLE FILTERING A raccal aroach o rackng general robably densy funcons, arcle flerng s descrbed n [3]. Insead of ryng o aroxmae an unknown funcon analycally, her sysem uses N dscree values called samles or arcles. A each gven me, a arcle s defned by a value weghs beng 1. 1) 2) 3) x and a weghπ, he sum of all The roblem of rackng becomes he roblem of evaluang he values and he weghs, gven a dynamc model and an observaon densy funcon. For algorhm omzaon uroses, a arameer s added o he each arcle, changng he arcle reresenaon o { x, π,, 1... N}. Ths arameer s defned as he c sum of he weghs of each arcle from 1 o a cumulave hsogram). Each eraon of he CONDENSATION algorhm has he am of evaluang a new se of arcles, gven he revous se, he dynamc model and he measuremens. The frs se of he algorhm s resamlng. A weghed samle se s ransformed no a new se of samles, of equal wegh bu uneven concenraon hrough he doman of values of x. Ths s acheved by erformng N random draws from he arcle se, usng he arcle weghs as robables for arcle selecon. A arcle havng a larger wegh wll be seleced several mes, whle a arcle havng a low wegh may no be seleced a all. The new se of weghless arcles and he weghed se aroxmae he same densy funcon. Fg. 2. Same robably densy funcon, aroxmaed by weghed and weghless arcles Predcon s he nex se of he CONDENSATION algorhm. In a general form, hs s acheved by samlng from he dynamc model densy funcon. Ths funcon descrbes he lkelhood of each ossble curren sae gven he assumon ha he as sae s descrbed by he value of he weghless arcle. A more ragmac aroach s o assume ha he new sae s derved from he as sae arly by a deermnsc rocess, descrbed by a funcon or a lnear ransformaon, and arly by a random facor. Each weghless arcle resuled from he resamlng se s subjeced o a deermnsc ransformaon, whch wll ake no accoun he sae ranson equaons of he sysem, and a sochasc dffuson whch wll accoun for he random evens ha may change he sae. Fg. 1. Analogy beween a robably densy funcon and a se of weghed samles

3 Fg. 3. Deermnsc drf usng weghless arcles Fg. 4. Sochasc dffuson usng weghless arcles. The fnal se of he algorhm s he measuremen/udae rocess. In he general formulaon of he rackng roblem as robablsc nference, udang means alyng Bayes rule o ge he oseror robably densy gven he ror and he measuremen. The ror sae robably densy s a hs on comleely encoded n he dsrbuon of he weghless arcles of value hrough he doman of ossble sae values. The oseror robably densy funcon s obaned by smly weghng he arcles usng he lkelhood of observaon, y x x ). Several cues can be combned n hs se by mullcaon, usng he cue condonal ndeendence assumon, f alcable. Fg. 5. Weghless arcles are weghed by measuremen IV. RELATED WORK Lane esmaon hrough Kalman flerng was oneered by Dckmanns [1], and snce hen many researchers have devsed workng soluons, such as [2][7]. The Kalman flerbased lane rackng reles on he model-based redcon for esablshng search regons for deecon, and uses he deecon resuls o udae he sae. Ths aroach execs a connuously varyng road suaon, and he dsconnues are usually handled by renalzng he rackng rocess. The soluon resened n [6] handles some arcular case of road dsconnues by usng wo nsances of he road model, bu s clear ha he Kalman fler s no he bes choce for rackng dsconnuous roads. A shf owards arcle flerng for lane esmaon s currenly akng lace. A arcle-based lane soluon usually sars wh arcle samlng, followed by drfng and measuremen. The measuremen se s consderably smler, n comarson o he Kalman fler, because usually consss of a comarson beween he arcle and he mage daa, from whch a wegh s derved, and herefore no comlex deecon algorhms are requred. However, he measuremen se s execued for each arcle, so he smlcy s essenal for adequae me erformance. [1] resens a lane deecor based on a condensaon framework, whch uses lane markng ons as measuremen feaures. Each on n he mage receves a score based on he dsance o he neares lane markng, and hese scores are used o comue he machng score of each arcle. The sysem uses aroned samlng wo-se samlng and measuremen usng subses of he sae sace, achevng a mulresoluon effec), morance samlng, and nalzaon samles comleely random samles from he whole arameer sace) whch coe faser wh lane dsconnues. In [4] we fnd a lane deecon sysem ha uses he arcle flerng framework o fuse mulle mage cues color, edges, Lalacan of Gaussan). For each cue a comarson mehod beween mage daa and he arcle s desgned, he lkelhood s comued, and hen he lkelhoods are combned by mullcaon. Ths soluon also uses nalzaon samles for faser lane relocaon, and addonal samlng around he bes weghed arcles for mrovemen of accuracy. The much smler way n whch a arcle fler handles he measuremen nformaon allows he use of a wder range of cues. Such s he case of he lane deecor for counry roads, resened n [5], where he mage sace s dvded no road and non-road areas and each xel n hese areas conrbue o he fnal wegh by s nensy, color, edge and exure nformaon. The lkelhood of each feaure value o belong o eher road or off-road areas s comued usng raned hsograms, hus allowng a non-gaussan, mulmodal robably densy no only for he lane sae, bu also for he measuremen. The work resened n [11] also shows he value of he arcle flerng echnque for heerogeneous cue fuson, when mage nformaon s fused wh GPS and ma nformaon for long dsance lane esmaon. In [12], he auhors descrbe a sysem ha uses a hybrd aroach, combnng lane border hyoheses generaed usng a RANSAC ye algorhm wh hyoheses from a arcle fler, and hen usng furher robablsc

4 reasonng o choose he bes border ar o delm he lane. V. SOLUTION OUTLINE The sysem connuously evaluaes he sae of he lane by means of a se of arcles. There s no nalzaon hase herefore each cycle s run exacly n he same way, as deced n fgure 6. The cycle sars wh arcle resamlng, whch s done arally from he revous arcle dsrbuon and arly from a generc dsrbuon ha covers all lane geomeres, n order o cover he ossble dsconnues ha may arse. The deermnsc drf s aled o all arcles, akng no accoun he ego moon arameers such as seed, yaw rae and frame mesams, and hen sochasc dffuson wll aler each arcle n a random way. Pch deecon s done ndeendenly of he arcle sysem, usng a robablsc aroach. The value of he deeced ch s se o each arcle. The ch value s also used o selec he road feaures, whch are hen used o wegh he arcles. A valdaon se ensures ha he arcles are locked on a lane, and f hs se succeeds a lane reresenaon s esmaed. Fg. 6. Lane deecon algorhm oulne A. The Lane Parcles VI. ALGORITHM DESCRIPTION The lane sae robably densy s descrbed a a gven me by a se of N weghed arcles x ) { x,, 1... N}. The arcle value x s π a lane sae hyohess, n he form of a lane descron vecor. The coordnae sysem ha s used has he on of orgn on he ground n fron of he ego vehcle, cenered relavely o he wdh of he car. The axs s osve owards he rgh of he vehcle, he Y axs s osve owards he ground, and he Z axs s osve along he forward drecon. The lane s a surface srechng forward, bounded by wo delmng curves. The coordnae of he delmng curves deends on he lane arameers, he chosen dsance Z and he delmer ye lef or rgh). h Z, ) We ll denoe he above equaon he horzonal rofle of he lane. The lane arameers ha affec he funcon h wll be denoed as horzonal rofle arameers such as he horzonal curvaure). In he same way we can descrbe he varaon of he Y coordnae of each of he delmers, wh he equaon of he vercal rofle of he lane. Y v Z, ) The lane rackng sysem was desgned n a modular fashon, he equaons for he vercal and horzonal rofle beng easly confgurable. The measuremen funcon s ndeenden on he 3D model, as long as ses of 3D ons for he delmers are avalable. We have found ha for he majory of cases he followng se of arameers was suffcen: W lane wdh CH horzonal curvaure C V vercal curvaure C laeral offse x α ch angle γ roll angle ψ yawangle Due o he confgurable naure of he sysem, we have been able o exermen wh several oher models and arameer ses. A model ha ncluded a wdh varaon arameer has been successfully esed n hghway scenaros he resuls secon ncludes he ess done wh hs model), bu he smler model descrbed above has roven o be more relable n urban scenaros. A que owerful argumen agans he use of a very comlex lane reresenaon model s ha he vsbly range s que lmed due o he camera focal dsance and o he comlexy of he cy raffc. B. Predcon Before redcon can be aled, he as sae descrbed by he arcle se has o be resamled no arcles of equal wegh. A fracon R.1 N of he arcles wll be seleced from a unform robably dsrbuon sannng he whole range of ossble lane arameers. These arcles

5 accoun for he robably ha he currenly racked lane can be erroneous, or ha a beer lane canddae aears, such as n he case of lane change, or road forkng. Each arcle s ransformed va redcon, acheved by alyng he followng equaon: ˆ 4) x A x + B u + w s 2 s A 1 s B u c 1 s 1 s The marx A s he lnear ransformaon ha encodes he way he lane evolves n me n he absence of any nu from he drver, and B s he marx ha relaes he drver nu o he lane evoluon. The nu consss of c, he curvaure of he vehcle s rajecory, derved from he yaw rae. Marces A and B deend on he sace s raveled by he vehcle beween measuremens. The ar A xˆ + B u s he deermnsc ar of he 1 redcon, when moon laws are aled and each ossble as lane confguraon s clearly maed no a resen confguraon. Besdes he deermnsc ar, each arcle s oson s alered by a random value w, drawn from a Gaussan dsrbuon of zero mean and covarance marx Q. The covarance marx Q s obaned by scalng a fxed marx Q, calbraed for a me of 1 ms beween frames, wh he acual elased me beween measuremens as he frame rae s no fxed). Ths s naural, as a longer me beween measuremens allows he lane o devae more from he redced confguraon. C. Pch deecon Pch deecon has o be handled somehow dfferenly, ousde of he arcle flerng framework, due o he followng reasons: ch does no rack well s no very redcable), and ch selecon nfluences he measuremen daa, seleced from he 3D se ons knowng he ch angle. Fg. 7. A comlex cy scene wh road, cars and walls, and a sde vew of he reconsruced 3D ons. The ossble doman of ch varaon s hghlghed. Assumng he orgn of he cener of coordnaes s a ground level, mmedaely n he fron of he car, can be assumed ha for abou 1-2 meers, he road seen from one sde wll be a lne assng hrough hs orgn. Ths lne s defned by he ch angle alone. Smlarly o our revous verson of he sereovson-based lane deecon [7], he rocess of ch deecon sars by buldng a olar hsogram ha couns he ons along each lne assng hrough he orgn n he laeral rojecon dsance-hegh lane). The lnes corresond o dscree values of he ch angle, saced a.1 degrees, rangng from -2 o 2 degrees. The algorhm for olar hsogram buldng s he followng: Inalze olar hsogram H ndex) o, for each ndex For each 3D on If dsance )> Lm go o nex on Fnd he angle of he lne assng hrough and he orgn hegh ) an 1 dsance ) α 5) If α o o > 2 or α < 2 Fnd he ndex of ndex α + 2 o.1 go o nex on α n he olar hsogram o Incremen he olar hsogram by a varable amoun akng no accoun he varably of he on densy wh he dsance 2 dsance ) H ndex) H ndex) + 6) K End For The dfference from he revous ch deecon mehod s how we rocess hs olar hsogram. Prevously, we found he maxmum of he hsogram, and hen scan he hsogram boom u unl a value greaer or equal o wo hrds of he maxmum was found. The reasonng behnd hs aroach s ha he road s he frs srucure of subsanal number of ons encounered scannng he scene from boom u, and he subsanal ar s relave o he scene. The roblem wh he revous aroach s ha s hard o jusfy s correcness, and one can magne some rare suaons when would fal. For he curren lane deecon algorhm, a robablsc aroach s used, whch descrbes beer relaons beween he real world and he ossble ch value. Ths means ha for each of he ch canddaes α we ll ndex α α gven aroxmae he robably densy ) ndex he avalable nformaon. There are several assumons ha wll govern he rocess of robably comuaon. The frs assumon s ha ch

6 hsory does no maer, as he ch varaon s due mosly o merfecons n he road surface, merfecons ha are no easly o redc one can argue ha an oscllaory model of he ch varaon can be used, bu would nroduce a consran ha can lead o wrong esmaons f no roerly calbraed). Ths means ha he ch robably densy wll be derved from curren measuremens alone. α y, y,..., y ) α y ) 7) 1 2 The second assumon s ha here s no ror, and herefore he robably densy of he ch varable s drecly rooronal o he measuremen lkelhood. α y ) y α) 8) The measuremen s comosed of wo cues, derved from he followng assumons abou he road ons 3D seen n he laeral rojecon: - The road ons should be nearly collnear - Mos of he ons n he 3D sace are above he road surface The cue corresondng o he frs assumon has he lkelhood drecly rooronal o he olar hsogram H, and he lkelhood for he cue of he second assumon s drecly rooronal o a cumulave hsogram derved from H, CH. y α α ) H ndex) 9) H ndex y α ) CH ndex) CH α 1) ndex α 11) α ndex y ) H ndex) CH ndex) a) b) c) Fg. 8. Combnng he cues for ch: a) olar hsogram, b) cumulave hsogram, c) combnaon The ch canddae wh he hghes lkelhood, corresondng o he hghes value of he hsogram roduc, s chosen as he ch esmae. Fgure 8 shows he effec of ch cue fuson, leadng o a clear maxmum even f he comlex scene leads o mulle srong eaks n he olar hsogram. Anoher esmaon mehod ha was aken no consderaon was he weghed sum of he ch canddaes, bu he maxmum lead o beer resuls. The value vecors x of he redced arcles are modfed by seng her ch feld o he esmaed ch value. Ths ch value s also used for selecng he road ons from he avalable 3D on se, n order o erform he nex sages of he measuremen. D. Mang he arcles n he mage sace Pch deecon s he only ar of he measuremen rocess ha haens n he 3D sace, and for he nex sages, he arcles have o be comared o mage sace measuremen daa. In order o acheve he comarson, from each arcle T value of he form x W, C, C,, α, γ, ψ ) a measuremen sace vecor s generaed, y v,... vp, ul,1,... ul, P, ur,1,..., ur, 1 P H V ) C. The values v are coordnaes of mage lnes and he values u are coordnaes of mage columns. The v values are common o he lef and rgh delmer. P s he number of ons chosen o descrbe each lane delmer n he mage sace. In order o derve y from x, he followng ses have o be aken: a) Generae P ons n he 3D sace, for each lane delmer. The ons wll be equally saced on he dsance axs Z, and her and Y coordnaes laeral and hegh) wll be gven by he horzonal rofle and vercal rofle of he lane. The neares ons wll sar a he dsance Z B, he closes dsance ha allows he road o be vsble o our sysem. The ons wll san a deecon dsance D. The deecon dsance D s varable, and s adjusmen s based on he vehcle s seed. The raonale behnd hs decson s ha a longer dsance s needed f he vehcle ravels a hgh seeds, usually on sragh or low curvaure roads, bu a shorer one s needed a slow seeds o handle narrower curvaures. The dsance D covers a second of vehcle movemen a he curren seed, bu no shorer han 5 m and no longer han 6 m. b) Projec he 3D ons n he mage sace, usng he camera arameers. For each lane delmer, a chan of unevenly saced ons wll be obaned. c) Inersec he segmens obaned by lnkng he rojeced ons, for each sde, wh a se of evenly saced horzonal lnes. The ons of nersecon are he ons ha wll form he arcle reresenaon n he mage sace y. E. The vsual cues Afer he ch angle has been deeced from he 3D on se, a rough aroxmaon of he road geomery can be made based on hs angle alone. The rough aroxmaon s used for road on selecon. The mage edges corresondng o hese 3D ons form our frs measuremen cue. The lane markng edge ons are deeced usng an algorhm based on he red and esed dark-lgh-dark ranson deecon rncle [8]. Besdes lane markngs, anoher hgh rory lane delmng feaure s he curb, and

7 he curbs are deeced usng hegh varaons n a dense sereovson ma [9], and hen convered no mage edges. Due o he fac ha lane markngs and curbs are of smlar rory, hey are nsered n a common secal edge ma, whch reresens he second lane measuremen cue. In order o allow comarson beween he arcles and he measuremen, each cue ma road edges or secal edges) undergoes a Dsance Transformaon. whch are ons belongng o he lane delmers rojecon n he mage sace, and negave ons, whch are ons near he borders, resdng nsde he rojeced lane area fg. 1). Fg. 1. Posve and negave ons. Posves are lane boundary ons, and negaves are ons nsde he lane area. a) b) c) Fg. 9. Vsual nformaon: a) he orgnal grayscale mage, b) he edges corresondng o 3d ons conaned n he road surface and he assocaed dsance ransform mage, c) markngs and curbs, and her assocaed DT mage F. Parcle Weghng by Measuremen Gven he a ror robably densy, encoded n he dsrbuon of he arcle values hroughou he sae sace, s now me o comue he oseror robably densy, whch wll encode all he knowledge abou he lane sae ha we are able o exrac from he sequence of measuremens u o he curren me. Ths s acheved by assgnng new weghs o he redced arcles, weghs rooronal o he measuremen lkelhood gven he sae hyohess. π y x ) 12) x The measuremen lkelhood s obaned by mullyng he edge lkelhood and he markng/curb lkelhood, under he measuremen ndeendence assumon. y x x ) road _ edges x x ) 13) mark _ curb x x ) In order o comue he lkelhood of he wo measuremen cues, a dsance beween he lane sae hyohess and he measuremen has o be comued. The dsance ransformaon of he wo edge mages becomes now very helful. Ideally, lane hyohess boundares rojecons n he mage sace have o f exacly on he edges of he vsual cues. Also, he area nsde he hyohec lane rojecon has o be as free of edges as ossble. In order o es hese wo condons, wo ses of ons are used: he osve ons, The osve ons wll generae he osve dsance; hs s obaned by averagng he dsance ransform xel values a hese ons coordnaes. The dsance corresondng o he negave ons s he comlemen of he dsance ransform mage a hese ons coordnaes. The wo dsances are combned by weghed averagng equaon 14). The value of he wegh arameers α and β has been se o 2 and 1, resecvely, hrough ral and error exermens. αd + ) + βd α + β ) ) M M M D 14) Now, for each measuremen M he measuremen lkelhood s comued, usng a Gaussan dsrbuon o relae robably o he dsance beween he redcon and he vsual daa. 2 D M 1 2 2σ M M x ) e σ M 2π x 15) Each arcle wll receve as wegh he roduc of he wo lkelhoods. A hs se he arcles ha show a degenerae lane, such as a lane ha s oo narrow, oo wde, or oo far from he vehcle s oson, wll receve a null wegh, revenng hem for sawnng new canddaes n he nex cycle. The fnal se s o normalze he new weghs so ha her sum s 1, and he sysem s ready o erform a new rackng cycle. G. Lane Valdaon Unlke a Kalman fler lane rackng soluon, he arcle flerng sysem does no need nalzaon or measuremen valdaon before rack udae. The arcles wll evolve freely, evenually cluserng around he bes lane esmae, f he sysem s roerly desgned and he measuremens are relevan. However, he sysem mus know when a vald lane s beng racked, f s o be used for raccal uroses. The frs aem was o analyze he arcle dsrbuon n he sae sace, and valdae he suaon when he arcles were reasonably clusered. However, we have observed ha arcles end o cluser even n he resence of weak measuremens, and hs cluserng does no guaranee he valdy of he fnal esmae. A much more successful soluon s o comare he

8 average wegh of he redced from samled) arcles agans he average wegh of he comleely random arcles ha are added n he samled se. Recallng ha N denoes he oal number of arcles, and R denoes he number of oally random arcles, and he random arcles are nsered a he head of he arcle ls whou alerng he robably densy), a qualy facor s defned as: q N R π R+ 1 R N R) π 1 16) If q s hgher han a hreshold, he lane rack s consdered vald for ouu, and a lane sae vecor wll be esmaed from he arcle se. A hgh qualy facor means ha he vsual cues suor he redced lane n a much hgher degree han some comleely random lane arameers, whch suors he hyohess ha he lane aroxmaed by he arcles s correc agrees wh he observaon). The hreshold ha we found o work bes s 1. If he qualy facor ndcaes a vald lane, he arameers of hs lane are esmaed by a weghed average of he arcle values. Only he arcles havng a hgher han average wegh are consdered for esmaon. VII. TESTS AND RESULTS 1. Comarson wh a Kalman fler soluon The sereovson-based arcle flerng lane deecon sysem has been desgned manly o mrove he handlng of dffcul suaons, when he Kalman fler soluon had sgnfcan roblems. Even f he scenaros osng roblems o a KF soluon can be varous, hey can be summarzed by a sngle erm, dsconnuous road somemes called road sngulary). The mos common suaons ha can be regarded as road dsconnues are: lane aearance and dsaearance, lane change maneuvers, lane forkng/jonng, shar changes of drecon, shar changes of curvaure, and emorary sensor falure due o nernal or exernal condons he mos ofen roblem s mage sauraon). A Kalman fler soluon has roblems wh road dsconnues due o he followng characerscs: - There s only one ossble lane confguraon ha s racked a one momen n me - The curren sae s used o redc search areas for he nex deecon, a feaure whch dros all measuremens ha ndcae a road dsconnuy - The sysem requres me o dro a rack and me o nalze a new rack - Inalzng a new rack means runnng deecon algorhms for he whole mage, whou he benef of a reduced search regon We have esed he arcle flerng soluon n scenaros conanng he secfed roblems, and he sysem has shown he followng behavor: 1. Lane aearance and dsaearance: due o he fac ha here s no deecon n he classcal sense, no addonal me s needed o sar or dro a rack. The arcles wll cluser around he bes lane vsual nformaon, and he ouu s valdaed afer 2-3 frames. 2. In lane changng maneuvers here are wo asecs of our algorhm ha make he ranson as smooh as ossble: he ably o rack mulle hyoheses and he use of random arcles o kee an eye on new racks. The random arcles wll seed a new cluser, and, due o he moon of he vehcle owards he new lane he arcles of he new cluser wll receve ncreasngly more wegh unl he old lane s lef behnd. When he lane change maneuver s comleed, he new lane s already racked. 3. The forkng/jonng suaons are handled n he same way as he lane change maneuvers. The sysem s always ready o rack a lane ha has beer chances of beng he rgh one. 4. Shar changes of curvaure are eher handled by generang he rgh hyohess fas enough o coe wh he change, smlarly o he way suaons 2 and 3 are handled, or, f hs s no ossble due o he severy of condons, by fas recovery once he dsconnuy has been assed, n eher case he suaon of false esmaon beng avoded. 5. Due o he fac ha here s no rack rese n he arcle fler sysem, sensor falures are reaed unformly by he racker. The arcles wll begn o sread as long as here s no nformaon o cluser hem, and when he sensor goes back onlne he arcles wll begn cluserng agan. If durng hs me hey sll descrbe a vald lane or no he lane valdaon sysem wll decde, ndeendenly of he rackng rocess self. A dynamc qualave comarson beween he behavor of he mehod descrbed n hs aer and a Kalman fler soluon s rovded n he move fle f-kf.av. In he lef half of he frame he arcle fler soluon s dslayed, and n he rgh half one can see he resuls of he Kalman fler. Fg. 11. Samles of sde by sde comarson. Lef arcle fler soluon, Rgh Kalman fler soluon.

9 2. Pch angle evaluaon In order o evaluae he resuls of he ch deecon mehod ha we roosed n he aer, a sequence of mages of hgh ch varaon was seleced. The road geomery forces he vehcle o change he ch n he range of -3 o +4 degrees. Because here s no ground ruh for he ch value, we have chosen o comare he ch resuls wh he ch esmaed by smle averagng of he heghs of he ons drecly n fron of he vehcle, n a narrow 3D wndow 1 meer wde, 7 meers long). Due o he fac ha we ensured he sequence o be obsacle-free n he seleced wndow, he 3D ons locaed here are mosly) n he road lane. The ch deecon ha we wsh o evaluae does no benef from he fac ha he road s obsacle-free, because he 3D ons used n he algorhm are no resrced o a narrow wndow, and on he sdes of he road here are leny of obsacle feaures. The grah n fgure 12 shows he wo ch values n degrees) agans he frame number. The ch deecon sysem resuls are shown wh doed lne, and he comarson ground ruh s shown wh connuous lne. The dfference beween he wo ch values s also shown. From he grah, s clear ha he ch deecon algorhm, whch works on an unresrced se of daa, follows closely he ch value obaned from he resrced daa se. The errors are whn he uncerany gven by he errors of sereo reconsrucon, and can affec eher our ch esmaon or he ground ruh. The behavor of he ch esmaor can be vewed n he fle chng.av, where he sde rojecon of he scene 3D ons and lane surface) s suermosed on he ersecve mage. The horzonal rofle f s no erfec, as he lane delmers are oor and he wdh of he lane s hgher han our acceance hreshold 6 meers). laeral offse and yaw angle he arameers needed for a o vew reresenaon of he lane). The valdaon of he lane deecon s shown n he grahs as a bnary sgnal, hgh meanng ha he lane s vald. The number of vald frames s 2585, ndcang a deecon rae of %. The sequence and he deecon resuls, n ersecve rojecon and brd-eye vew, can be seen n he fle hghway.av. The urle ons n he brd-eye vew are he 3D ons rovded by sereovson ha are marked as road ons by flerng agans he deeced ch and a Canny edge deecor. Fg. 13. Hghway behavor: wdh mm) versus frame number. Fg. 14. Hghway behavor: horzonal curvaure m -1 ) versus frame number. Fg. 12. Pch angle comarson. 3. Hghway erformance evaluaon The erformance of he lane deecon sysem on hghways s evaluaed usng a sequence of 2644 frames, acqured a abou 1 frames er second he acquson frame rae s no consan, bu each frame s mesamed, and he algorhms are able o handle he varable frame rae), whch means abou 4.4 mnues of drvng. The sequence conans lane changes, hghway exs and reenry, and mared vsbly due o ran and wndsheld wers. Fgures 13 o 16 show he evoluon of some of he lane arameers wdh, curvaure, Fg. 15. Hghway behavor: laeral offse dsance of he vehcle from lane cener, n mm) versus frame number.

10 frame number Fg. 16. Hghway behavor: yaw angle n degrees) versus frame number. 4. Urban erformance evaluaon The erformance of he lane deecon sysem n he cy s evaluaed usng a sequence of 1763 frames, acqured a abou 1 frames er second varable), whch means abou 3 mnues of drvng. The sequence conans lane changes, lane forkng and jonng, assng hrough a unnel and assng hrough nersecons. Fgures 17 o 2 show he evoluon of some of he lane arameers wdh, curvaure, laeral offse and yaw angle). The valdaon of he lane deecon s also shown n he grahs. The number of vald frames s 1559, ndcang a deecon rae of %. The resence of a low vsbly unnel n he sequence s he man reason why he deecon rae s so low. The sequence and he deecon resuls, n ersecve rojecon and brd-eye vew, can be seen n he fle urban.av. A crowded urban sequence, wh oor qualy lane delmers, leny of obsacles, drvng on he lane border and so on s avalable for qualave analyss only n he fle crowded.av. Fg. 19. Urban behavor: laeral offse dsance of he vehcle from lane cener, n mm) versus frame number Fg. 2. Urban behavor: yaw angle n degrees) versus frame number 5. Exermenal esng lnear varable wdh model Due o he hgh adaably of he arcle fler framework, new lane models can be easly red. Inserng a wdh varaon arameer whch descrbes he ncrease/decrease of wdh wh he longudnal dsance, requred only o change he funcon ha comues he laeral coordnaes ) of he lane delmers wh he dsance Z). The resuled exermenal sysem was esed on he hghway sequence, a he ex/reenry momens, when he wdh varaon was more obvous. The resuls can be shown n he fle varwdh.av. A grah comarng he esmaed wdh varaon wh a dfferenaon of he esmaed lane wdh agans he raveled sace s shown n fgure J. The modeled wdh varaon s he smooher sgnal. Fg. 17. Urban behavor: wdh mm) versus frame number Fg. 21. Comarson beween wdh varaon esmaed by lane deecon, and he dfferenaon of esmaed wdh agans raveled sace Fg. 18. Urban behavor: horzonal curvaure m -1 ) versus 6. Tme erformance

11 The me erformance has been evaluaed on an Inel Core2 Duo CPU, a 2 GHz, usng a sngle hread. The lane deecon me has a fxed ar, ndeenden on he number of arcles, amounng o 9.6 ms, and a me er rocessed arcle of.75 ms. Our 2 arcle soluon akes a oal of 11 ms o comlee. Noe: The move fles showng he es resuls can be also downloaded from h://users.ucluj.ro/~rdanescu/lane/lane_eval.hm. roc. of IEEE Inellgen Transoraon Sysems Conference, 27, Seale, USA [1] B. Souhall, C.J. Taylor, Sochasc road shae esmaon, n roc. of IEEE Inernaonal Conference on Comuer Vson, 21, Vancouver, Canada [11] P. Smuda, R. Schweger, H. Neumann, W. Rer, Mulle Cue Daa Fuson wh Parcle Flers for Road Course Deecon n Vson Sysems, n roc. of IEEE Inellgen Vehcles Symosum, 26, Tokyo, Jaan [12] Z. Km,"Robus Lane Deecon and Trackng n Challengng Scenaros," IEEE Trans. on Inellgen Transoraon Sysems, vol. 9, no. 1, , 28 VIII. CONCLUSION AND FUTURE WORK We have resened a sysem ha uses he advanages of sereovson and grayscale mage rocessng hrough a arcle flerng framework, n order o robusly deec he lanes n dffcul condons. The sysem does no use deecon n he classcal sense, here s no rack nalzaon or rack loss, and hus he rocessng me s ke consan, regardless of scenaro. The sysem shows remarkable sably when he condons are favorable, bu grea caably of adaaon when condons change. Fuure work wll nclude ncreasng he accuracy of he esmaed arameers usng more measuremen cues lke mage graden orenaon) or a mulresoluon aroach, and rackng of he sde lanes. Trackng he sde lanes wll rovde he addonal benef of reducng he deecon falure me n he case of lane changes. Due o he fac ha he descrbed mehod s relavely model-ndeenden, exermens wh several models wll be carred ou o fnd he bes comromse beween generaly and sably. REFERENCES [1] E.D. Dckmanns, B.D. Myslwez, Recursve 3-d road and relave ego-sae recognon, IEEE Transacons on Paern Analyss and Machne Inellgence, vol. 14, no.2, , 1992 [2] R. Aufrere, R. Chaus, F. Chausse, A model-drven aroach for real-me road recognon, Machne Vson and Alcaons, Srnger-Verlag 21 [3] M. Isard, A. Blake, CONDENSATION condonal densy roagaon for vsual rackng, Inernaonal Journal of Comuer Vson, vol. 29, nr. 1,. 5-28, 1998 [4] K. Macek, B. Wllams, S. Kolsk, R. Segwar, A Lane Deecon Vson Module for Drver Asssance, n roc. of IEEE/APS Conference on Mecharoncs and Robocs, 24 [5] U. Franke, H. Loose, C. Knoeel, Lane Recognon on Counry Roads, n roc. of IEEE Inellgen Vehcles Symosum, 27, Isanbul, Turkey [6] R. Labayrade, J. Doure, D. Auber, A Mul-Model Lane Deecor ha Handles Road Sngulares, n roc. of IEEE Inellgen Transoraon Sysems Conference, 26, Torono, Canada [7] S. Nedevsch, R..Schmd, T. Graf, R. Danescu, D. Frenu, T. Mara, F. Onga, C. Pocol, 3D Lane Deecon Sysem Based on Sereovson, n roc. of IEEE Inellgen Transoraon Sysems Conference, 24, Washngon, USA [8] R. Danescu, S. Nedevsch, M.M. Menecke, T.B. To, Lane Geomery Esmaon n Urban Envronmens Usng a Sereovson Sysem, n roc. of IEEE Inellgen Transoraon Sysems Conference, 27, Seale, USA [9] F. Onga, S. Nedevsch, M.M. Menecke, T.B. To, Road Surface and Obsacle Deecon Based on Elevaon Mas from Dense Sereo, n