Fitting: Deformable contours

Size: px

Start display at page:

Download "Fitting: Deformable contours"

Juniper Bryant
5 years ago
Views:

regos, objects, ad shapes. Groupg: Pxels vs.

boudares mage clusters o testy By groupg

accordg to the features ad smlarty metrc we

dges useful sgal to dcate occludg boudares,

Here the raw edge output s ot so bad Images

Jacobs but qute ofte boudares of terest are

overcome some of the mssg ad osy edges usg

Wth votg methods lke the Hough trasform,

1 //0 Recap so far: Groupg ad Fttg Fttg: Deformable cotours Moday, Feb Prof. UT-Aust Goal: move from array of pxel values (or flter outputs to a collecto of regos, objects, ad shapes. Groupg: Pxels vs. regos Fttg: dges vs. boudares mage clusters o testy By groupg pxels based o Gestaltspred attrbutes, we ca map the pxels to a set of regos. mage clusters o color ach rego s cosstet accordg to the features ad smlarty metrc we used to do the clusterg. dges useful sgal to dcate occludg boudares, shape. Here the raw edge output s ot so bad Images from D. Jacobs but qute ofte boudares of terest are fragmeted, ad we have extra clutter edge pots. Fttg: dges vs. boudares Gve a model of terest, we ca overcome some of the mssg ad osy edges usg fttg techques. Wth votg methods lke the Hough trasform, detected pots vote o possble model parameters. Votg wth Hough trasform Hough trasform for fttg les, crcles, arbtrary shapes y (x, y y 0 (x0, y 0 x 0 x mage space b m Hough space I all cases, we kew the explct model to ft. CS376 Lecture 0 K. Grauma

arbtrary shape wth actve deformable cotours Deformable cotours a.k.a. actve cotours, sakes

Terzopoulos, ICCV987] Fgure credt: Yur Boykov Deformab

k.a. actve cotours, sakes Deformable cotours: tuto Gve: tal cotour (model ear desred object

as to be ear mage postos wth hgh gradets, ad satsfy shape prefereces or cotour prors [Sakes:

http://www.healthle.com/blogs/exercse_ftess/uploaded_mages/hadbad-795868.

2 //0 Today Fttg a arbtrary shape wth actve deformable cotours Deformable cotours a.k.a. actve cotours, sakes Gve: tal cotour (model ear desred object [Sakes: Actve cotour models, Kass, Wtk, & Terzopoulos, ICCV987] Fgure credt: Yur Boykov Deformable cotours a.k.a. actve cotours, sakes Deformable cotours: tuto Gve: tal cotour (model ear desred object Goal: evolve the cotour to ft exact object boudary Ma dea: elastc bad s teratvely adjusted so as to be ear mage postos wth hgh gradets, ad satsfy shape prefereces or cotour prors [Sakes: Actve cotour models, Kass, Wtk, & Terzopoulos, ICCV987] Fgure credt: Yur Boykov Image from Deformable cotours vs. Hough Lke geeralzed Hough trasform, useful for shape fttg; but Why do we wat to ft deformable shapes? tal termedate fal Hough Rgd model shape Sgle votg pass ca detect multple staces Deformable cotours Pror o shape types, but shape teratvely adjusted (deforms Requres talzato earby Oe optmzato pass to ft a sgle cotour Some objects have smlar basc form but some varety the cotour shape. CS376 Lecture 0 K. Grauma

//0 Why do we wat to ft deformable shapes?

No-rgd, deformable objects ca chage ther shape over tme, e.g. lps, hads Fgure from Kass et al.

No-rgd, deformable objects ca chage ther shape over tme.

fuctos xteral Iteral Mmzg the eergy fucto xtesos: Trackg Iteractve

represetato of the cotour, cosstg of a lst of d pot postos ( vertces.

we adjust the curret cotour to form the ew cotour at each terato?

Seek ext cofgurato that mmzes that cost fucto.

3 //0 Why do we wat to ft deformable shapes? No-rgd, deformable objects ca chage ther shape over tme, e.g. lps, hads Why do we wat to ft deformable shapes? No-rgd, deformable objects ca chage ther shape over tme, e.g. lps, hads Fgure from Kass et al. 987 Why do we wat to ft deformable shapes? No-rgd, deformable objects ca chage ther shape over tme. Aspects we eed to cosder Represetato of the cotours Defg the eergy fuctos xteral Iteral Mmzg the eergy fucto xtesos: Trackg Iteractve segmetato Fgure credt: Jule Jomer Represetato We ll cosder a dscrete represetato of the cotour, cosstg of a lst of d pot postos ( vertces. ( 0 x, y 0 ( x 9, y9 x, y, for ( 0,,, Fttg deformable cotours How should we adjust the curret cotour to form the ew cotour at each terato? Defe a cost fucto ( eergy fucto that says how good a caddate cofgurato s. Seek ext cofgurato that mmzes that cost fucto. At each terato, we ll have the opto to move each vertex to aother earby locato ( state. tal termedate fal CS376 Lecture 0 K. Grauma 3

//0 ergy fucto The eergy (cost of the curret sake s defed as: teral exteral xteral eergy: tuto Measure how well the

Iteral eergy: ecourage pror shape prefereces: e.g., smoothess, elastcty, partcular kow shape.

A good ft betwee the curret deformable cotour ad the target shape the mage wll yeld a low value for ths cost fucto.

Thk of exteral eergy from mage as gravtatoal pull towards areas of hgh cotrast Gradet mages G x ( x, y ad ( x, y G y

of gradet G x( I Gy( I xteral eergy for the whole curve: exteral Gx( x, y Gy ( x, y 0 Iteral eergy: tuto Iteral eergy:

4 //0 ergy fucto The eergy (cost of the curret sake s defed as: teral exteral xteral eergy: tuto Measure how well the curve matches the mage data Attract the curve toward dfferet mage features dges, les, texture gradet, etc. Iteral eergy: ecourage pror shape prefereces: e.g., smoothess, elastcty, partcular kow shape. xteral eergy ( mage eergy: ecourage cotour to ft o places where mage structures exst, e.g., edges. A good ft betwee the curret deformable cotour ad the target shape the mage wll yeld a low value for ths cost fucto. xteral mage eergy How do edges affect sap of rubber bad? Thk of exteral eergy from mage as gravtatoal pull towards areas of hgh cotrast Gradet mages G x ( x, y ad ( x, y G y xteral eergy at a pot o the curve s: xteral mage eergy exteral ( ( G ( G ( x y Magtude of gradet G x( I Gy( I - (Magtude of gradet G x( I Gy( I xteral eergy for the whole curve: exteral Gx( x, y Gy ( x, y 0 Iteral eergy: tuto Iteral eergy: tuto A pror, we wat to favor smooth shapes, cotours wth low curvature, cotours smlar to a kow shape, etc. to balace what s actually observed (.e., the gradet mage. What are the uderlyg boudares ths fragmeted edge mage? Ad ths oe? CS376 Lecture 0 K. Grauma 4

for the whole relatve curve: to posto---ot spatal mage gradets. teral 0 Why do these reflect teso ad curvature?

5 //0 Iteral eergy For a cotuous curve, a commo teral eergy term s the bedg eergy. At some pot v(s o the curve, ths s: d ds Teso, lastcty d d s teral ( ( s Stffess, Curvature For our dscrete represetato, d ds Iteral eergy ( x, y 0 d ds v ( ( Iteral Note these eergy are dervatves for the whole relatve curve: to posto---ot spatal mage gradets. teral 0 Why do these reflect teso ad curvature? xample: compare curvature curvature( v ( x x x ( y y y (,5 (, Pealzg elastcty Curret elastc eergy defto uses a dscrete estmate of the dervatve: elastc 0 ( x x ( y y 0 (, (3, (, (3, What s the possble problem wth ths defto? Pealzg elastcty Curret elastc eergy defto uses a dscrete estmate of the dervatve: elastc 0 Istead: ( x x ( y y d 0 where d s the average dstace betwee pars of pots updated at each terato. Dealg wth mssg data The prefereces for low-curvature, smoothess help deal wth mssg data: Illusory cotours foud! [Fgure from Kass et al. 987] CS376 Lecture 0 K. Grauma 5

//0 xtedg the teral eergy: capture shape pror If object s some smooth varato o a kow shape, we ca use a term that wll pealze devato from that shape: teral 0 ( ˆ where { ˆ } are the

6 //0 xtedg the teral eergy: capture shape pror If object s some smooth varato o a kow shape, we ca use a term that wll pealze devato from that shape: teral 0 ( ˆ where { ˆ } are the pots of the kow shape. Total eergy: fucto of the weghts exteral teral 0 d teral exteral Gx( x, y Gy ( x, y 0 Fg from Y. Boykov Total eergy: fucto of the weghts e.g., weght cotrols the pealty for teral elastcty large medum small Recap: deformable cotour A smple elastc sake s defed by: A set of pots, A teral eergy term (teso, bedg, plus optoal shape pror A exteral eergy term (gradet-based To use to segmet a object: Italze the vcty of the object Modfy the pots to mmze the eergy Fg from Y. Boykov ergy mmzato Several algorthms have bee proposed to ft deformable cotours. We ll look at two: Greedy search Dyamc programmg (for d sakes ergy mmzato: greedy For each pot, search wdow aroud t ad move to where eergy fucto s mmal Typcal wdow sze, e.g., 5 x 5 pxels Stop whe predefed umber of pots have ot chaged last terato, or after max umber of teratos Note: Covergece ot guarateed Need decet talzato CS376 Lecture 0 K. Grauma 6

7 //0 ergy mmzato Several algorthms have bee proposed to ft deformable cotours. We ll look at two: Greedy search Dyamc programmg (for d sakes v 4 v 3 ergy mmzato: dyamc programmg v 5 v v 6 v Wth ths form of the eergy fucto, we ca mmze usg dyamc programmg, wth the Vterb algorthm. Iterate utl optmal posto for each pot s the ceter of the box,.e., the sake s optmal the local search space costraed by boxes. Fg from Y. Boykov [Am, Weymouth, Ja, 990] ergy mmzato: dyamc programmg Possble because sake eergy ca be rewrtte as a sum of par-wse teracto potetals:,, (, ( Or sum of trple-teracto potetals. (,, (,, Sake eergy: par-wse teractos ( x,, x, y,, y Gx ( x, y Gy ( x, y Re-wrtg the above wth v x, y : (,, G( where ( x x ( y y (,, ( v, v ( v, v3... ( v, v (, G( Ma dea: determe optmal posto (state of predecessor, for each possble posto of self. The backtrack from best state for last vertex. states m vertces ( v, v ( v, v3... ( v, v ( v, v ( v, v3 3( v3, v4 v v v3 v4 ( 0 ( 0 (3 0 ( m 0 Complexty: Vterb algorthm ( ( (3 ( m O( m 3 ( 3 ( 3 (3 ( 4 ( m 3 m 4 ( 4 ( 4 (3 4 ( v 4, v v vs. brute force search? ( ( (3 (m xample adapted from Y. Boykov v 4 v 3 ergy mmzato: dyamc programmg v 5 v v 6 v Wth ths form of the eergy fucto, we ca mmze usg dyamc programmg, wth the Vterb algorthm. Iterate utl optmal posto for each pot s the ceter of the box,.e., the sake s optmal the local search space costraed by boxes. Fg from Y. Boykov [Am, Weymouth, Ja, 990] CS376 Lecture 0 K. Grauma 7

//0 ergy mmzato: dyamc programmg DP ca be appled to optmze a ope eded sake ( v, v ( v, v3... ( v, v For a closed sake, a loop s troduced to the eergy. ( v, v ( v, v3... ( v, v ( v, v 4 3 Work aroud: Fx v ad solve for rest.

Aspects we eed to cosder Represetato of the cotours Defg the eergy fuctos xteral Iteral Mmzg the eergy fucto xtesos: Trackg Iteractve segmetato Trackg va deformable cotours

Repeat, talzg wth most recet frame. Vsual Dyamcs Group, Dept. geerg Scece, Uversty of Oxford.

8 //0 ergy mmzato: dyamc programmg DP ca be appled to optmze a ope eded sake ( v, v ( v, v3... ( v, v For a closed sake, a loop s troduced to the eergy. ( v, v ( v, v3... ( v, v ( v, v 4 3 Work aroud: Fx v ad solve for rest. Fx a termedate ode at ts posto foud (, solve for rest. Aspects we eed to cosder Represetato of the cotours Defg the eergy fuctos xteral Iteral Mmzg the eergy fucto xtesos: Trackg Iteractve segmetato Trackg va deformable cotours Trackg va deformable cotours. Use fal cotour/model extracted at frame t as a tal soluto for frame t+. volve tal cotour to ft exact object boudary at frame t+ 3. Repeat, talzg wth most recet frame. Vsual Dyamcs Group, Dept. geerg Scece, Uversty of Oxford. Trackg Heart Vetrcles (multple frames Applcatos: Traffc motorg Huma-computer teracto Amato Survellace Computer asssted dagoss medcal magg 3D actve cotours Lmtatos May over-smooth the boudary Jörge Ahlberg Caot follow topologcal chages of objects CS376 Lecture 0 K. Grauma 8

//0 Lmtatos xteral eergy: sake does ot really see object boudares the mage uless t gets very close to t.

orgal -gradet dstace trasform mage gradets I are large oly drectly o the boudary Value at (x,y tells how far that

Pros: Useful to track ad ft o-rgd shapes Cotour remas coected Possble to fll subjectve cotours Flexblty how eergy

Cos: Must have decet talzato ear true boudary, may get stuck local mmum Parameters of eergy fucto must be set well

Trackg: prevous frame s estmate serves to talze the ext Fttg actve cotours: Defe terms to ecourage certa shapes,

9 //0 Lmtatos xteral eergy: sake does ot really see object boudares the mage uless t gets very close to t. Dstace trasform xteral mage ca stead be take from the dstace trasform of the edge mage. orgal -gradet dstace trasform mage gradets I are large oly drectly o the boudary Value at (x,y tells how far that posto s from the earest edge pot (or other bary mage structure edges >> help bwdst Deformable cotours: pros ad cos Pros: Useful to track ad ft o-rgd shapes Cotour remas coected Possble to fll subjectve cotours Flexblty how eergy fucto s defed, weghted. Cos: Must have decet talzato ear true boudary, may get stuck local mmum Parameters of eergy fucto must be set well based o pror formato Summary Deformable shapes ad actve cotours are useful for Segmetato: ft or sap to boudary mage Trackg: prevous frame s estmate serves to talze the ext Fttg actve cotours: Defe terms to ecourage certa shapes, smoothess, low curvature, push/pulls, Use weghts to cotrol relatve fluece of each compoet cost Ca optmze d sakes wth Vterb algorthm. Image structure (esp. gradets ca act as attracto force for teractve segmetato methods. CS376 Lecture 0 K. Grauma 9

Image Analysis. Active contour models (snakes)

Image Analysis. Active contour models (snakes) Image Analyss Actve contour models (snakes) Chrstophoros Nkou cnkou@cs.uo.gr Images taken from: Computer Vson course by Krsten Grauman, Unversty of Texas at Austn. Unversty of Ioannna - Department of Computer