Video Data Analysis. Video Data Analysis, B-IT

Transcription

1 Lecture Vdeo Data Analyss Deformable Snakes Segmentaton Neural networks Lecture plan:. Segmentaton by morphologcal watershed. Deformable snakes 3. Segmentaton va classfcaton of patterns 4. Concept of a neural network 5. Supervsed learnng & classfcaton by a neural network

2 Lecture Segmentaton by morphologcal watersheds Prncple: vsualzng an mage n 3-Dmensons Readng: :0.5 The Topographc representaton of mages generates ponts of three types:. Regonal mnmum ponts;. Ponts at whch a drop of water would fall wth certanty to a sngle mnmum. 3. Ponts at whch a drop of water would lkely to fall nto more than one mnmum. Basc dea Basc dea: suppose a hole s punched n each regonal mnmum and the entre topography s flooded from below Maor applcaton: extracton of nearly unform obects from the background. These obects are characterzed by small varatons n gray levels. Thus watershed s often appled to the gradent of an mage: small grayscale varatons correspond to catchment basns regons

3 Lecture Readng: :0.5 Segmentaton by morphologcal watersheds Dam constructon: Condtonal dlaton of two connected components: ) Dlaton has to be constraned to q. ) Dlaton cannot be performed on ponts that would cause the two sets to merge. q two connected components C[n-] (catchment basns) at floodng step, n- Result C[n] of the next floodng step, n The water has splled and the dam must be bult Condton ) faled durng the second dlaton; Condton ) was met for ponts at separatng dam

4 Lecture Watershed segmentaton algorthm Readng: :0.5 Let M, M,.. M R be sets denotng the coordnates of the ponts n the regonal mnma of an mage g(x,y). Let C(M ) be a set denotng the coordnates of the ponts n the catchment basn assocated wth the regonal mnma M. Let T[n] represent the set of coordnates (s,t) for whch g(s,t) < n: T [ n] = ( s, t) g( s, t) < n { } Geometrcally, T[n] represent ponts n g(x,y) layng below the plane g(x,y) = n. The topography wll be flooded n nteger ncrements from n = mn to n = max. The coordnates of ponts n the catchment basn M flooded at stage n s a bnary mage: C ( M ) = C( M ) I T[ n] Let C[n] s the unon of the flooded catchment basns portons at stage n: C[ n] = The algorthm s ntalzed wth : C [mn] = T[mn] The algorthm proceeds recursvely assumng at step n that C[n-] has been constructed. ) Flood T[n]; ) Recover C[n]; - How to obtan C[n] from C[n-]? Let Q s a set of connected components n T[n]. For each connected component q Q there exst three possbltes: U R = n C n ( M ) A) B) C) q I C[ n ] q I C[ n ] q I C[ n ] s empty new mnmum s encountered contans one connected component of C[ n ] q les wthn catchment basn contans more than one connected component of C[ n ] floodng occurs

5 Lecture Watershed segmentaton algorthm (contnued) Readng: :0.5 A) B) C) q I C[ n ] q I C[ n ] q I C[ n ] s empty contans one connected component of C[ n ] contans more than one connected component of C[ n ] Condton A) occurs when a new mnmum s encountered. Connected component q s ncorporated nto C[n-] to form C[n]. Condton B) occurs when q les wthn a catchment basn of some regonal mnmum. Connected component q s ncorporated nto C[n-] to form C[n]. Condton C) occurs when a rdge separatng two or more catchment basns encountered. A dam must be bult wthn q to prevent overflow between catchment basns. A one-pxel-thck dam can be constructed by dlatng q C[n-] wth a 3x3 structurng element and constranng the dlaton to q. Hnt: n corresponds to gray level values n g(x,y).

6 Lecture Watershed segmentaton Readng: :0.5

7 Lecture Watershed segmentaton Readng: :0.5 Drect applcaton of the watershed segmentaton may lead to oversegmentaton Lmt the number of allowable regons by pre-processng such as: - Smoothng - Defne a lmted number of nternal markers accordng to some crteron; These markers are assocated wth obects. - External markers are assocated wth the background; Image Oversegmentaton Watershed result appled to the smoothed mage f nternal markers are the only allowed mnma Watershed segmentaton of each regon appled to the gradent of the smoothed mage

8 Lecture Segmentaton by deformable contours: Snakes Readng: 5:5.4 Snake or actve contour s an energy mnmzng splne. The task s to ft a curve of arbtrary shape to a set of mage ponts by mnmzng an energy functonal: E = ( α( s) E + β ( s) E + γ ( s) E and a contour s gven by : c = c( s) cont curv parameterzed by ts length s. Internal energy s composed of the contnuty term (elastcty) and smoothness term (stffness). The contnuty term s based on the frst dervatve : E In the descreet case : E cont cont = = pars of p ponts : ) mage ) ds In order to prevent the formaton of clusters of snake ponts an approxmaton uses the average dstance between the E cont dc ds p = ( d p p

9 Lecture Segmentaton by deformable contours: Snakes Readng: 5:5.4 The smoothness term tres to avod oscllatons of the contour by mnmzng the curvature : E curv d c = ds In the descreet case t s approxmated Ecurv = p p + p+ by settng β = 0 at pont allows the snake to develop a corner by : External energy s assocated to the external force attractng the contour towards the desred mage contour: E mage = I

10 Lecture Segmentaton by deformable contours: Snakes Readng: 5:5.4 Assumptons: Let I be and mage and p, p, p N the chan of mage locatons representng the ntal poston of the deformable contour, whch s close (!) to the mage contour of nterest. Two ways to solve the mnmzaton problem: - usng the calculus of varatons and the Euler-Lagrange condton for mnmzaton, or the greedy algorthm, whch makes locally optmal choces n the hope to fnd the global mnmum.

11 Lecture Segmentaton by deformable contours: Snakes Normalzaton It s mportant to normalze the contrbuton of each energy term, for nstance by dvdng each term by the largest value n the neghborhood n whch the pont can move. For E mage apply usual mnmax normalzaton: Readng: 5:5.4 Greedy algorthm conssts of two steps: seekng for a new contour pont n a local neghborhood that mnmzes the energy functonal and handlng the occurrence of corners.. Greedy mnmzaton. Select a small local neghborhood (3x3, or 5x5). The local mnmzaton s done by drect comparson of the energy functonal at each locaton.. Allowng corner formaton. Search for corners as the curvature maxma along the contour. If a curvature maxma s found at p, then β s set to zero. Useful extra condton for the corner s the large value of ntensty gradent n that pont. I mn max mn Stoppng crteron: when a predefned fracton of all ponts reaches a local mnmum.

12 Lecture Readng: 5:5.4 Segmentaton by deformable contours Snakes

13 Lecture Readng: 5:5.4 Segmentaton by deformable contours Snakes

14 Lecture Image features Readng: :.3.4 Image segmentaton va classfcaton of mage patterns, or solvng obect recognton problem. Task: fnd descrptors, whose values are smlar for members of same pattern and dfferent for members comng from other patterns. These descrptors descrptors characterzng mage patterns are called features. A collecton of features forms feature vector, whch s a pont n the feature space.

15 Lecture Lnear classfer Readng: :. If features are chosen properly, ponts belongng to dfferent classes are form clusters n the feature space. In ths case classes can be separated by a dscrmnaton hyper-surface. If classes can be separated by a hyper-plane, than t s a lnearly-separable task called the lnear classfer. Equaton for a "decson" hyperplane s : t g( x) = w x + w two - category decson rule, If w w If x t w x and g( x) x to the hyperplane: x = x g( x) = 0, and + w w gves p f f g( x) > 0 g( x) < 0 s x t = w x w w x normal t g( x) = w x + w 0 + r 0 are both on the decson hyperplane, then : to an algebrac 0 can be assgned to ether class. + w = r 0 w decde : t w ( x measure of x any vector n the r = the dstance from g( x) w ) = 0 hyperplane.

16 Lecture Lnear classfer The Mnmum dstance classfer Readng: :. m Determne the closeness based on the D Selectng the smallest dstance s equvalent to assgnng x to the class w largest value for the followng decson functons : T T d ( x) = m x m m =,..., W The decson hyperplane g( x) between classes w g( x) = d Ths = x N ( x) = gves : g( x) = ( m w and ( x) d x m m w ( x) = 0 ) T =,..., W =,..., W s equdstant : x ( m whch yelds the m ) T dstance: ( m + m )

17 Lecture Bayes classfer for Gaussan pattern classes Bayes classfer mnmzes the total average loss due to msclassfcaton of pxels between n classes: Readng: :.. Classes PDF s are Gaussan functons Two-stage classfcaton procedure:. Tranng.. Classfcaton. The Bayes classfer assgnes the pattern x to class w f : p( x / w )P( w ) > p( x / w )P( w ) =,,..,W; Gvng the decson functons of the form : d ( x) = p( x / w )P( w ) =,,..,W C w w d ( x) = m T xx x T d ( x) = ln( P( w m m m T m Approxmatng the expected value by the average yelds the mean vector and covarance: m = x N = N If C = I and If all covarance matrces are equal then Bayes decson functon for class T C = C, T T ) + m C x m C m P( w ) = W then the lnear classfer follows : w s :

18 Lecture SVM classfer Readng: :7.. SVM preprocess the data to represent patterns n a hgh dmenson, typcally much hgher than the orgnal feature space. Usng approprate non-lnear mappng: y = φ(x) two a suffcently hgh dmenson, data can be always separated by a hyper-plane: g ( y) = a The goal n tranng the SVM s to fnd a separatng hyperplane wth the largest margn b separatng the two classes. If the margn exst then: t y z g( y a ) b, =,... N, z = ±

19 Lecture Neural Networks Readng: :..3; :7.3 The decson boundary: d( x) y = ( y, y w = ( w n n+ = w x + wn+ = = =, w,..., y, n, ) n T..., w, w n+ ) T w y = w T y = 0

20 Lecture Neural Networks Readng: :..3; :7.3 Equvalent condton on the output: O = + f f n = n = w x w x > w < w n+ n+

21 Lecture Tranng for lnearly separable classes Supervsed learnng Readng: :..3; :7.3 Iteratve algorthm for tranng sets belongng to classes w and w : - Frst choce for w() s arbtrary. - At the k teraton step: f y( k) w and w w( k + ) = w( k) + cy( k) T ( k) y( k) 0 update w( k) wth c > 0 : f y( k) w and w w( k + ) = w( k) cy( k) Otherwse leave w( k) : w( k + ) = w( k) T ( k) y( k) 0 update w( k) : Convergence s sad to be occurred when the whole set s cycled through wthout any errors

22 Lecture Tranng for lnearly separable classes Supervsed learnng -Tranng example: x Readng: :..3; :7.3 Two-class tranng set: Let c=, w()=0 : w w = {(0,0,) = {(,0,) T T,(0,,),(,,) T T } } 0 w w x Convergence s acheved after 4 teratons: w(4) = (,0,) d( x) = x + = 0 T 0 x d( x) = x + = 0

23 Lecture Multlayer feedforward neural networks Problem: Decson functons for multclass pattern recognton Readng: :..3; :7.3

24 Lecture Multlayer feedforward neural networks Readng: :..3; :7.3 Actvaton functon wth necessary dfferentablty: h ( I ) = ( I + θ )/ θ + e 0 I θ θ 0, =,,..., N Total nput to each node n layer J Offset & shape of the sgmod functon

25 Lecture Multlayer feedforward neural networks Actvaton functon Readng: :..3; :7.3 Offset θ s analogous to the last coeffcent w n+, the actvaton functon on each node n layer : h ( I ) = + e Nk ( w k O k + k= θ )/ θ 0

26 Lecture Multlayer feedforward neural networks Tranng of the mult-layer network s the process of adustng weghts of neural unts n all layers: Readng: :..3; :7.3 Output layer: Adaptng weghts s smple because the desred output of each node s known. Hdden layers: Dffcult! The obectve s to develop a tranng rule, whch allows adustment of the weghts n each of the layers such that mnmzes an error functon of the form: E r q Q N Q q q NQ ( rq Qq ) q= = Desred node Actual node responses responses the number of nodes n the output layer Q Here Q s the output layer.

27 Lecture Multlayer feedforward neural networks Step : Adustng weghts n the layer P precedng the output layer Q n proporton to the partal dervatves of the error wth respect to these weghts gves: Δw Δw qp qp E = α w = α( r q Q qp α postve correctve M and after some algebra : O ) h ( I q q q ncrement ) O p = αδ O q p Readng: :..3; :7.3 For complete dervaton see : p.7-75 After the functon h q (I q ) s specfed all the terms are ether known or can be observed n the network: - r q - s known for any tranng pattern shown to the network; - O q the value of each output node can be observed; - I q - the nput to the actvaton elements of layer Q can be observed; - O p the output of the nodes n layer P can be observed;

28 Lecture Multlayer feedforward neural networks Step propagaton of weght adustments nto nternal layers: For any layers K and J, where K mmedately precedes layer J, compute the weghts w k whch modfy the connectons between these two layers: Readng: :..3; :7.3 Δw δ = ( r δ = h δ = α( r δ = O = αδ O O p p= Usng the actvaton functon wth θ =, yelds k wth : ( I N k O ) h ( I for the output layer ; ( O ) p N δ w p p= p for the nternal layer. for the output layer: ) ) O ( O ) for the nternal layer: ) δ w p p and : 0 Generalzed delta rule for tranng the multlayer feedforward neural network NB: For complete dervaton see : p.7-75

29 Lecture Multlayer feedforward neural networks Summary on the tranng process: Start wth an arbtrary but not all equal weghts throughout the network; -The applcaton of the generalzed delta rule nvolves two basc phases: Phase : - A tranng vector s presented to the network and s propagated through the layers to compute the output, O for each node. - The outputs O q of the nodes n the output layer are compared aganst the desred response, r p to generate the error terms σ q Phase : -Update weghts of all network nodes gradually passng the error sgnal backwords. Apply same procedure to update the bas weghts, θ treatng them as addtonal weghts that modfy a unt nput nto the summng uncton of every node n the network. Readng: :..3; :7.3

30 Lecture Multlayer feedforward neural networks Readng: :..3; :7.3 Example -Input 48-dmensonal vectors normalzed shape descrptors; - Synthetc nosy samples through random perturbaton of contour coordnates wthn 8x8 wndow.

31 Lecture Multlayer feedforward neural networks Example - Input layer wth 48 nodes, same as the dmensonalty of nput vectors; - Output layer wth 4 nodes corresponds to the number of classes; - no rule for the number of nodes n nternal layers; - classfcaton condton for pattern : correspondng output unt s hgh 0,95, whereas other unts are low 0,05. Readng: :..3; :7.3

32 Lecture Multlayer feedforward neural networks Tranng - 0 samples for each class wth zero nose; - Gradual re-tranng wth nosy sets; -Recognton - recognton rate of nosy shapes close to 77% when traned wth nose-free data; - recognton rate ncreases to about 99% when traned wth noser data. Readng: :..3; :7.3 R t s the probablty of a boundary pxels to be perturbed due to nose.

33 Lecture Complexty of decson surfaces What s the nature of the decson surface mplemented by a mult-layer network? Readng: :..3; :7.3 Two-nput twolayer network AND (0,) Each node n the frst layer defnes a lne. More nodes would defne a convex regon. Possble decson boundares that can be mplemented. The network can dstngush between two classes, that couldn t be separated by a sngle lnear surface

34 Lecture Complexty of decson surfaces In the 3-layer network nodes n the: - Layer mplement lnes; - Layer perform AND operaton formng regons from varous lnes; -3Layer assgn class membershp to varous regons; Readng: :..3; :7.3 In general: -a two-layer network mplements arbtrary convex regons through ntersecton of hyperplanes; -A three-layer network mplements decson surfaces of arbtrary complexty.