Grouping with Directed Relationships

Transcription

1 Grouping with Directed Relationships Stella. Yu Jianbo Shi Robotics Institute Carnegie Mellon University Center for the Neural Basis of Cognition

2 Grouping: finding global structures coherence salience figure-ground Doc stocks Fed Doc 2 M Doc n M cut jobs M Condit M document clustering cyber community 7/3/2003 2

3 Outline of the talk Motivation Model Examples Conclusions 7/3/2003 3

4 Motivation: grouping with pairwise similarity attraction Cut distance Separation cost: pairwise similarity = attraction Attraction unites elements who have common friends 7/3/2003 4

5 Motivation: similarity grouping is not enough Cut- Cut-2 Cost- + Cost-2 > min (Cost-, Cost-2) Cannot unite elements who have common enemies 7/3/2003 5

6 Motivation: similarity vs. dissimilarity Coherent ground Incoherent ground Coherent figure Yes Bad Incoherent figure No 7/3/2003 6

7 Motivation: grouping with ordering M Hubs Authorities 7/3/2003 7

8 Model Goal Similarity grouping, dissimilarity grouping, figure-ground in one step Representation A pair of directed graphs for any pairwise relationships Criteria Generalized normalized cuts Energy formulation Rayleigh quotients of Hermitian matrices Solution Phase plane partitioning 7/3/2003 8

9 Review: segmentation in a graph framework G=(V, E, W) V: each node denotes a pixel E: each edge denotes a pixel-pixel relationship W: each weight measures pairwise similarity Segmentation = node partitioning break V into disjoint sets V, V 2 [Shi & Malik, 97] [Puzicha et al, 98] [Perona & Freeman, 99] M 7/3/2003 9

10 Review: cuts, associations, degrees Cuts: between-group similarity 3 4 Associations: within-group similarity Degrees: total similarity 7/3/2003 0

11 Review: criteria and properties wo goals Maximize normalized associations Minimize normalized cuts Duality to achieve both goals at the same time Normalization for larger structures Min-cut Min-Ncut 7/3/2003

12 7/3/ Review: energy formulation = l l l V u V u u 0,, ) ( Variables: indicators = = , 0 n n n n = = 2 2 ), ( t t t t t D W Nassoc Energy functions Change of variables ) deg( ) deg(, ) ( 2 V V y = = α α α 0.., ) ( = = D y t s Dy y Wy y y Nassoc Rayleigh quotient opt y opt D y W 2 λ = Solution: eigenvector

13 Review: interpretation of the eigensolution he derivation holds so long as + 2 = y = α he eigenvector solution is a linear transformation (scaled and offset version) of the probabilistic membership indicator for one group. If y is well separated, then two groups are well defined; otherwise, the separation is ambiguous 7/3/2003 3

14 Model: goal Dissimilarity Similarity Pragnanz Ordering Grouping Figure-ground 7/3/2003 4

15 Model: representation G=(V, E, A) A asymmetric Separation cost G=(V, E, R) R asymmetric Separation gain 7/3/2003 5

16 Model: attraction, repulsion, difference flow = A 3 4 A = A + u A 3 4 A d = A A = R 3 4 R = R + u R 3 4 R d = R R 7/3/2003 6

17 Model: criteria Maximize normalized within-region similarity between-region dissimilarity figure-to-ground order Minimize normalized between-region similarity within-region dissimilarity ground-to-figure order /3/2003 7

18 Model: is repulsion what we want? Cut- Cut-2 Cost- + Cost-2 < min (Cost- + Gain-2, Cost-2 + Gain-) Repulsion unites elements who have common enemies 7/3/2003 8

19 Model: is difference flow what we want? Cut- Cut-2 Cut (V, V2) Cut (V2, V) Ordered partitioning 7/3/2003 9

20 Model: energy formulation l ( u) =, 0, u V l u V l Variables: group indicators U = A u V = A d + R d D = D Au R u + DR Weight matrices and degree matrix + D Ru Energy functions of indicator variables Nassoc (, 2 ) = 2 t = t t U D t t + 2 D V 2 2 D 2 7/3/

21 Model: energy formulation Nassoc (, 2 ) = 2 t = t t U D t t + 2 D V 2 2 D 2 deg( V ) z = α i α 2, α = deg( V ) Change of variables W = U + i V Generalized affinity z Nassoc ( z) = z s. t. z 2 H H D = 0 Wz Dz Rayleigh quotient 7/3/2003 2

22 Model: three aspects of solutions Efficient solutions in the continuous domain: Eigenvector corresponding to the largest eigenvalue of (W, D) Little increase in complexity Hermitian matrices and sparse matrix eigendecomposition Interpretation of complex-valued solutions Phase plane partitioning 7/3/

23 7/3/ Model: segmentation as embedding Assign region identity. u = n opt u u z M grouping brightness textureness location color u v + + = n n opt v i u v i u z M figure-ground

24 Results: difference flow for relative depth image A by proxmity A by brightness similarity Relative depth from -junctions R d for a figure pixel R d for a ground pixel 7/3/

25 Results: interaction of attraction and repulsion Image Attraction Repulsion Result: A Result: R Result: A and R 7/3/

26 Results: figure-ground segregation Oversegmentation Image Solutions: A based on A R Figure Ground Ground 2 Phase plot 7/3/

27 Results: from Gaussian to Mexican hat U = A u R + u D R Attraction Repulsion 7/3/

28 Results: popout Attraction to bind similar elements Repulsion to bind dissimilar elements Regularization to equalize Stimuli Attraction +Repulsion +Regularization 7/3/

29 Results: computational efficiency 30 x 30 Image A: r = A: r = 3 A: r = 5 A: r = 7 [A, R]: r = 7/3/

30 Results: spatial attention attraction + repulsion Bias for A Bias for R + bias 7/3/

31 Conclusions Pairwise relationships Attraction: similarity grouping Repulsion: dissimilarity grouping Difference flow: relative ordering Advantages of repulsive and ordinal relationships: Complementary Computational efficiency reat 2D and 3D configuration cues equally Figure-ground organization Coherent figure Incoherent figure Coherent ground Attraction +Repulsion Incoherent ground +Regularization 7/3/2003 3