3.09.09 Shape descriptors Anne Solberg 3.09.009 1 Mandatory exercise1 http://www.uio.no/studier/emner/matnat/ifi/inf4300/h09/undervisningsmateriale/inf4300 termprojecti 009 final.pdf / t / / t t/ifi/inf4300/h09/ d i i t i /INF4300 t ti l df Deadline Friday October 9, 009. Task: classify handwritten numerals as good as possible: 10 classes, 000 samples from each class Data: 14 moment features, 6 Fourier features, 4 Zernike features. 1. Divide iid available data into training i and test sets.. Study feature selection: 1. Find the best Fourier features. Find the best moment features 3. Find the best Zernike features 4. Create a set of combined features 3. Select a classifier, train it to the selected features. 4. Evaluate classifier perfomance.
Shape representationvs. shape descriptors Shaperepresentation representation: storing and representing objects. Shape descriptors: methods for characterizing object shapes. The resulting feature values should be useful for discrimination between different object types. Remark: Gonzales and Woods (Chapter 11) is not a very good source for this topic. Curriculum: Lecture foils. A better book on shape description is M. Nixon and A. Aguado, Feature extraction & Image Processing 3 What is a good feature for character recognition? Features computed from different object representing the same symbol (object class) should be similar. Features computed from objects from different classes should be different. Visualize the feature values in scatter plots. Challenge yourself : Which features from today s lecture do you think from their definition are good for character recognition? Try to compute them and produce scatter plots do they still seem useful? What is the limitation ofusing scatterplots compared to using a n dimensional feature vector? 4
A D scatter plot is a plot of feature values for two different features. Each object s feature values are plotted in the position given by the features values, and with a class label telling its object class. Matlab: gscatter(feature1, feature, labelvector) Classification (next lectures) will be done based on more than two features, but this is difficult to visualize. Features with good class separation are show clusters for eachclass, butdifferent clusters should ideally be separated. Scatter plots Feature : major axis length Feature 1: minor axis length 5 Descriptors from the contour of the object Boundary length/perimeter Area Curvature Diameter/major/minor axis Eccentricity Bending energy Basis expansion (Fourier) last week 6
Descriptors fromthecontour Boundary length/perimeter t Simple to derive from chain code Count 1 for each horz vert move, and for each diagonal move Distance measure differs when using 8 or 4 neighborhood Area Straight forward implementation: traverse all object pixels. Can also be calculated from the boundary by Greens theorem Surface integral equals boundary integral Simple to implement, follow the contour, x and dy follow from pixels in the sequence Complexity (very simple measure) C=P /A (P=Perimeter, A=Area) For the curious: learn how area and perimeter is computed from chain codes: http://www.ifi.uio.no/inf69/forelnotater/handouts_l7.pdf 7 Descriptors fromthecontour Curvature In the continous case, curvature is the rate of change of slope. In the discrete case, difficult because boundary is locally ragged. Use difference between slopes of adjacent boundary segments to describe curvature at point of segment intersection. Curvature can be calculated from chain code. 8
Discretecomputation computation of curvature Trace the boundary and insert vertices at a given distance (e.g. 3 pixels apart). Compute local curvature c i as the difference between the directions of two edge segments joining a vertex: r r ci = d i d i 11 Curvature feature: sum all local curvature measures along the border. A rectangular n sided polygon will have sum of angles of π. More complex regions get higher curvature. v i : edge segment i d^t-1 : unit vectors of edge segments d t-1 and dt c i : local l curvature at point i 9 Contour based features Diameter = Major Mj axis (a) () Longest distance of a line segment connecting two points on the perimeter Minor axis (b) Computed along a direction perpendicular to the major axis. Largest length possible between two border points in the given direction. Eccentricity of the contour (a/b) 10
Features computed from the object Bounding box features Features from projection histograms Topological features Statistical moments computed from the objects 11 Some object features Area Area of convex hull Extent = Area/(Area of bounding box) Solidity = Area/Convex Area Compactness or circularity 1 for a circular disk, <1 otherwise 4π area perimeter 1
Bounding box features Center of mass Regular bounding box Width of boundingbox box Height of bounding box Object-oriented bounding box If the object s orientation is known, a bounding box can also be oriented along this direction. Computation of object orientation: later. Regular bounding box 13 Topologic features This is a groupofwarpof invariant integer features Some is based on the object skeleton Features: Number of holes in the object (H) Number of terminations (one line from a point) Number of breakpoints or corners (two lines from a point) Number of branching points (three lines from a point) Number of crossings (more than three lines from a point) Number of components (C) Euler number, E=C H Number of connected components holes Symmetry Region with two holes Regions with three connected components 14
1D Vertical projection Projection onto the vertical axis Histogram for each row Image binary region pixels For each row in the region, count the number of object pixels. 15 Projections 1D horizontal projection of the region: p h( x) = f ( x, y) y 1D vertical projection of the region: p v ( y ) = f ( x, y ) x Can be made scale independent by using a fixed number of bins and normalizing thehistograms. 16
Use of projection histograms Divide the object into different regions and compute projection histogramsfor each region. How can we use this to separate 6 and 9? The histograms can also be used as features directly. 17 Use of projection histograms Check if a page pg with text is rotated x 14 x 10 1 10 8 1 10 8 6 6 4 4 0 100 150 00 50 300 350 400 450 500 550 0 100 150 00 50 300 350 400 450 500 550 600 Detecting lines, connected objects or single symbols 140 7000 150 6000 5000 160 4000 170 3000 180 000 190 1000 00 150 00 50 300 350 0 0 50 100 150 00 50 300 18
Moments Borrow ideas from physics and statistics. For a given intensity distribution g(x, y) we define moments m pq by For sampled (and bounded) intensity distributions f (x, y) f(x,y) is a grey level image A moment m pq is of order p + q. grey-level image 19 Momentsfrombinary images For binary images, where f (x, y) = 1 object pixel f (x, y) = 0 background pixel Area Center of mass / tyngdepunkt 0
Grayscale moments In gray scale images, where f(x,y) [0,...,G 1] we may regard f(x,y) as a discrete D probability distribution over (x,y) For probability distributions, we should have And if this is not the case we can normalize by 01.10.008 1 Central moments These are position invariant moments where The total intensity and the center of mass are given by This corresponds to computing ordinary moments after having translated the object so that center of mass is in origo. Central moments are independent of position, but are not scaling or rotation invariant. What is μ 00 for a binary object?
Moments of inertia/variance Treghetsmoment The two second order central moments measure the spread of points around the centre of mass From physics: moment of inertia about an axis: how much energy is required to rotate the object about this axis: Statisticans like to call these measurements variance, while physicists will use the term moments of inertia. The cross moment of intertia is given by and this is what statisticianscallcall covariance or correlation. Orientation of the object can be derived from these moments, which means that they are not invariant to rotation. 3 Moments of inertia for simple shapes Rectangular object: 4/3a 3 b Square: 4/3a 4 b Elliptical object: Moment of inertias: π/4a 3 b and π/4ab 3 Circle with radius R: π/4r 4 a a b 4
Object orientation Orientation is defined as the angle (relative to the x axis) of the axis through the center of mass that gives the lowest moment of inertia. Orientation, θ, with respect to the x axis is found by minimizing the sum I( θ ) = ( β β ) α β f ( α, β ) α = x cosθ + y sinθ, β = y cosθ xsinθ α and β are the rotated coordinates axes, rotated by θ. 5 Object rotation Inserting for α and β we get: I( θ ) = [( y y) cosθ ( x x)sinθ ] f ( x, y) x y I ( θ ) We require that = 0 and after some math get: θ 1 1 tan μ θ = 11 μ0 μ0 where θ 0, π / if μ [ ] > 0, andθ [ π /, π ] if μ < 0, 11 11 6
We require that x y x y = x μ11 μ μ The computations... I ( θ ) = 0 θ and get: [( y y)cos ( x x)sinθ ] [ ( y y)sinθ ( x x)cosθ ] f ( x, y) θ = 0 f ( x, y) y ( ) 0 [( x x)( y y) ( cos θ sin θ )] f ( x, y) sinθ = cos θ sin [( x x) ( y y) ] tanθ = θ 1 tan θ 11 = 0 sinθ cosθ tan θ 7 Bounding rectangle Image oriented bounding box: The smallest rectangle around the object, having sides that are parallell to the edges of the image. Found by searching for min and max x and y within the object (xmin, ymin, xmax, ymax) Object oriented bounding box: Smalles rectangle around the object, having one side parallell to the orientation of the object (θ). The transformation α = x cosθ + y sinθ, β = y cosθ xsinθ is applied to all pixels in the object (or its boundary). Then search for α min, β min, α max, β max 8
Fitting an ellipse The best fitting ellipse hasthe same second order central moments as the object. Assume that the ellipse has semimajor and semiminor axes (a,b). For anellipse given by ( x / a) + ( y / b) = 1 the largest second order central moment is given by I I 0 = 0 b a a a x a x ( ) dx 4 b x a 1 x = x a a x + sin a 8 8 a 4 b a π π π 3 I 0 = + = a b a 8 4 The smallest moment of inertia is I min a a π == a b 4 3 9 If we have found the orientation θ of the object, as well as the center of mass ( x, y). We can find the smallest and largest moment of inertia: I ' min ' I max = x = x [ ( y y)cosθ ( x x)sinθ ] y [ ( y y )sin θ ( x x)cos θ ] This gives the two semi axes of the best fitting ellipse 4 a = π 1/ 4 y 1/8 ' 3 4 ' ( I ) 4 1/ ( I ) max ' min I, b = π min ' max I 3 1/8 30
What if we want scale invariance? Changing the scale of f(x,y) by (α,β) gives a new image: ( x / α, / β ) f ( x, y) = f y The transformed central moments μ 1+ p 1+ q pq = α β If α=β, Scale invariant central moments are given by the normalization: μ pq η pq μ pq p + q =, γ = + 1, p + q γ ( μ 00 ) 31 Hu s moments:a set of moments invariant to translation, scalingandand rotation 3
Using momentsas shape features The central moments are not used directly as shape descriptors. Mj Major and minor axis are useful shape descriptors. Object orientation is normally not used directly, but to estimate rotation. The set of Hu moments can be used as shape descriptors. (Start with the first four as the last half are often zero for simple objects). 33 Moments that are invariant to general affine transforms 34
For the curious student: What about using another basis? So you want something that works, even if it is difficult to understand??? Just as the contour can be represented on a Fourier basis, regions can be mapped on a orthogonal set of complex (two dimensional) polynomials The Zernike basis ( stolen from physics) has been very popular in OCR The basis functions, so you normally need fewer moments for good reconstruction. The Zernike moment of order n is: where We project the image inside the unit circle, since We must map the image coordinates so they are inside a circle. Rescale the symbol so that it is within a smaller square in a larger quadratic image array m is an integer such that n m is even and m<n. The magnitudes A nm are rotation invariant. 35 Zernike moments The Zernike moments are projections of the input image onto a space spanned by the orthogonal V functions where j = 1, n 0, m n, n m is even, and 36
Zernike moments The image within ihi the unit circle may be reconstructed to an arbitrary precision ii by where the second sum is taken over all m n, such that n m is even. 37 Zernike moments Zernike software (matlab): http://www.mathworks.com/matlabcentral/fileexchange/loadfile.do?objectid=797&objecttype=file do?objectid=797&objecttype=file (We have only tried an older version of this ) 38
Zernike reconstruction examples Original symbol, followed by reconstructed symbol with Zernike moments of order 1-13. 39 Now back to something you can understand: A small example of shape features 40
Which numbers are well and bad separated? 41 Two correlatedfeatures 4
43 44