EECS490: Digital Image Processing. Lecture #26

Transcription

1 Lecture #26 Moments; invariant moments Eigenvector, principal component analysis Boundary coding Image primitives Image representation: trees, graphs Object recognition and classes Minimum distance classifiers Correlation Statistical pattern recognition; Bayes decision rule Neural network classifiers

2 Moments The calculation of moments of for images and objects in images is very similar to that used to calculate statistical moments m pq = m pq = + + x p y q f( x, y)dxdy For an LxL discrete image we can write L 1 i=0 L 1 j =0 x p i y q j f( x i, y ) j

3 Moments We can also define central moments in a similar manner m 10 = μ pq = L 1 i=0 L 1 j =0 ( x i x ) p y j y Some simple moments and means L 1 L 1 j =0 μ 00 = m 00 = ( ) ( ) q f( x i, y ) j L 1 L 1 i=0 j =0 f( x i, y ) j x i f x i, y j m 01 = y i f x i, y j i=0 L 1 i=0 L 1 j =0 x = m 10 y = m 01 m m ( )

4 Moments Computing higher moments and means μ 20 = L 1 L 1 ( ) ( x i x i ) 2 f x i, y j = m 20 xm 10 i=0 j =0 μ 02 = L 1 L 1 ( y j y j ) 2 f( x i, y j ) = m 02 ym 01 i=0 j =0 20 = μ 20 2 μ = μ 02 2 μ 00 We can use these second moments to define the first invariant moment 1 =

5 Invariant Moments 1 = ( ) = = ( ) 2 + ( ) 2 7 = ( )( ) ( ) ( ) 2 + ( )( ) 3( ) ( ) 2

6 Moments Original Reduced 50% in size Mirrored Rotated 2 Rotated 45

7 Moments There are seven moments which are invariant to translation, rotation, and scale. Each row in this table should remain constant. The constancy shows the utility of the moment

8 Moments Translated Original Reduced 50% in size Mirrored Rotated 90 Rotated 45

9 Representation and Description Each row in this table should also remain constant. The constancy shows the utility of the moment This example is much better than the example from the 2nd edition

10 Eigenvectors and Eigenvalues For RGB images (with 3 components) we can write each pixel as x x = x 1 x 2 x 3 = R G B For registered multi-spectral images (with n components) we can write x as x = x 1 x 2 x n n=6 components

11 Eigenvectors and Eigenvalues For these n registered images we can compute the mean vector (nx1) and covariance matrix (nxn) for the set of all x x = x 1 x 2 x n m x = E{ x}= { } { } { } E x 1 E x 2 E x n C x = E{ ( x m x )( x m x ) T }= {( )( x 1 m 1 )} E ( x 1 m 1 )( x 2 m 2 ) { } E{ ( x 1 m 1 )( x n m n )} E x 1 m 1 E{ ( x 2 m 2 )( x 1 m 1 )} E{ ( x 2 m 2 )( x 2 m 2 )} E{ ( x n m n )( x 1 m 1 )} E x n m n {( )( x n m n )}

12 Eigenvectors and Eigenvalues Transform the data by a Hotelling* transformation y = A( x m ) x where the rows of A are the eigenvectors of the covariance matrix C x A = e 1 T e 2 T e n T * Also known as the Karhunen-Loeve transformation

13 Eigenvectors and Eigenvalues The transformed variable y has the properties that m y = E{ y}= 0 and C y = AC x A T = n where 1 > 2 > > n C y is diagonal which indicates that the transformed y vectors are uncorrelated.

14 Eigenvectors and Eigenvalues This transformation can be inverted to give back the original image vectors x x = A T y + m x where A = e 1 T e 2 T e k T

15 Eigenvectors and Eigenvalues What if I replace A by B which has only the k eigenvectors corresponding to the k largest eigenvalues? We now have an approximation to x given by B = e 1 T e 2 T e k T ˆx = B T y + m x The difference (rms error) between this approximation and the original x is given by the sum of the eigenvalues corresponding to the removed eigenvectors. e rms = n j = k +1 j

16 Eigenvectors and Eigenvalues The transformed vectors y are orthogonal and form a orthoginal basis set for describing the original set of images. ˆx = k i=1 a i y i Each y i is an eigenvector of C x and represents a feature of the original set of images. y 1 corresponds to the most significant feature since y 1 corresponds to the largest eigenvalue 1. y 2 corresponds to the next most significant feature since y 2 corresponds to the next largest eigenvalue 2.

17 Eigenvector Example Six registered multi-spectral images: blue, green, red, near infrared, middle infrared, and far (thermal) infrared bands.

18 Eigenvector Example Each pixel in this multi-spectal image can be described by a sixcomponent pixel vector x These are 384x239 pixels so there are pixel vectors describing the complete image

19 Eigenvector Example We can use a computer program to compute the mean and 6x6 covariance matrix C x for these pixels. We can then calculate the eigenvectors e 1 to e 6 for C x. For this example the eigenvalues are

20 Eigenvector Example These are the six principal component images given by y i =e it (x-m x ) This is the most significant image with the most information ( 1 ) y 1 y 2 y 3 This is the least significant image with the least information ( 1 ) y 4 y 5 y 6 This transformation rearranges the information presented in the multispectral image.

21 Eigenvector Example Now reconstruct the six original images using only the two most significant eigenvectors. Compute y = A( x m x ) Use only B = e 1 T e 2 T Invert y ˆx = B T 1 y + m x 2

22 Eigenvector Example Original images

23 Eigenvector Example Differences between the original images and the images reconstructed using only two eigenvectors.

24 Eigenface Analysis Original dataset of registered faces Corresponding 15 eigenfaces Eigenvector vector expansions have been finding increasing application in such areas as face recognition.

25 Eigenvalues & Eigenvectors Translating the object to its centroid (mean) and aligning the principal axes (variances) can often be used to facilitate object analysis and recognition. The first eigenvector e 1 corresponds to the axis of largest variance, i.e., the principal axis.

26 Eigenvector Example #2 2. Compute the eigenvectors of C x 1. Represent the boundary points of an object as a set of 2-D vectors x and compute C x 3. Compute the mean m x and translate the mean to the origin 4. Rotate the object so that e 1 corresponds to the horizontal axis and e 2 to the vertical axis.

27 Manual Eigenvector Example This is a simple boundary analysis which you can replicate by hand.

28 Relational Descriptors This is similar to the texture grammars which we discussed earlier. a,b are the elements shown above; S is the starting symbol; A is a variable These are the re-writing rules which can be used to generate abab from S S aa abs abaa abab etc.

29 Representation and Description S aa abs abaa abab The rules can be repeated to describe larger objects. The numbers correspond to the rules which were applied.

30 Representation and Description Object boundaries can be coded using headto-tail connected directed line segments.

31 Representation and Description More complex relationships can be defined for directed line segments.

32 Representation and Description

33 Representation and Description

34 Object Recognition This shows a clearly defined region for Iris setosa based upon petal length and width.

35 Object Recognition Signatures can be used to code objects to be recognized.

36 Object Recognition String descriptions can also be used to code objects.

37 Object Recognition More complex structures such as trees are needed to describe the objects in a typical image

38 Object Recognition

39 Minimum Distance Classifiers The prototype for each class (out of W classes) is the mean vector of that class m j = 1 N j xw j x j j = 1,2,...,W Compute closeness using Euclidian distance D j ( x)= x m j j = 1,2,...,W Assign x to class j if D j (x) is the smallest distance

40 Minimum Distance Classifiers More mathematically the distance from a candidate pattern x to the pairs of mean vectors m i and m j can be written as d ij ( x)= d i ( x) d j ( x)= x T ( m i m j ) 1 ( 2 m i m ) T ( j m i + m ) j This describes a plane marking the classification boundary between classes i and j, i.e., d ij ( x)= 0 If d ij (x) >0 then assign x to class j otherwise assign to class i

41 Object Recognition d 12 ( x)= 0 Decision classifier

42 Object Recognition Check characters are read horizontally by a single vertical sensor which travels from left to right to generate a distinct waveform for each character The characters and sensors are optimized to give distinct responses in a simple x-y grid.

43 Object Recognition Correlation can be used to find similar patterns

44 Object Recognition

45 Statistical Pattern Recognition The threshold for classifying patterns is where the pdf s overlap If the class probabilities are not equal we must use a Bayesian classifier and use the maximum to classify unknown patterns d j ( x)= p( x j )P j ( ) j = 1,2,...,W

46 Object Recognition In more dimensions the classifier forms a plane (or hyperplane) for Gaussian pattern classes d j ( x)= ln P( j ) 1 2 ln C j 1 2 ( x m ) T 1 j C j ( x m ) j j = 1,...,W

47 Object Recognition We can use the same registered multi-spectral image data as input to a pattern classifier

48 Object Recognition Prototypes Errors using half of the prototypes to train, the other to classify (1) Water classification (3) Vegetation classification (2) Urban dev. classification

49 Object Recognition Bayes classification of pixels in the training regions 1,2,3

50 Object Recognition The perceptron is an early neural network architecture for learning decision classifiers

51 Object Recognition

52 Object Recognition

53 Object Recognition

54 Object Recognition

55 Object Recognition

56 Object Recognition

57 Object Recognition

58 Object Recognition Neural networks can be used to implement (and learn) geometrically complex decision functions.

59 Object Recognition Multi-layer networks can be used to approximate (learn) more complex decision functions.

60 Shape Number Classification Objects can be similar up to different orders of shape number. This can be used to classify objects.

61 Shape Number Classification