PATTERN RECOGNITION AND IMAGE UNDERSTANDING

Transcription

1 PATTERN RECOGNITION AND IMAGE UNDERSTANDING The ultmate objectve of many mage analyss tasks s to dscover meanng of the analysed mage, e.g. categorse the objects, provde symbolc/semantc nterpretaton of the mage or global understandng of the mage scene. Because these tasks are applcaton specfc no ready solutons are avalable. Hence, mage analyss problems have strong resemblance to ntellgent cognton rather than rely on applcaton of ready to use set of standard processng technques. IE Techncal Unversty of Lodz

2 Image understandng Intellgence: ablty to extract meanngful nformaton from a set of rrelevant detals, capablty to learn from examples and to generalse the acqured knowledge (so that t can be useful n dfferent processng contexts), ablty to comprehend data from ncomplete nformaton.

3 The concept of learnng - example Emergence

4 An example of a dffcult pattern recognton problem Invarance

5 Gestalt Theory of mnd and bran Multstablty Refcaton (llusory objects)

6 Man parts of an mage analyss system Segmentaton Representaton and descrpton Analysed scene Preprocessng Image acquston Knowledge processng Recognton and nterpretaton Result

7 Man parts of an mage analyss system Intermdate-level processng Segmentaton Representaton and descrpton Analysed scene Preprocessng Image acquston Knowledge processng Recognton and nterpretaton Result Low-level processng Hgh-level processng

8 Image analyss system A complete mage analyss system comprses three major data processng modules: Low-level processng- comprses mage acquston, mage pre- processng and enhancement, e.g. nose reducton, contrast mprovement, flterng, zoomng, etc. Intermedate level processng- deals wth extracton and descrpton of mage components dentfed from a knowledge base, e.g. mage segmentaton, boundary and edge descrpton, morphologcal processng. Hgh-level processng- nvolves classfcaton, recognton and nterpretaton of the mage scene.

9 Patterns and pattern classes A pattern s a set of features that form a qualtatve or structural descrpton of an analysed object; n a mathematcal sense a pattern s defned as a vector of features x=[x 1, x,, x N ]. A pattern class s a group of patterns that have smlar feature vectors (accordng to some smlarty measure). Pattern classes are denoted ω 1, ω 1, ω M, where M s the number of classes. Pattern recognton (alternatvely termed pattern classfcaton) s the task of assgnng patterns to ther respectve classes. It s equvalent to establshng a mappng: x ω from the feature space X to the pattern class space Ω.

10 Pattern classfcaton defnton feature space pattern class space many to one mappng ω 1 ω ω 3 x 3 ω 4 x ω 5 x 1 ω 6

11 Pattern classfcaton problem The mappng x ω should pont to the true class label ω. The optmum among all the possble mappngs x ω s the one that generates mnmum error rate. Ths rate depends on the dstrbuton characterstcs of feature vectors n the pattern space X,.e., ther dstrbutons, overlap between dstrbutons, etc. If the knowledge about dstrbutons of the patterns s unavalable the classfcaton system must rely on the learnng paradgms, e.g. artfcal neural networks. In such approaches the knowledge s acqured from a suffcently large number set of tranng samples drawn from the pattern examples

12 Selecton of feature propertes dscrmnaton features should assume consderably dfferent values for patterns from dfferent classes, e.g., dameter of fruts s a good feature for classfcaton between grapefruts and cherres, robustness features should assume smlar values for all patterns belongng to the same class, e.g., colour s a poor feature for apples, ndependence features used n a classfcaton system should be uncorrelated, e.g., weght and sze of a frut are strongly correlated features, small number of features complexty of data classfcaton system grows substantally wth the number of classfed features, e.g., features that are strongly correlated should be elmnated.

13 Selecton of feature propertes Correlaton coeffcent between features X and Y: γ xy = 1 P P =1 ( x µ )( y µ ) σ x x σ y y where: P s the number of patterns, µ x, µ y are the average values for features X and Y correspondngly, and σ x,σ y are ther standard devatons. The correlaton coeffcent assumes values n the range [- 1,1].

14 Selecton of feature propertes A measure of separaton for feature X between classes j and k: ˆ D xjk = µ Large value for ths measure means feature X yelds good separaton between classes. σ xj xj µ + σ xk xk bad good

15 Reducton of the number of features θ Drecton of the frst prncpal component Angle θ chosen for best separaton of features

16 Decson-theoretc methods Pattern classfcaton problem can be approached by means of the so called decson functons. The number of these functons s equal to the number of classes. Let x=[x 1, x,, x N ] T represent an N- dmensonal pattern vector. For M pattern classes ω 1, ω, ω M, we are searchng for M decson functons d 1 (x), d (x),, d M (x) wth a property that, f a pattern x belongs to class ω then d ( x) > d ( x) j = 1,, K, M ; j j In other words, decson functon d (x), wns the competton for assgnng the nput feature vector x to the pattern class ω.

17 Decson functons Wley 1

18 Decson functons The decson boundary between two arbtrary classes and j ( j) s defned by the functon: d j (x) = d (x) - d j (x) = 0 Then, for patterns of class ω : d j (x)>0 and for patterns of class ω j : d j (x)<0.

19 Mnmum dstance classfer Assume that each pattern class s represented by a mean vector (also called a class prototype): m 1 = P j x ω j x, where N j s the number of pattern vectors from class ω j. Possble way to determne the class membershp of an unknown pattern vector x s to assgn t to the class of ts closest prototype vector. j = 1,, K,M?

20 Mnmum dstance classfer If the Eucldean dstance s used the dstance measure s of the form: D j ( x) = x m j j = 1,, K, M x 1/ and. T = ( x x) Feature vector x s assgned to class ω j f D j (x) s the smallest dstance.

21 Mnmum dstance classfer The followng dstance functon can be constructed d j T 1 T ( x) = x m j m j m j j = 1,, K, M and assgnng x to class ω j f d j (x) gves the largest value.

22 Mnmum dstance classfer The decson boundary between classes ω and ω j for a mnmum dstance classfer s: d j ( x) = d ( x) - d j ( x) = = x T ( m m j ) 1 ( m m j ) T ( m m j ) = 0 The surface defned by ths equaton s the perpendcular bsector to the lne jonng m and m j. For N= the bsector s a lne, for N=3 t s a plane, and for N>3 t s called a hyperplane.

23 Mnmum dstance classfer Wley d 1 (x) =.8x x = 0 ( ) ( )

24 Mnmum dstance classfer Example 1: Consder two classes denoted by ω 1 and ω, that have mean vectors m 1 = (4.3, 1.3) T and m = (1.5, 0.3) T. The decson functons for each of the classes are: d 1 (x) = x T m m 1T m 1 = 4.3x x d (x) = x T m m T m = 1.5x x Equaton for the decson boundary : d 1 (x) = d 1 (x) - d (x) =.8x x = 0 Class membershp of a new feature vector s defned on the bass of the sgn of d 1 (x).

25 Vorono mosac %MATLAB rand('state',0); 1 x = rand(1,10); y = rand(1,10); [vx, [vx, vy] vy] = vorono(x,y); plot(x,y,'r+',vx,vy,'b-'); 0.5 axs axsequal; class prototype

26 Example : Classfcaton of Amercan Bankers Assocaton font character set. The desgn of the font ensures that the waveform correspondng to each character s dstnct from that of all others. There are N=14 prototype vectors n 10- Dfeature space.

27 Mnmum dstance classfer The mnmum dstance classfer works well on the followng condtons: the dstance between class means s large compared to the spread of each class. classes are lnearly separable (class boundares are: lnes, planes or hyperplanes).

28 Evaluaton of pattern classfcaton qualty Statstcal dstrbuton of sngle feature TN A B FN FP TP decson border tested feature

29 Evaluaton of pattern classfcaton qualty Confuson matrx True classfcaton Class B (wrong, ll, ) Class A Classfer output Class B Class A TP (ang. true-postve) FN (ang. false-negatve) FP (ang. false-postve) TN (ang. true-negatve)

30 Evaluaton of pattern classfcaton qualty Accuracy: ACC = TP TP + TN + TN + FP + FN Senstvty: SE = TP TP + FN TN A B All class B examples FN FP TP

31 Evaluaton of pattern classfcaton qualty Specfty: SP = TN FP + TN All class A examples A False alarm rate: TN B FA = 1 SP = FP FP + TN FN FP TP

32 Recever Operator Characterstc (ROC curve) good test senstvty 1 dagnostc parameter poor test 0 false alarm rate 1 dagnostc parameter See excellent demo explanng ROC:

33 Types of classfers Parametrc: probablty densty functons are known and only parameters of these functons are unknown e.g., the Gaussan dstrbuton s defned by ts mean µ and standard devaton σ), The Bayes classfer s a parametrc classfer,.e., requres knowledge about densty functon dstrbutons Nonparametrc: probablty densty functons are not known and must be estmated from a suffcently large measurement data.

34 Image segmentaton example

35 Colour based segmentaton {R,G,B} Statstcal dstrbuton of colour components for the ndcated mage regons B G R

36 The Bayes classfer c 1 - class 1 c - class c a pror probabltes p(x) c x

37 The Bayes classfer From the probablty theory the followng holds: p ( a / b) = p( a) p( b / p( b) a) hence: p ( c / x) = P ( c ) ( ) p x / c p( x) a pror probablty Bayes rule a posteror probablty

38 Maxmum lkelhood estmator An mage pxel s assgned to - th class for whch: L = arg max p ( ) P( c ) p( / c ) x c / x = j p ( x), =1,, K,N.e. L > L j, j, = 1,, K,N

39 The Bayes classfer c 1 - class 1 c - class c a pror probabltes p( ω / x) > p( ω1 / x) p(x) c p ( c / x) > p( c1 / x) p ( c1 / x) > p( c / x)

40 ( ) 1, ) ( exp 1 / = = m x c x p σ πσ A pror probablty for 1D Gaussan dstrbuton: ( ) ( ) 1, ) ( ) ( exp 1 ) ( / / = = = c P m x c P c x p x c p σ πσ A posteror probablty: The Bayes classfer

41 The Bayes classfer - example Pot A Pot B 0 red 0 whte 30 red 10 whte We pck a pot randomly and then the ball randomly. Queston: From whch pot the ball was pcked?

42 The Bayes classfer - example Pot A Pot B 0 red 0 whte 30 red 10 whte Before we see the colour of the ball we pcked (a pror knowledge) we assume the probablty of choosng each of the pots s equal,.e. 0.5.

43 The Bayes classfer - example Pot A Pot B 0 red 0 whte 30 red 10 whte Suppose we pcked a red ball. Can we verfy our frst hypothess havng ths a posteror knowledge? Yes, the answer comes from the Bayes theorem.

44 The Bayes classfer -example Pot A Pot B That was the pot! probably 0 red 30 red 0 whte 10 whte p ( A / red ) = P P( A) p( red / A) ( A) p( red / A) + P( B) p( red / B) = = 0. 4 p ( B / red ) = P P( B) p( red / B) ( A) p( red / A) + P( B) p( red / B) = = 0. 6

45 Multvarate Gaussan dstrbuton A pror probablty for D Gassan dstrbuton x T =[x 1 x ]: Where s class ndex of N gven classes and the covarance matrx: ( ) ( ) N c p T, 1,, ) ( ) ( exp 1 / 1 1/ 1/ K = = m x m x x π x x x x x x = Σ σ σ σ σ σ σ

46 D Gaussan dstrbuton - example Let m 1 =m =0: 6 4 Σ = Σ 1 = Σ = p ( x ) 1 T = exp 1/ x ( π ) 1/ 1 1 x = Aexp( B) A = 1 = 1 1/ 1/ ( π ) 8 4 π B = 1 1 x 1 1 x 1 x x x x 1

47 Colour based segmentaton {R,G,B} Multvarate Gaussan dstrbuton B G R

48 Selecton of the feature vector Feature vector: coordnates θ = { x, y, R, G, B, u, v} colour u v dx = dt dy = dt optcal flow

49 Optcal flow features Optcal flow based segmentaton {u,v} optcal flow vector feld

50 k-nearest Neghbours Classfer (k-nn) x x In each pont of the feature space, a number of k- neghbours s counted. The pont s assgned to the class from whch there s the largest number of ponts n the neghbourhood. x 1

51 Image segmentaton examples

52 Image segmentaton examples

53 Fngerprnt recognton FBI mage database of fngerprnts (199)

54 Irs recognton (bometry) The patented Daugman s algorthm

55 Analyss of bomedcal mages Mamografa

56 Image database querng Idea of the search mage or a copy of the mage database ht DWT C.E. Jacobs, A. Fnkelsten, D.H. Saless, Fast multresoluton mage querng, 1999

57 Analyss of X-ray mages of grans P. Strumllo, J. Newczas, P. Szczypńsk, P. Makowsk, W. Woznak, Computer system for analyss of X ray mages of wheat grans, Internatonal Agrophyscs, 1999, vol. 13, No. 1, pp

58 Automatc text detecton