INF 43 3.. Repeon Anne Solberg (anne@f.uo.no Bayes rule for a classfcaon problem Suppose we have J, =,...J classes. s he class label for a pxel, and x s he observed feaure vecor. We can use Bayes rule o fnd an expresson for he class wh he hghes probably: p( x P( P( x p( x pror probably poseror probably lkelhood normalzng facor P( s he pror probably for class. If we don' have specal knowledge ha one of he classes occur more frequen han oher classes, we se hem equal for all classes. (P( =/J, =.,,,J. 3.. INF 43 INF 43 Eucldean dsance vs. Mahalanobs dsance Dscrmnan funcons for he normal densy Eucldean dsance beween pon x and class cener : T x x x Mahalanobs dsance beween ee x and : r T x x We saw las lecure ha he mnmum-error-rae rae classfcaon can compued usng he dscrmnan funcons g ( x p ( x P ( Wh a mulvarae Gaussan we ge: g ( x ( x μ d ( x μ P( Le u look a hs expresson for some specal cases: INF 43 3 INF 43 4
Case : Σ =σ I Asmplemodel model, Σ =σ I An equvalen formulaon of he dscrmnan funcons: g ( x w x w where w μ and ( w μ μ P The equaon g (x=g (x canbe wrenas w ( x x where w μ -μ P( μ - μ μ - μ and x μ -μ P( w= - s he vecor beween he mean values. Ths equaon defnes a hyperplane hrough he pon x, and orhogonal o w. If P( =P( he hyperplane wll be locaed halfway beween he mean values. INF 43 5 The dsrbuons are sphercal n d dmensons. The decson boundary s a generalzed hyperplane of d- dmensons The decson boundary s perpendcular o he e separang he wo mean values Ths knd of a classfer s called a ear classfer, or a ear dscrmnan funcon Because he decson funcon s a ear funcon of x. If P( = P(, he decson boundary wll be half-way beween and INF 43 6 Case : Common covarance, Σ = Σ If we assume ha all classes have he same shape of daa clusers, an nuve model s o assume ha her probably dsrbuons have he same shape By hs assumpon we can use all he daa o esmae he covarance marx Ths esmae s common for all classes, and hs means ha also n hs case he dscrmnan funcons become ear funcons T g ( x ( x μ Σ ( x μ Σ P ( T T T ( x Σ x μ ( Σ x μ Σ μ Σ P ( ( I Case : Common covarance, Σ = Σ An equvalen formulaon of he dscrmnan funcons s g ( x w x w where w Σ μ and w μ Σ μ P( The decson boundares are agan hyperplanes. Because w = Σ - ( - s no n he drecon of ( -, he hyperplan wl no be orhogonal o he e beween he means. Common for all classes, no need o compue Snce x T x s common for all classes, g (x agan reduces o a ear funcon of x. INF 43 7 INF 43 8
Case 3:, Σ =arbrary The dscrmnan funcons wll be quadrac: g ( x x W x w x w where W Σ, w Σ μ and w μ Σ μ Σ P( The decson surfaces are hyperquadrcs and can assume any of he general forms: hyperplanes p hypershperes pars of hyperplanes hyperellsods, hyperparabolods hyperhyperbolod The curse of dmensonaly In pracce, he curse means ha, for a gven sample sze, here s a maxmum number of feaures one can add before he classfer sars o degrade. 6.. INF 43 9 INF 43 How do we bea he curse of dmensonaly? Exploraory daa analyss Use regularzed esmaes for he Gaussan case Use dagonal covarance marces Apply regularzed covarance esmaon (INF 53 Generae few, bu nformave feaures Careful feaure desgn gven he applcaon Reducng he dmensonaly Feaure selecon (INF53 Feaure ransforms (INF 53 For a small number of feaures, manual daa analyss o sudy he feaures s recommended. d Choose nellgen feaures. Evaluae e.g. Error raes for sngle- feaure classfcaon Scaer plos Scaer plos of feaure combnaons INF 43 INF 43
Desgn nellgen feaures! Remember ha we alked abou feaure exracon pror o classfcaon: Desgn feaures alored o he applcaon Thnk carefully before mplemenng a feaure - wha knd of nformaon do we hnk s useful o separae he classes n our classfcaon problem. Good feaures are more mporan han he choce of classfer! Is he Gaussan classfer he only choce? The Gaussan classfer gves ear or quadrac dscrmnan funcon. Oher classfers can gve arbrary complex decson surfaces (ofen pecewse-ear Neural neworks Suppor vecor machnes knn (k-neares-neghbor classfcaon Bu wh good feaure ha are well separaed, he choce of classfer s no so mporan. INF 43 3 INF 43 4 k-neares-neghbor Neghbor classfcaon A very smple classfer. Classfcaon of a new sample x s done as follows: Ou of N ranng vecors, denfy he k neares neghbors (measure by Eucldean dsance n he ranng se, rrespecvely of he class label. k should be odd. Ou of hese k samples, denfy he number of vecors k ha belong o class, :,,...M (f we have M classes Assgn x o he class wh he maxmum number of k samples. k mus be seleced a pror. Morphology Repeon of bnary dlaaon, eroson, openng, closng Bnary regon processng: conneced componens, convex hull, hnnng, skeleon. Grey-level morphology: eroson, dlaon, openng, closng, smoohng, graden, opha, boom-ha, granulomery. INF 43 5 INF 43 6
Openng Closng Eroson of an mage removes all srucures ha he srucurng elemen can no f nsde, d and shrnks all oher srucures. If we dlae he resul of he eroson wh he same srucurng elemen, he srucures ha survved he eroson (were shrunken, no deleed d wll be resored. Ths s calles morphologcal openng: f S f θ S S A dlaon of an obec grows he obec and can fll gaps. If we erode he resul afer dlaon wh he roaed srucure elemen, he obecs wll keep her srucure and form, bu small holes flled by dlaon wll no appear. Obecs merged by he dlaon wll no be separaed agan. Closng s defned as f S f Ŝˆ θ S ˆ The name ells ha he operaon can creae an openng beween wo srucures ha are conneced only n a hn brdge, whou shrnkng he srucures (as eroson would do. Ths operaon can close gaps beween wo srucures whou growng he sze of he srucures lke dlaon would. INF 43 7 INF 43 8 Inerpreaon of grey-level openng and closng H or mss - ransformaon Inensy values are nerpreed as hegh curves over he (x,y- plane. Openng of f by b: push he srucure elemen up from below owards he surface of f. The value assgned s he hghes level b can reach. (smooh brgh values Closng: push he srucure elemen from above down a he surface of f. (smooh dark values Transformaon used o deec a gven paern n he mage emplae machng Subec: fnd exacly he shape gven by he obec D. D can f nsde many obecs, so we need o look a he local background W-D. Frs, compue he eroson of A by D, AθD (all pxels where D can f nsde A To f also he background: Compue A C, he complemen of A. The se of locaons where D exacly fs s he nersecon of AθD and he eroson of A C by W-D, A C θ(w-d. H-or-mss s expressed as A D: (AD AC ( W D Man use: Deecon of a gven paern or removal of sngle pxels INF 43 9 INF 43
Top-ha ransformaon Purpose: deec (or remove srucures of a ceran sze. Top-ha: lgh obecs on a dark background (also called whe op-ha. Boom-ha: dark obecs on a brgh background (also called black op-ha Top-ha: f ( f b Boom-ha: ( f b f Very useful for correcng uneven llumnaon/obecs on a varyng background INF 43