5. POLARIMETRIC SAR DATA CLASSIFICATION
|
|
- Brittney Jacobs
- 5 years ago
- Views:
Transcription
1 Polarmetrc SAR data Classfcaton 5. POLARIMETRIC SAR DATA CLASSIFICATION 5.1 Classfcaton of polarmetrc scatterng mechansms - Drect nterpretaton of decomposton results - Cameron classfcaton - Lee classfcaton based on Freeman decomposton H-A-α classfcaton of scatterng mechansms Cloude and Potter proposed an algorthm to dentfy n an unsupervsed way polarmetrc scatterng mechansms n the H-α plane. The key dea s that entropy arses as a natural measure of the nherent reversblty of the scatterng data and that α can be used to dentfy the underlyng average scatterng mechansm. Fgure 1 Optcal mage (left, polarmetrc color coded mage (rghtof the Oberpfaffenhofen test ste. The H-α classfcaton plane s sub-dvded nto 8 basc zones characterstc of dfferent scatterng behavors. The basc scatterng mechansm of each pxel of a polarmetrc SAR mage can then be dentfed by comparng ts entropy and α parameters to fxed thresholds. The dfferent class boundares, n the H-α plane, have been determned so as to dscrmnate surface reflecton (SR, volume dffuson (VD and double bounce reflecton (DB along the α axs and low, medum and hgh degree of randomness along the entropy axs. Detaled explanatons, examples and comments concernng the dfferent classes can be found n the publcaton from Cloude and Potter. Fgure 2 shows the H-α plane and the occurrence of the studed polarmetrc data nto ths plane. 1
2 Polarmetrc SAR data Classfcaton α( H A Fgure 2 Polarmetrc decomposton man parameters: H, A and α. Grey zones n the H-α plane correspond to unfeasble areas. It can be seen, n Fgure 3, that the largest denstes n the occurrence plane correspond to volume dffuson and double bounce scatterng wth moderate and hgh randomness. Medum and low entropy surface scatterng mechansms are also frequently encountered n the scene under examnaton. 2
3 Polarmetrc SAR data Classfcaton α( 90 DR VD SR H Fgure 3 H-α scatterng mechansm dentfcaton plane (top. Polarmetrc data occurrence n the H- α plane (bottom. Data dstrbuton n the H-α plane does not show, for the consdered scene, dstnct natural clusters belongng to a sngle scatterng mechansm class. Therefore, dentfcaton results may hghly depend on segmentaton thresholds. Results of the unsupervsed dentfcaton procedure are presented n Fgure 4. It can be observed n Fgure 4 that the proposed segmentaton n the H-α plane permts to dentfy n a macroscopc way the type of scatterng mechansm. Agrcultural felds and bare sols are characterzed by surface scatterng. Scatterng over forested areas s domnated by volume dffuson whle urban areas are manly characterzed by double bounce scatterng. It may be noted that the dentfcaton process slghtly overestmates volume dffuson and double bounce scatterng over surfaces. 3
4 Polarmetrc SAR data Classfcaton Fgure 4 Unsupervsed scatterng mechansm dentfcaton n the H-α plane. 5.2 Polarmetrc data statstcal segmentaton Maxmum Lkelhood Supervsed segmentaton Bayes optmal decson rule The problem of optmal segmentaton over a SAR mage over a fxed number of clusters may be formulated as follows. How to assgn, n an optmal way, a SAR mage pxel, {, L, } 1 M, accordng to a SAR observable, x? p, to one of the M possble clusters The answer s gven by Bayes optmal decson rule usng the condtonal probablty of the dfferent clusters Decde p f P( x > P( x (1 A pxel s assgned to the most probable cluster condtonally to the observaton of x over the pxel under consderaton. The probablty of error related to ths decson s obtan from the a posteror probabltes of the unselected clusters by ( x P ( error x P P( x It s obvous, that assgnng a pxel to the cluster wth the hghest a posteror probablty mnmzes the condtonal error probablty. The total probablty s then computed usng (2 4
5 Polarmetrc SAR data Classfcaton P ( error x P( error x P( x dx (3 The set of a posteror probabltes P( x express such quanttes usng Bayes rule P ( x P x P( x P( s generally uneasy to derve. One prefers then to ( (4 Insertng ths expresson n (1, the expresson of the optmal decson rule becomes Decde p f P( x P( > P( x P( (5 Where P( x s the lkelhood of the observaton x, gven and probablty of. P( s the pror Addtonally, f the pror probabltes are supposed to be equal, the optmal decson rule may wrtten usng only cluster lkelhood as Decde p f Arg max P( x ( Supervsed segmentaton scheme A supervsed segmentaton scheme may be represented as a three-step process descrbed n the followng. Frst tranng data, separated nto M dstnct groups or classes, are suppled to the segmentaton algorthm. Tranng nputs may be : - user suppled under the correct format - sampled from the mage under study - sampled from SAR mages acqured over other scenes Durng the second phase, the segmentaton algorthm learns from the tranng data sets how to dscrmnate the M classes and establshes decson rules. In the case of supervsed segmentaton based on Bayes optmal decson rule, the segmentaton algorthm learns the dfferent statstcal quanttes useful to compute the expresson mentoned n (6. Once the learnng phase has converged, the algorthm assgns each pxel of the SAR mage under study to one of the M clusters provded n the frst phase, accordng to decson procedure derved durng the second step Supervsed ML segmentaton of [S matrx mages In the case of [S matrx mages, the SAR observaton x s replaced by a 3 element complex T target vector k S, 2S, S. [ It has been verfed that when the radar llumnates an area of random surface of many elementary scatterers, k can be modeled as havng a multvarate complex crcular gaussan probablty densty functon N C ( 0,[ Σ of the form : 5
6 Polarmetrc SAR data Classfcaton exp( k [ Σ P( k q π [ Σ 1 k (7 where q stands for the number of elements of k, equal to three n the monostatc case, represents the determnant, and [ Σ E( kk s the global (3 3 covarance matrx of k. The lkelhood of a target vector k gven a cluster s then gven by ( P k exp( k [ Σ q π [ Σ 1 k (8 Wth [ Σ practce the actual value of the covarance matrx of cluster [ Σ ts maxmum lkelhood estmate Wth n computed durng the learnng phase. In remans unknown and the covarance matrx s replaced by Σˆ defned as [ 1 [ Σˆ kk (9 n p the number of pxels belongng to the tranng cluster From a computatonal pont of vew, t s generally preferable to deal wth log-lkelhood functon, L( k nstead of the expresson gven n (8 Where ( ˆ 1 tr [ kk L ( ln[ ˆ ([ ˆ 1 k Σ tr Σ kk q lnπ. 1 Σ stands for the trace of ˆ 1 [ Σ kk and equals k [ ˆ Σ k. The logarthm functon beng strctly ncreasng wth ts argument, the optmal decson rule s transformed, n the case of [S matrx segmentaton, to Decde p f ( [ ˆ Arg mn d k Σ wth ˆ 1 ln[ ˆ [ ˆ (11 d k [ Σ + Σ + tr Σ kk ( ( Where the varable ( ˆ (10 d k [ Σ may be assmlated to a statstcal dstance, derved from the opposte of log-lkelhood functon wthout constant terms. Durng the segmentaton process a pxel s then assgned to the mnmum dstance cluster, havng the closest statstcs. The supervsed segmentaton algorthm may be summarzed as follows 6
7 Polarmetrc SAR data Classfcaton Intalze pxel dstrbuton over M custers fro m tranng data sets For each cluster 1 [ Σˆ k k N For each pxel, k m f d ( k, < d ( k / 1, L, M Fgure 5 Supervsed ML segmentaton scheme Supervsed ML segmentaton of [T or [C matrx mages It has been shown that assumng that target vectors have a N C ( 0,[ Σ dstrbuton, a sample n-look covarance matrx degrees of freedom, Where W C [ C 1/ n kk, follows a complex Wshart dstrbuton wth n ( n,[ Σ, gven by n p([ T wth qn n [ T n q K( n, q π Γ(. represents the Gamma functon. exp( tr( n[ Σ K( n, q[ Σ q( q 1 / 2 q 1 n 1 [ T Γ( n + 1 A development smlar to the one presented n the case of scatterng matrces leads to the followng decson rule Decde p f ( [ ˆ Arg max L [ T Σ wth [ ˆ ln[ ˆ [ ˆ 1 (13 L [ T Σ n Σ ntr Σ [ T + qnln n + ( n qln[ T ln K( n, q Wth ( ( Σˆ, the maxmum lkelhood estmate of the coherency matrx defned as [ 1 [ Σˆ [ T (14 n p Takng the opposte of the lower expresson of relaton (13 and removng terms that do not depend on the cluster under test, the optmal decson rule becomes Decde p f ( [ ˆ Arg mn d [ T Σ wth [ ˆ ln[ ˆ [ ˆ (15 d [ T Σ + Σ + tr Σ 1 [ T ( ( 7
8 Polarmetrc SAR data Classfcaton It may be noted that, snce covarance and coherence matrces are related by a smlarty transformaton, the form of the statstcal dstance expressed n (15 s nvarant under a change from coherency matrx form to covarance matrx form and (15 s adapted to both [T and [C representatons. The supervsed segmentaton algorthm may be summarzed as follows Intalze pxel dstrbuton over M custers fro m tranng data sets For each cluster 1 [ Σˆ [ T N For each pxel, [ T m f d ([ T, < d ([ T / 1, L, M Fgure 6 Supervsed ML segmentaton scheme Supervsed segmentaton performance The performance of a supervsed segmentaton process s generally estmated va a confuson matrx computed from the tranng data. Each element of the tranng data set s processed through the supervsed classfer. The confuson matrx summarzes the percentage of pxels belongng to an orgnal cluster,, effectvely affected to one of the M possble clusters, matrx of an deal, error-free, segmentaton s dagonal., wth 1L M. The confuson User-defned tranng data sets are generally orented towards the dscrmnaton of dfferent knds of medum and may not represent all scatterng behavors encountered over a natural scene. Therefore, some parts of a SAR mage may have statstcs that do not ft wth any of the proposed cluster ones. In ths case, the assgnment of the correspondng pxels may lead to rrelevant nterpretaton of the segmentaton results. One possble soluton to ths problem conssts n creatng a reect class. A pxel s assgned to ths reect class f t s too much dstant, from a statstcal pont of vew, from all the learnng clusters. A possble metrc may be defned from the statstcal dstances defned n (11 and (15. A pxel belongs to the reect class f ts dstance to the closest cluster exceeds a gven factor of the wthn-class dstance standard devaton. Ths decson process may be summarzed as follows Decde p re d [ T [ Σ ˆ > σ d Arg mn d [ T [ Σˆ f ( r wth (. (16 8
9 Polarmetrc SAR data Classfcaton and ( ( σ d [ T [ ˆ d [ T [ ˆ d Σ N p N p Where the varable r represents a user-defned reecton factor. Σ Maxmum Lkelhood unsupervsed segmentaton It has been showed, n former paragraphs that a pxel can be assgned to a cluster n an optmal way, accordng to the followng decson rule. Decde p f Arg max P( x (17 In the case of an unsupervsed segmentaton scheme, the lkelhood of a cluster, ( x P, cannot be estmated snce ts calculaton requres to compute the ML estmate of the cluster coherency matrx from the samples contaned n the cluster. An optmal soluton to the unsupervsed segmentaton problem conssts n dstrbutng the pxels of an mage over the set of M clusters so as to maxmze the global lkelhood defned as the product of all ndvdual lkelhood functons. A rgorous resoluton requres to test all the possble combnatons and to select the one correspondng to the maxmum ont lkelhood value. Ths optmal soluton cannot be appled due to the unrealstc computatonal load t nvolves. Alternatve solutons based on sub-optmal teratve optmzaton procedures are generally preferred Prncple of statstcal K-means clusterng The K-means procedure s an teratve optmzaton algorthm descrbed by the followng synopss : 9
10 Polarmetrc SAR data Classfcaton Intalze pxel dstrbuton over M clusters Compute each cluster center Assgn each pxel to the closests cluster No Convergence? Yes Fgure 7 Synopss of the K-means clusterng algorthm. The algorthms begns wth the ntalzaton of the mage pxel dstrbuton over the M clusters. Ths dstrbuton may be done n a random way or accordng to user specfcatons. Once all pxels are affected, the dfferent cluster centers are computed accordng to the processed data type. Each pxel s then affected to the closest cluster accordng to a dstance measure. The convergence of the algorthm s then tested usng stablty metrcs. If a termnaton crteron s met, the segmentaton stops, otherwse a new teraton starts over from the class center computaton step. The K-means algorthm ams to optmze a global functon by teratvely optmzng local expressons. It s known ths type of technques may get stuck nto locally stable states and fal to determne the optmal pxel dstrbuton. The ntalzaton of the pxel dstrbuton nto N classes s then a crtcal stage of the K-mean clusterng algorthm. An adequate ntalzaton permts a fast convergence and provdes correctly segmented clusters. The convergence of the algorthm s evaluated by testng a condton of termnaton. Such a crteron may be defned from the estmaton of the classfcaton qualty, or consst n a maxmum number of teratons or n a suffcently low number of pxels that are dfferently classfed from one teraton to the other Unsupervsed ML segmentaton of [T or [C matrx mages The K-means clusterng algorthm may be adapted to the segmentaton of [T or [C polarmetrc representatons as shown on the followng synopss 10
11 Polarmetrc SAR data Classfcaton Intalze pxel dstrbuton over M custers For each cluster 1 [ Σˆ [ T N For each pxel, [ T m f d ([ T, < d ([ T / 1, L, M No Convergence? Yes Fgure 8 Unsupervsed Maxmum Lkelhood segmentaton scheme. The class center ntalzaton step conssts n computng each cluster coherency matrx ML estmate. Pxels are then affected to a cluster accordng to the ML dstance measure derved n ( Unsupervsed Wshart H-α segmentaton The partcularty of the dentfcaton procedure based on H and α, ntroduced n a former paragraph, resdes n the estmaton of the type of observed meda from a physcal nterpretaton of canoncal scatterng mechansms usng robust ndcators. Nevertheless, the analyss of natural scenes usng ths unsupervsed approach may reach some lmtatons: - The arbtrarly fxed lnear decson boundares n the H-α plane may not ft data dstrbuton. A natural cluster correspondng to smlar targets may le across a fronter n the decson plane. In ths case, pxels wth very smlar characterstcs may be assgned, n an almost random way, to dfferent classes due slghtly dfferent locatons n the H-α plane. Ths effect can be observed n Fgure 4 where the varablty n natural meda polarmetrc features lead to nosy classfcaton results. - Even f the computaton of H and α requres fully polarmetrc data, these two parameters do not represent the whole polarmetrc nformaton. The use of other ndcators such as the span or specfc correlatons coeffcents may mprove the classfcaton results n a sgnfcant way. Segmentaton procedures based on the whole coherency matrx statstcs permt to overcome the lmtatons mentoned above. Nevertheless, t s shown n the followng, that the physcal nterpretaton of the scatterng phenomenon permts to enhance n a sgnfcant way the performance of statstcal segmentaton schemes. 11
12 Polarmetrc SAR data Classfcaton The unsupervsed segmentaton scheme mentoned above s ntalzed n an effcent way wth the results of the unsupervsed dentfcaton of a scatterng mechansm, usng H, A and α. Ths ntalzaton provdes 8 stable clusters relatng to the underlyng physcal scatterng mechansm. Intalze pxel dstrbuton over 8 clusters For each cluster compute Σ ˆ [ Segmented H- α plane Assgn each pxel to a cluster No Convergence? Yes Fgure 9 Unsupervsed Wshart H-α segmentaton scheme. Fgure 10 Wshart H-α segmentaton results. C1 C2 C3 C4 C5 C6 C7 C8 An mportant mprovement n the segmentaton accuracy can be observed n the mage presented n Fgure 10. The man knds of natural meda are clearly dscrmnated by the Wshart H-α segmentaton scheme. Ths unsupervsed classfcaton algorthm modfes the 12
13 Polarmetrc SAR data Classfcaton decson boundares n an adaptve way to better ft the natural dstrbuton of the scatterng mechansms and takes nto account nformaton related to the back-scattered power Unsupervsed Wshart H-A-α segmentaton The ML Wshart segmentaton may be further mproved by explctly ncludng the ansotropy nformaton durng the segmentaton procedure. As mentoned prevously, the ansotropy ndcates the relatve mportance of secondary mechansms obtaned from the expanson of a coherency matrx. Ths polarmetrc ndcator s partcularly useful to dscrmnate scatterng mechansms wth dfferent egenvalue dstrbutons but wth smlar ntermedate entropy values. In such cases, a hgh ansotropy value ndcates two domnant scatterng mechansms wth equal probablty and a less sgnfcant thrd mechansm, whle a low ansotropy value corresponds to a domnant frst scatterng mechansm and two non-neglgble secondary mechansms wth equal mportance. Among the dfferent approaches tested, the best way to ntroduce the ansotropy nformaton n the classfcaton algorthm conssts n mplementng two successve segmentaton procedures as shown n Fgure 11. Intalze pxel dstrbuton over 8 clusters Segmented H- α plane A > 0.5 Χ m Χ m clusters Ansotropy Fgure 11 Unsupervsed Wshart H-A-α segmentaton scheme. Polarmetrc data are frst segmented accordng to the algorthm presented n the former paragraph. Once ths procedure has converged, the 8 resultng clusters are splt nto 16 ones 13
14 Polarmetrc SAR data Classfcaton by comparng the ansotropy of each pxel to a threshold fxed to 0.5. The 16 segments are then used to ntalze a second Wshart ML segmentaton procedure. C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 Fgure 12 Wshart H-A-α segmentaton results. The segmentaton results presented n Fgure 12 show an enhanced descrpton of the Oberpfaffenhofen scene. The ntroducton of the ansotropy n the clusterng process permts to splt large segments nto smaller clusters dscrmnatng small dspartes n a refned way. Several knds of agrcultural felds are separated. The runway and other low ntensty targets are dstngushed from other surfaces. Buldngs are dscrmnated from other types of scatterers present n urban areas. The Wshart H-A-α classfcaton scheme gathers nto segments pxels wth smlar statstcal propertes, but does not provde any nformaton concernng the nature of the scatterng mechansm assocated to each cluster. 14
Composite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationLecture 12: Classification
Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationA Bayes Algorithm for the Multitask Pattern Recognition Problem Direct Approach
A Bayes Algorthm for the Multtask Pattern Recognton Problem Drect Approach Edward Puchala Wroclaw Unversty of Technology, Char of Systems and Computer etworks, Wybrzeze Wyspanskego 7, 50-370 Wroclaw, Poland
More informationUnified Subspace Analysis for Face Recognition
Unfed Subspace Analyss for Face Recognton Xaogang Wang and Xaoou Tang Department of Informaton Engneerng The Chnese Unversty of Hong Kong Shatn, Hong Kong {xgwang, xtang}@e.cuhk.edu.hk Abstract PCA, LDA
More informationMixture o f of Gaussian Gaussian clustering Nov
Mture of Gaussan clusterng Nov 11 2009 Soft vs hard lusterng Kmeans performs Hard clusterng: Data pont s determnstcally assgned to one and only one cluster But n realty clusters may overlap Soft-clusterng:
More informationLecture Nov
Lecture 18 Nov 07 2008 Revew Clusterng Groupng smlar obects nto clusters Herarchcal clusterng Agglomeratve approach (HAC: teratvely merge smlar clusters Dfferent lnkage algorthms for computng dstances
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationRelevant polarimetric parameters for surface characterization using SAR data
Relevant polarmetrc parameters for surface characterzaton usng SAR data INTRODUCTION S. Allan, L. Ferro-Faml, E. Potter Unversty of Rennes I.E.T.R, UMR CNRS 664, Image and Remote Sensng Group Campus de
More informationBoostrapaggregating (Bagging)
Boostrapaggregatng (Baggng) An ensemble meta-algorthm desgned to mprove the stablty and accuracy of machne learnng algorthms Can be used n both regresson and classfcaton Reduces varance and helps to avod
More informationMULTISPECTRAL IMAGE CLASSIFICATION USING BACK-PROPAGATION NEURAL NETWORK IN PCA DOMAIN
MULTISPECTRAL IMAGE CLASSIFICATION USING BACK-PROPAGATION NEURAL NETWORK IN PCA DOMAIN S. Chtwong, S. Wtthayapradt, S. Intajag, and F. Cheevasuvt Faculty of Engneerng, Kng Mongkut s Insttute of Technology
More informationFinite Mixture Models and Expectation Maximization. Most slides are from: Dr. Mario Figueiredo, Dr. Anil Jain and Dr. Rong Jin
Fnte Mxture Models and Expectaton Maxmzaton Most sldes are from: Dr. Maro Fgueredo, Dr. Anl Jan and Dr. Rong Jn Recall: The Supervsed Learnng Problem Gven a set of n samples X {(x, y )},,,n Chapter 3 of
More informationClustering & Unsupervised Learning
Clusterng & Unsupervsed Learnng Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Wnter 2012 UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y
More informationPsychology 282 Lecture #24 Outline Regression Diagnostics: Outliers
Psychology 282 Lecture #24 Outlne Regresson Dagnostcs: Outlers In an earler lecture we studed the statstcal assumptons underlyng the regresson model, ncludng the followng ponts: Formal statement of assumptons.
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationClassification as a Regression Problem
Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationClustering & (Ken Kreutz-Delgado) UCSD
Clusterng & Unsupervsed Learnng Nuno Vasconcelos (Ken Kreutz-Delgado) UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y ), fnd an approxmatng
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationA Robust Method for Calculating the Correlation Coefficient
A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More information2016 Wiley. Study Session 2: Ethical and Professional Standards Application
6 Wley Study Sesson : Ethcal and Professonal Standards Applcaton LESSON : CORRECTION ANALYSIS Readng 9: Correlaton and Regresson LOS 9a: Calculate and nterpret a sample covarance and a sample correlaton
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationMaximum Likelihood Estimation (MLE)
Maxmum Lkelhood Estmaton (MLE) Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Wnter 01 UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationUncertainty in measurements of power and energy on power networks
Uncertanty n measurements of power and energy on power networks E. Manov, N. Kolev Department of Measurement and Instrumentaton, Techncal Unversty Sofa, bul. Klment Ohrdsk No8, bl., 000 Sofa, Bulgara Tel./fax:
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationPulse Coded Modulation
Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationChapter 12 Analysis of Covariance
Chapter Analyss of Covarance Any scentfc experment s performed to know somethng that s unknown about a group of treatments and to test certan hypothess about the correspondng treatment effect When varablty
More informationStatistics II Final Exam 26/6/18
Statstcs II Fnal Exam 26/6/18 Academc Year 2017/18 Solutons Exam duraton: 2 h 30 mn 1. (3 ponts) A town hall s conductng a study to determne the amount of leftover food produced by the restaurants n the
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationWhy Bayesian? 3. Bayes and Normal Models. State of nature: class. Decision rule. Rev. Thomas Bayes ( ) Bayes Theorem (yes, the famous one)
Why Bayesan? 3. Bayes and Normal Models Alex M. Martnez alex@ece.osu.edu Handouts Handoutsfor forece ECE874 874Sp Sp007 If all our research (n PR was to dsappear and you could only save one theory, whch
More informationConjugacy and the Exponential Family
CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the
More informationCluster Validation Determining Number of Clusters. Umut ORHAN, PhD.
Cluster Analyss Cluster Valdaton Determnng Number of Clusters 1 Cluster Valdaton The procedure of evaluatng the results of a clusterng algorthm s known under the term cluster valdty. How do we evaluate
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationClustering gene expression data & the EM algorithm
CG, Fall 2011-12 Clusterng gene expresson data & the EM algorthm CG 08 Ron Shamr 1 How Gene Expresson Data Looks Entres of the Raw Data matrx: Rato values Absolute values Row = gene s expresson pattern
More information9.913 Pattern Recognition for Vision. Class IV Part I Bayesian Decision Theory Yuri Ivanov
9.93 Class IV Part I Bayesan Decson Theory Yur Ivanov TOC Roadmap to Machne Learnng Bayesan Decson Makng Mnmum Error Rate Decsons Mnmum Rsk Decsons Mnmax Crteron Operatng Characterstcs Notaton x - scalar
More informationFREQUENCY DISTRIBUTIONS Page 1 of The idea of a frequency distribution for sets of observations will be introduced,
FREQUENCY DISTRIBUTIONS Page 1 of 6 I. Introducton 1. The dea of a frequency dstrbuton for sets of observatons wll be ntroduced, together wth some of the mechancs for constructng dstrbutons of data. Then
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationPATTERN RECOGNITION AND IMAGE UNDERSTANDING
PATTERN RECOGNITION AND IMAGE UNDERSTANDING The ultmate objectve of many mage analyss tasks s to dscover meanng of the analysed mage, e.g. categorse the objects, provde symbolc/semantc nterpretaton of
More informationMarkov Chain Monte Carlo Lecture 6
where (x 1,..., x N ) X N, N s called the populaton sze, f(x) f (x) for at least one {1, 2,..., N}, and those dfferent from f(x) are called the tral dstrbutons n terms of mportance samplng. Dfferent ways
More informationGaussian Mixture Models
Lab Gaussan Mxture Models Lab Objectve: Understand the formulaton of Gaussan Mxture Models (GMMs) and how to estmate GMM parameters. You ve already seen GMMs as the observaton dstrbuton n certan contnuous
More informationTurbulence classification of load data by the frequency and severity of wind gusts. Oscar Moñux, DEWI GmbH Kevin Bleibler, DEWI GmbH
Turbulence classfcaton of load data by the frequency and severty of wnd gusts Introducton Oscar Moñux, DEWI GmbH Kevn Blebler, DEWI GmbH Durng the wnd turbne developng process, one of the most mportant
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationDETERMINATION OF UNCERTAINTY ASSOCIATED WITH QUANTIZATION ERRORS USING THE BAYESIAN APPROACH
Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata TC XVII IMEKO World Congress Metrology n the 3rd Mllennum June 7, 3,
More informationINF 5860 Machine learning for image classification. Lecture 3 : Image classification and regression part II Anne Solberg January 31, 2018
INF 5860 Machne learnng for mage classfcaton Lecture 3 : Image classfcaton and regresson part II Anne Solberg January 3, 08 Today s topcs Multclass logstc regresson and softma Regularzaton Image classfcaton
More informationMACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression
11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationThe big picture. Outline
The bg pcture Vncent Claveau IRISA - CNRS, sldes from E. Kjak INSA Rennes Notatons classes: C = {ω = 1,.., C} tranng set S of sze m, composed of m ponts (x, ω ) per class ω representaton space: R d (=
More informationNegative Binomial Regression
STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationQuantitative Discrimination of Effective Porosity Using Digital Image Analysis - Implications for Porosity-Permeability Transforms
2004, 66th EAGE Conference, Pars Quanttatve Dscrmnaton of Effectve Porosty Usng Dgtal Image Analyss - Implcatons for Porosty-Permeablty Transforms Gregor P. Eberl 1, Gregor T. Baechle 1, Ralf Weger 1,
More informationAPPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14
APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationEstimating the Fundamental Matrix by Transforming Image Points in Projective Space 1
Estmatng the Fundamental Matrx by Transformng Image Ponts n Projectve Space 1 Zhengyou Zhang and Charles Loop Mcrosoft Research, One Mcrosoft Way, Redmond, WA 98052, USA E-mal: fzhang,cloopg@mcrosoft.com
More informationCSC 411 / CSC D11 / CSC C11
18 Boostng s a general strategy for learnng classfers by combnng smpler ones. The dea of boostng s to take a weak classfer that s, any classfer that wll do at least slghtly better than chance and use t
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationProbability Density Function Estimation by different Methods
EEE 739Q SPRIG 00 COURSE ASSIGMET REPORT Probablty Densty Functon Estmaton by dfferent Methods Vas Chandraant Rayar Abstract The am of the assgnment was to estmate the probablty densty functon (PDF of
More informationRegularized Discriminant Analysis for Face Recognition
1 Regularzed Dscrmnant Analyss for Face Recognton Itz Pma, Mayer Aladem Department of Electrcal and Computer Engneerng, Ben-Guron Unversty of the Negev P.O.Box 653, Beer-Sheva, 845, Israel. Abstract Ths
More informationCommunication with AWGN Interference
Communcaton wth AWG Interference m {m } {p(m } Modulator s {s } r=s+n Recever ˆm AWG n m s a dscrete random varable(rv whch takes m wth probablty p(m. Modulator maps each m nto a waveform sgnal s m=m
More informationChat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980
MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and
More informationA General Method for Assessing the Uncertainty in Classified Remotely Sensed Data at Pixel Scale
Proceedngs of the 8th Internatonal Symposum on Spatal Accuracy Assessment n Natural Resources and Envronmental Scences Shangha, P. R. Chna, June 25-27, 2008, pp. 86-94 A General ethod for Assessng the
More informationThermodynamics and statistical mechanics in materials modelling II
Course MP3 Lecture 8/11/006 (JAE) Course MP3 Lecture 8/11/006 Thermodynamcs and statstcal mechancs n materals modellng II A bref résumé of the physcal concepts used n materals modellng Dr James Ellott.1
More informationGrover s Algorithm + Quantum Zeno Effect + Vaidman
Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the
More informationSDMML HT MSc Problem Sheet 4
SDMML HT 06 - MSc Problem Sheet 4. The recever operatng characterstc ROC curve plots the senstvty aganst the specfcty of a bnary classfer as the threshold for dscrmnaton s vared. Let the data space be
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationNatural Images, Gaussian Mixtures and Dead Leaves Supplementary Material
Natural Images, Gaussan Mxtures and Dead Leaves Supplementary Materal Danel Zoran Interdscplnary Center for Neural Computaton Hebrew Unversty of Jerusalem Israel http://www.cs.huj.ac.l/ danez Yar Wess
More informationRELIABILITY ASSESSMENT
CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department
More informationGoodness of fit and Wilks theorem
DRAFT 0.0 Glen Cowan 3 June, 2013 Goodness of ft and Wlks theorem Suppose we model data y wth a lkelhood L(µ) that depends on a set of N parameters µ = (µ 1,...,µ N ). Defne the statstc t µ ln L(µ) L(ˆµ),
More informationChapter - 2. Distribution System Power Flow Analysis
Chapter - 2 Dstrbuton System Power Flow Analyss CHAPTER - 2 Radal Dstrbuton System Load Flow 2.1 Introducton Load flow s an mportant tool [66] for analyzng electrcal power system network performance. Load
More informationInductance Calculation for Conductors of Arbitrary Shape
CRYO/02/028 Aprl 5, 2002 Inductance Calculaton for Conductors of Arbtrary Shape L. Bottura Dstrbuton: Internal Summary In ths note we descrbe a method for the numercal calculaton of nductances among conductors
More informationis the calculated value of the dependent variable at point i. The best parameters have values that minimize the squares of the errors
Multple Lnear and Polynomal Regresson wth Statstcal Analyss Gven a set of data of measured (or observed) values of a dependent varable: y versus n ndependent varables x 1, x, x n, multple lnear regresson
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationENG 8801/ Special Topics in Computer Engineering: Pattern Recognition. Memorial University of Newfoundland Pattern Recognition
EG 880/988 - Specal opcs n Computer Engneerng: Pattern Recognton Memoral Unversty of ewfoundland Pattern Recognton Lecture 7 May 3, 006 http://wwwengrmunca/~charlesr Offce Hours: uesdays hursdays 8:30-9:30
More information3.1 ML and Empirical Distribution
67577 Intro. to Machne Learnng Fall semester, 2008/9 Lecture 3: Maxmum Lkelhood/ Maxmum Entropy Dualty Lecturer: Amnon Shashua Scrbe: Amnon Shashua 1 In the prevous lecture we defned the prncple of Maxmum
More informationUncertainty as the Overlap of Alternate Conditional Distributions
Uncertanty as the Overlap of Alternate Condtonal Dstrbutons Olena Babak and Clayton V. Deutsch Centre for Computatonal Geostatstcs Department of Cvl & Envronmental Engneerng Unversty of Alberta An mportant
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More information