Recognition Performance from SAR Imagery Subject to System Resource Constraints
|
|
- Ada Rogers
- 5 years ago
- Views:
Transcription
1 Recognition Performance from SAR Imagery Subject to System Resource Constraints Michael D. DeVore Advisor: Joseph A. O SullivanO Washington University in St. Louis Electronic Systems and Signals Research Laboratory Supported by: ARO Center for Imaging Science DAAH ONR MURI N Boeing Foundation
2 Motivation Scene (source) Model Database (quantized source) Encoding Sensor Platform (channel) Interconnection Network (channel) ATR Processor (receiver) Inferred Target and Pose Inference based on conditional probabilistic models that incorporate assumptions and approximations Accuracy depends upon validity of models Resource consumption fixed by model and architecture Successive refinability to dynamically tradeoff accuracy and resource consumption 2
3 Outline Point #1: Assessment of model validity from sample data Point #2: Relating resource consumption to models Point #3: Hierarchical representations for refinability Conclusions 3
4 Point #1: Conditional Models for SAR Pixel distribution is a function of target class, pose, and location Pixels assumed pairwise independent Complex Gaussian: Pixel value r i distributed as pr ( i a,θ)= 1 ( ) e πσ i 2 a,θ r i µ i ( a,θ ) 2 σ 2 i ( a,θ ) Lognormal: x i = log r i is Gaussian Quarter power normal: x i = r i 1/2 is Gaussian px ( i a,θ)= 1 2πσ 2 i ( a,θ ) e ( ) 2 x i µ i ( a,θ ) 2σ 2 i ( a,θ ) 4
5 Point #1: Large Numbers of Small Samples Discretize azimuth, θ = observations per interval Assess model validity over 10 targets x 60 intervals x 400 pixels For each assumption evaluate H 0 : model valid vs. H A : model invalid Find test statistic D(r 1, r 2,, r N ), with known distribution under H 0 P-value is probability of a more extreme observation if H 0 were true: P = 1 - P D (d)) = Pr[D > d H 0 ] Small values of P evidence for rejecting H 0. P-value uniformly distributed when H 0 true: Pr[P D (D) p] = Pr[D P -1 D (p)] = P(P -1 (p)) = p P-value distribution for test of symmetry 5
6 Point #1: D Agostino-Pearson Method Let P G1 (g 1 ) and P G2 (g 2 ) be the cumulative distribution functions for sample skewness and kurtosis of N Gaussian random variables g 1 = k 3 32 g 2 = k 4 and k 2 k 2 2 where k i is the ith k-statistic D Agostino-Pearson statistic D DP given in terms of the inverse CDF of a standard normal variate, Φ -1 (z) [ ( ( )] 2 + Φ 1 P G2 ( g 2 ) D DP = Φ 1 P G1 g 1 [ ( )] 2 Pr[D DP > d] ] = 1 - P D (d)) is the P-value P of the test Distributions P G1 P and P G2 D approximated from 1,000,000 samples 6
7 Point #1: D Agostino-Pearson Method Quarter-power normal a good match, especially for clutter Gaussian model results unaffected by zero-mean assumption Log-normal model not as accurate for clutter pixels Pixels on Target Pixels on Clutter 7
8 Point #1: Kolmogorov-Smirnov Method From observations R 1, R 2,, R N P ˆ R ()= r 1 N u(r R i) i=1 where u( ) ) is the unit-step function N determine the empirical CDF The Kolmogorov-Smirnov test statistic is the supremum D KS = sup r P ˆ R ( r) P R ( r) Distribution of D KS is the same for all continuous P R Pr[D KS > d] ] is the P-value P of the test Probability computation: Press, et al. Numerical Recipes in C 8
9 Point #1: Kolmogorov-Smirnov Method Distribution parameters µ(a,θ) ) and σ 2 (a,θ)) are unknown Kolmogorov method requires known parameters» Convert independent Gaussian variates with unknown parameters to dependent T variates with known DOF» Test for T distribution Given independent Z 1, Z 2, Z N ( Z k Z ) N 1 N S 2 (k ) Gaussian with unknown µ, σ 2 is T distributed with N - 1 DOF, where 2 S (k ) = 1 N 1 ( Z j Z ) 2 (k) j k and Z (k ) = 1 N 1 Z j j k 9
10 Point #1: Kolmogorov-Smirnov Method Gaussian and quarter-power normal a better fit for clutter than target Lognormal less accurate for ground clutter Pixels on Target Pixels on Clutter 10
11 Point #2: Accuracy and Resource Consumption Scene (source) Model Database (quantized source) Encoding Sensor Platform (channel) Interconnection Network (channel) ATR Processor (receiver) Inferred Target and Pose ATR systems explicitly or implicitly based on target models with complexity C More complex models can yield better accuracy; Pr(error)=f(C,α SAR Complexity and computation power determine throughput; T CHIP SAR ) CHIP =h(c,α COMP ) Given an architecture, both accuracy,, Pr(error), and throughput, R=1/ CHIP, are parameterized by target model complexity =1/T CHIP 11
12 Point #2: Conditionally Gaussian Model Model each pixel i as independent, zero mean, complex conditionally Gaussian p ( R Θ,A,C 2 r θ, a,c 2 )= 1 π c 2 σ 2 i ( θ, a) e i r i 2 c 2 σ 2 i ( θ,a) Where: σ i2 (θ,a)) = variance function over pose and class c 2 = constant over all pixels to account for power fluctuation Recognition by maximizing the log-likelihood likelihood ratio * [ a ˆ, ˆ θ, c ˆ 2 ]= argmax [ a,θ,c 2 ] ln i ( ) p r a,θ,c 2 ( ) p r ξ 2 I i (a,θ) Where: ξ 2 = average clutter variance I i = mask function * Schmid & O Sullivan O Thresholding Method for Reduction of Dimensionality 12
13 Point #2: Target Model Estimation Given N registered training images q j of a target with pose φ j, estimate variances over N w windows of width d ˆ σ 2 ( θ k,a)= n 1 2 q k j { j:φ j w k } Registered Variance Images where w k = 2πk d N w 2,2πk + d N w 2 T 0 T 90 T 180 T 270 Variance estimate for an unregistered image r with pose θ formed by transforming the estimate from the closest w k Transformed Estimates 13
14 Point #2: Target Model Segmentation Pixel information relative to null-hypothesis used for target recognition Retain pixels i that are informative relative to the null-hypothesis: S a = i : 1 N w k ( ( ( ) p ( ;ξ 2 ) Dp ; ˆ σ 2 i θ k, a η Segmentation of target models, not of images Ordering of pixels by their empirical information relative to null ll-hypothesis. η = 0.2 η = 1 η = 5 η = 25 η = 125 For null-hypothesis ξ 2 = approximate background variance 14
15 Point #2: Cost of Likelihood Evaluation Let N(a,θ) ) = # nonzero mask elements, (i)( ) = ith nonzero mask pixel [ a ˆ, ˆ θ ]= argmax N(a,θ)lnˆ c 2 [ a,θ ] ˆ c 2 = 1 Na,θ ( ) Na,θ ( ) i=1 2 σ (i) r (i) 2 ( a,θ ) N(a,θ) [ lnσ 2 (i) (a,θ) +1 lnξ 2 ] i =1 + 1 ξ 2 N(a,θ ) i =1 r (i) 2 Expression evaluation consumes (on average): T mem 2N(a,θ) ) +2 memory reads totaling (2N(a,θ) ) +2) mem seconds P CPI T 2N(a,θ) ) +15 operations totaling (2N(a,θ) ) +15) cyc seconds P where: T mem = effective memory read time T cyc = clock period CPI = average cycles per instruction P = # processors 15
16 Point #3: Successively-Refinable Encoding Likelihood is a function of σ 2 i (a,θ), a function of continuous θ. Discrete approximation with N d non-overlapping overlapping intervals of width d: 2 σ d,i 2 σ d,i ( θ k, a)= 1 d 2πk Nd + d 2 2πk N d d 2 σ 2 i ( θ, a)dθ Smaller d yields more complex, piecewise constant representations Approximations d and d/2 are hierarchically related: ( θ k, a)= 1 n 1 +n 2 [ n 1 σ 2 ( d,i θ 2k,a )+ n 2 σ 2 ( d,i θ 2k +1,a)] 2 2 Given parent and 1 child, compute other child costing 4N(a,θ) 4 operations saving b(a,θ)/bw in communication time where: b(a,θ) ) = bits to represent σ 2 (a,θ)) and BW = bandwidth 16
17 Point #3: Successively-Refinable Encoding Consider decreasing interval widths d 1 =2π, d 2 =π,, d m =2π/2 m-1 Approximate variance images for bulldozer (D7) from d 1 through d 5 Successively-refinable models yield successively- refinable decisions Search over θ k in level i ordered by the most likely pose at level i-1 Resource consumption depends on extent of search (complexity) 17
18 Example Classification error rate as a function of the number of bits transmitted between the database and processor Classification depends on extent of search Eventually, search covers all possibilities Breadth-first search quickly finds good combinations of (a,θ)( Small overhead present with ordered searches 18
19 Example Relative entropy to reject target classes not in the database Error occurs if incorrectly classified or falsely rejected One-sample estimate of variance from observation: r i 2 Relative entropy quantifies difference from training estimate Reject observation if the empirical relative entropy is too large 1 S a ˆ i S a ˆ ( ( 2 σ ( i ˆ θ, a ˆ ) Dp ; ( 2 r i )p ; ˆ > γ Rejection behavior also a function of segmentation and search depth 19
20 Point #3: Computational Model Time to process through approximation d m includes time to: distribute SAR image to each processor process each approximation until local memory is full process each remaining approximation l mem T chip = S c BW log 2 ( P +1) + 2 l 1 N T τ dl + 2 l 2 N T τ dl + Where: l=1 m l =l mem +1 S c = bits per SAR image BW = network bandwidth P = number of processors N T = number of target classes ( τ ) dl τ d = average time per template at approximation d exploiting hierarchy. Receive variance, compute variance, and compute likelihoods. τ d = average time per template without exploiting hierarchy. Receive variance and compute likelihood. 20
21 Example With: Chip Rate = 1/T chip 1 GHz clock 64 bit read per clock cycle 10 target classes 0.5 CPI 25 target locations considered High throughput corresponds to coarse approximations Markers denote start of a new level in model representation tree 1 Gbps interconnection 10 Gbps interconnection 21
22 Conclusions Automatic recognition as a two-source communication problem Accuracy depends on model validity» assessed from sample data, aggregating over many samples» function of approximation level Resource consumption also depends on approximation level Successively-refinable decisions from hierarchical approximations» Process until resources exhausted» Generate good candidates quickly Open Issues Include Implication of good fit for quarter-power model Refinability over depression angle and articulation Theoretical minimum error as a function of resources consumed 22
23 Contributions Recognition as a two-source communication problem Detailed assessment of conditional models for SAR imagery Approaches to rejection of confuser objects Probability of error expression for two-class tests Relating accuracy to resource consumption through complexity Successively-refinable models and decisions Experimental results for recognition from SAR imagery 23
ONR MURI: Performance Analysis of Multiple Sensor Systems in Automatic Target Recognition
ONR MURI: Performance Analysis of Multiple Sensor Systems in Automatic Target Recognition Joseph A. O SullivanO Electronic Systems and Signals Research Laboratory Department of Electrical Engineering Washington
More informationH 2 : otherwise. that is simply the proportion of the sample points below level x. For any fixed point x the law of large numbers gives that
Lecture 28 28.1 Kolmogorov-Smirnov test. Suppose that we have an i.i.d. sample X 1,..., X n with some unknown distribution and we would like to test the hypothesis that is equal to a particular distribution
More informationWHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred O. Hero
WHEN IS A MAXIMAL INVARIANT HYPTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred. Hero Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 489-222 ABSTRACT
More informationDetection theory. H 0 : x[n] = w[n]
Detection Theory Detection theory A the last topic of the course, we will briefly consider detection theory. The methods are based on estimation theory and attempt to answer questions such as Is a signal
More informationPATTERN RECOGNITION AND MACHINE LEARNING
PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 3: Detection Theory January 2018 Heikki Huttunen heikki.huttunen@tut.fi Department of Signal Processing Tampere University of Technology Detection theory
More informationPerformance Evaluation and Comparison
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Cross Validation and Resampling 3 Interval Estimation
More informationHypothesis testing (cont d)
Hypothesis testing (cont d) Ulrich Heintz Brown University 4/12/2016 Ulrich Heintz - PHYS 1560 Lecture 11 1 Hypothesis testing Is our hypothesis about the fundamental physics correct? We will not be able
More informationSIGNAL COMPRESSION. 8. Lossy image compression: Principle of embedding
SIGNAL COMPRESSION 8. Lossy image compression: Principle of embedding 8.1 Lossy compression 8.2 Embedded Zerotree Coder 161 8.1 Lossy compression - many degrees of freedom and many viewpoints The fundamental
More informationCFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE
CFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE Anshul Sharma and Randolph L. Moses Department of Electrical Engineering, The Ohio State University, Columbus, OH 43210 ABSTRACT We have developed
More informationBayesian Learning (II)
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP
More informationDr. Maddah ENMG 617 EM Statistics 10/15/12. Nonparametric Statistics (2) (Goodness of fit tests)
Dr. Maddah ENMG 617 EM Statistics 10/15/12 Nonparametric Statistics (2) (Goodness of fit tests) Introduction Probability models used in decision making (Operations Research) and other fields require fitting
More informationDetection theory 101 ELEC-E5410 Signal Processing for Communications
Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off
More informationInformation-Theoretic Imaging
with Applications in Hyperspectral Imaging and Transmission Tomography Joseph A. O SullivanO Electronic Systems and Signals Research Laboratory Department of Electrical Engineering Washington University
More informationParameter Estimation and Fitting to Data
Parameter Estimation and Fitting to Data Parameter estimation Maximum likelihood Least squares Goodness-of-fit Examples Elton S. Smith, Jefferson Lab 1 Parameter estimation Properties of estimators 3 An
More informationSTUDIES OF OCEAN S SCATTERING PROPERTIES BASED ON AIRSAR DATA
STUDIES OF OCEAN S SCATTERING PROPERTIES BASED ON AIRSAR DATA Wang Wenguang *, Sun Jinping, Wang Jun, Hu Rui School of EIE, Beihang University, Beijing 00083, China- wwenguang@ee.buaa.edu.cn KEY WORDS:
More informationProblem Set 2. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 30
Problem Set 2 MAS 622J/1.126J: Pattern Recognition and Analysis Due: 5:00 p.m. on September 30 [Note: All instructions to plot data or write a program should be carried out using Matlab. In order to maintain
More informationRecall the Basics of Hypothesis Testing
Recall the Basics of Hypothesis Testing The level of significance α, (size of test) is defined as the probability of X falling in w (rejecting H 0 ) when H 0 is true: P(X w H 0 ) = α. H 0 TRUE H 1 TRUE
More informationHyung So0 Kim and Alfred 0. Hero
WHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung So0 Kim and Alfred 0. Hero Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 48109-2122
More informationPartitioning the Parameter Space. Topic 18 Composite Hypotheses
Topic 18 Composite Hypotheses Partitioning the Parameter Space 1 / 10 Outline Partitioning the Parameter Space 2 / 10 Partitioning the Parameter Space Simple hypotheses limit us to a decision between one
More informationLeast Squares Classification
Least Squares Classification Stephen Boyd EE103 Stanford University November 4, 2017 Outline Classification Least squares classification Multi-class classifiers Classification 2 Classification data fitting
More informationLecture 12. Block Diagram
Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationIntroduction to Statistical Inference
Structural Health Monitoring Using Statistical Pattern Recognition Introduction to Statistical Inference Presented by Charles R. Farrar, Ph.D., P.E. Outline Introduce statistical decision making for Structural
More informationProblem Set 2. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 30
Problem Set MAS 6J/1.16J: Pattern Recognition and Analysis Due: 5:00 p.m. on September 30 [Note: All instructions to plot data or write a program should be carried out using Matlab. In order to maintain
More informationNaval Air Warfare Center Weapons Division China Lake, CA. Automatic Target Recognition via Sparse Representations
Naval Air Warfare Center Weapons Division China Lake, CA. Automatic Target Recognition via Sparse Representations Outline Research objective / ATR challenges Background on Compressed Sensing (CS) ATR in
More informationBayesian Decision Theory
Bayesian Decision Theory Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent University) 1 / 46 Bayesian
More informationAlgorithm-Independent Learning Issues
Algorithm-Independent Learning Issues Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2007 c 2007, Selim Aksoy Introduction We have seen many learning
More informationVisual Object Detection
Visual Object Detection Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL 60208 yingwu@northwestern.edu http://www.eecs.northwestern.edu/~yingwu 1 / 47 Visual Object
More informationDistribution Fitting (Censored Data)
Distribution Fitting (Censored Data) Summary... 1 Data Input... 2 Analysis Summary... 3 Analysis Options... 4 Goodness-of-Fit Tests... 6 Frequency Histogram... 8 Comparison of Alternative Distributions...
More informationDetection and Estimation Theory
ESE 524 Detection and Estimation Theory Joseph A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University 211 Urbauer
More informationEconomics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1,
Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem
More informationNaïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
More informationClustering with k-means and Gaussian mixture distributions
Clustering with k-means and Gaussian mixture distributions Machine Learning and Category Representation 2012-2013 Jakob Verbeek, ovember 23, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.12.13
More informationChapter 2 Signal Processing at Receivers: Detection Theory
Chapter Signal Processing at Receivers: Detection Theory As an application of the statistical hypothesis testing, signal detection plays a key role in signal processing at receivers of wireless communication
More informationAlgorithmisches Lernen/Machine Learning
Algorithmisches Lernen/Machine Learning Part 1: Stefan Wermter Introduction Connectionist Learning (e.g. Neural Networks) Decision-Trees, Genetic Algorithms Part 2: Norman Hendrich Support-Vector Machines
More informationSYDE 372 Introduction to Pattern Recognition. Probability Measures for Classification: Part I
SYDE 372 Introduction to Pattern Recognition Probability Measures for Classification: Part I Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 Why use probability
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Hypothesis Testing: Suppose we have two or (in general) more simple hypotheses which can describe a set of data Simple means explicitly defined, so if parameters have to be fitted, that has already been
More information- An Image Coding Algorithm
- An Image Coding Algorithm Shufang Wu http://www.sfu.ca/~vswu vswu@cs.sfu.ca Friday, June 14, 2002 22-1 Agenda Overview Discrete Wavelet Transform Zerotree Coding of Wavelet Coefficients Successive-Approximation
More informationSelf-Normalized Linear Tests
Self-Normalized Linear Tests Sachin Gangaputra Dept. of Electrical and Computer Engineering The Johns Hopkins University Baltimore, MD 21218 sachin@jhu.edu Donald Geman Dept. of Applied Mathematics and
More informationMonte Carlo Studies. The response in a Monte Carlo study is a random variable.
Monte Carlo Studies The response in a Monte Carlo study is a random variable. The response in a Monte Carlo study has a variance that comes from the variance of the stochastic elements in the data-generating
More informationSpatial Bayesian Nonparametrics for Natural Image Segmentation
Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes
More informationEE382 Processor Design Winter 1999 Chapter 2 Lectures Clocking and Pipelining
Slide 1 EE382 Processor Design Winter 1999 Chapter 2 Lectures Clocking and Pipelining Slide 2 Topics Clocking Clock Parameters Latch Types Requirements for reliable clocking Pipelining Optimal pipelining
More informationQuantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing
Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October
More informationLinear discriminant functions
Andrea Passerini passerini@disi.unitn.it Machine Learning Discriminative learning Discriminative vs generative Generative learning assumes knowledge of the distribution governing the data Discriminative
More informationUncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department
Decision-Making under Statistical Uncertainty Jayakrishnan Unnikrishnan PhD Defense ECE Department University of Illinois at Urbana-Champaign CSL 141 12 June 2010 Statistical Decision-Making Relevant in
More informationFalse Discovery Rate Based Distributed Detection in the Presence of Byzantines
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS () 1 False Discovery Rate Based Distributed Detection in the Presence of Byzantines Aditya Vempaty*, Student Member, IEEE, Priyadip Ray, Member, IEEE,
More informationPose Estimation in SAR using an Information Theoretic Criterion
Pose Estimation in SAR using an Information Theoretic Criterion Jose C. Principe, Dongxin Xu, John W. Fisher III Computational NeuroEngineering Laboratory, U. of Florida. {principe,xu,fisher}@cnel.ufl.edu
More informationStatistical learning. Chapter 20, Sections 1 3 1
Statistical learning Chapter 20, Sections 1 3 Chapter 20, Sections 1 3 1 Outline Bayesian learning Maximum a posteriori and maximum likelihood learning Bayes net learning ML parameter learning with complete
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Hypothesis testing. Anna Wegloop Niels Landwehr/Tobias Scheffer
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Hypothesis testing Anna Wegloop iels Landwehr/Tobias Scheffer Why do a statistical test? input computer model output Outlook ull-hypothesis
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Decision Trees. Tobias Scheffer
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Decision Trees Tobias Scheffer Decision Trees One of many applications: credit risk Employed longer than 3 months Positive credit
More informationSTATS 306B: Unsupervised Learning Spring Lecture 5 April 14
STATS 306B: Unsupervised Learning Spring 2014 Lecture 5 April 14 Lecturer: Lester Mackey Scribe: Brian Do and Robin Jia 5.1 Discrete Hidden Markov Models 5.1.1 Recap In the last lecture, we introduced
More informationUNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS
UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS F. C. Nicolls and G. de Jager Department of Electrical Engineering, University of Cape Town Rondebosch 77, South
More informationNegative binomial distribution and multiplicities in p p( p) collisions
Negative binomial distribution and multiplicities in p p( p) collisions Institute of Theoretical Physics University of Wroc law Zakopane June 12, 2011 Summary s are performed for the hypothesis that charged-particle
More informationNaïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability
Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish
More informationDecentralized Sequential Hypothesis Testing. Change Detection
Decentralized Sequential Hypothesis Testing & Change Detection Giorgos Fellouris, Columbia University, NY, USA George V. Moustakides, University of Patras, Greece Outline Sequential hypothesis testing
More informationLearning with Probabilities
Learning with Probabilities CS194-10 Fall 2011 Lecture 15 CS194-10 Fall 2011 Lecture 15 1 Outline Bayesian learning eliminates arbitrary loss functions and regularizers facilitates incorporation of prior
More informationKernel Methods. Lecture 4: Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton
Kernel Methods Lecture 4: Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton Alexander J. Smola Statistical Machine Learning Program Canberra,
More informationSequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process
Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University
More informationSUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION
SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION 1 Outline Basic terminology Features Training and validation Model selection Error and loss measures Statistical comparison Evaluation measures 2 Terminology
More informationPattern Recognition. Parameter Estimation of Probability Density Functions
Pattern Recognition Parameter Estimation of Probability Density Functions Classification Problem (Review) The classification problem is to assign an arbitrary feature vector x F to one of c classes. The
More informationBayesian Classifiers and Probability Estimation. Vassilis Athitsos CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington
Bayesian Classifiers and Probability Estimation Vassilis Athitsos CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington 1 Data Space Suppose that we have a classification problem The
More informationMATCHED SUBSPACE DETECTORS FOR DISCRIMINATION OF TARGETS FROM TREES IN SAR IMAGERY
MATCHED SUBSPACE DETECTORS FOR DISCRIMINATION OF TARGETS FROM TREES IN SAR IMAGERY A. Shanna and R. L. Moses Department of Electrical Engineering The Ohio State University Columbus, OH 43210 USA ABSTRACT
More informationCourse 495: Advanced Statistical Machine Learning/Pattern Recognition
Course 495: Advanced Statistical Machine Learning/Pattern Recognition Goal (Lecture): To present Probabilistic Principal Component Analysis (PPCA) using both Maximum Likelihood (ML) and Expectation Maximization
More informationPattern recognition. "To understand is to perceive patterns" Sir Isaiah Berlin, Russian philosopher
Pattern recognition "To understand is to perceive patterns" Sir Isaiah Berlin, Russian philosopher The more relevant patterns at your disposal, the better your decisions will be. This is hopeful news to
More informationSupplementary Material for:
Supplementary Material for: Correction to Kyllingsbæk, Markussen, and Bundesen (2012) Unfortunately, the computational shortcut used in Kyllingsbæk et al. (2012) (henceforth; the article) to fit the Poisson
More informationEEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1
EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationUniversal Anytime Codes: An approach to uncertain channels in control
Universal Anytime Codes: An approach to uncertain channels in control paper by Stark Draper and Anant Sahai presented by Sekhar Tatikonda Wireless Foundations Department of Electrical Engineering and Computer
More informationDETECTION theory deals primarily with techniques for
ADVANCED SIGNAL PROCESSING SE Optimum Detection of Deterministic and Random Signals Stefan Tertinek Graz University of Technology turtle@sbox.tugraz.at Abstract This paper introduces various methods for
More informationGenerative v. Discriminative classifiers Intuition
Logistic Regression Machine Learning 070/578 Carlos Guestrin Carnegie Mellon University September 24 th, 2007 Generative v. Discriminative classifiers Intuition Want to Learn: h:x a Y X features Y target
More informationIntro. ANN & Fuzzy Systems. Lecture 15. Pattern Classification (I): Statistical Formulation
Lecture 15. Pattern Classification (I): Statistical Formulation Outline Statistical Pattern Recognition Maximum Posterior Probability (MAP) Classifier Maximum Likelihood (ML) Classifier K-Nearest Neighbor
More informationMaximum Mean Discrepancy
Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia
More informationLecture 3: Pattern Classification
EE E6820: Speech & Audio Processing & Recognition Lecture 3: Pattern Classification 1 2 3 4 5 The problem of classification Linear and nonlinear classifiers Probabilistic classification Gaussians, mixtures
More informationExpectation Maximization
Expectation Maximization Bishop PRML Ch. 9 Alireza Ghane c Ghane/Mori 4 6 8 4 6 8 4 6 8 4 6 8 5 5 5 5 5 5 4 6 8 4 4 6 8 4 5 5 5 5 5 5 µ, Σ) α f Learningscale is slightly Parameters is slightly larger larger
More informationRandom Number Generation. CS1538: Introduction to simulations
Random Number Generation CS1538: Introduction to simulations Random Numbers Stochastic simulations require random data True random data cannot come from an algorithm We must obtain it from some process
More informationBiometrics: Introduction and Examples. Raymond Veldhuis
Biometrics: Introduction and Examples Raymond Veldhuis 1 Overview Biometric recognition Face recognition Challenges Transparent face recognition Large-scale identification Watch list Anonymous biometrics
More informationEECS490: Digital Image Processing. Lecture #26
Lecture #26 Moments; invariant moments Eigenvector, principal component analysis Boundary coding Image primitives Image representation: trees, graphs Object recognition and classes Minimum distance classifiers
More informationPsychology 282 Lecture #4 Outline Inferences in SLR
Psychology 282 Lecture #4 Outline Inferences in SLR Assumptions To this point we have not had to make any distributional assumptions. Principle of least squares requires no assumptions. Can use correlations
More informationClustering with k-means and Gaussian mixture distributions
Clustering with k-means and Gaussian mixture distributions Machine Learning and Object Recognition 2017-2018 Jakob Verbeek Clustering Finding a group structure in the data Data in one cluster similar to
More informationEEC 216 Lecture #3: Power Estimation, Interconnect, & Architecture. Rajeevan Amirtharajah University of California, Davis
EEC 216 Lecture #3: Power Estimation, Interconnect, & Architecture Rajeevan Amirtharajah University of California, Davis Outline Announcements Review: PDP, EDP, Intersignal Correlations, Glitching, Top
More informationFinancial Econometrics and Quantitative Risk Managenent Return Properties
Financial Econometrics and Quantitative Risk Managenent Return Properties Eric Zivot Updated: April 1, 2013 Lecture Outline Course introduction Return definitions Empirical properties of returns Reading
More informationPolitical Science 236 Hypothesis Testing: Review and Bootstrapping
Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall 2007 1 Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The
More informationSelf-Normalized Linear Tests
Self-Normalized Linear Tests Sachin Gangaputra Dept. of Electrical and Computer Engineering The Johns Hopkins University Baltimore, MD 21218 sachin@jhu.edu Donald Geman Dept. of Applied Mathematics and
More informationIntroduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf
1 Introduction to Machine Learning Maximum Likelihood and Bayesian Inference Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf 2013-14 We know that X ~ B(n,p), but we do not know p. We get a random sample
More informationDistributed Binary Quantizers for Communication Constrained Large-scale Sensor Networks
Distributed Binary Quantizers for Communication Constrained Large-scale Sensor Networks Ying Lin and Biao Chen Dept. of EECS Syracuse University Syracuse, NY 13244, U.S.A. ylin20 {bichen}@ecs.syr.edu Peter
More informationAsymptotic Statistics-VI. Changliang Zou
Asymptotic Statistics-VI Changliang Zou Kolmogorov-Smirnov distance Example (Kolmogorov-Smirnov confidence intervals) We know given α (0, 1), there is a well-defined d = d α,n such that, for any continuous
More informationFace detection and recognition. Detection Recognition Sally
Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification
More informationBlind phase/frequency synchronization with low-precision ADC: a Bayesian approach
Blind phase/frequency synchronization with low-precision ADC: a Bayesian approach Aseem Wadhwa, Upamanyu Madhow Department of ECE, UCSB 1/26 Modern Communication Receiver Architecture Analog Digital TX
More informationBayes Decision Theory
Bayes Decision Theory Minimum-Error-Rate Classification Classifiers, Discriminant Functions and Decision Surfaces The Normal Density 0 Minimum-Error-Rate Classification Actions are decisions on classes
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Lesson 7 Delta Modulation and DPCM Instructional Objectives At the end of this lesson, the students should be able to: 1. Describe a lossy predictive coding scheme.
More informationTesting for Poisson Behavior
Testing for Poisson Behavior Philip B. Stark Department of Statistics, UC Berkeley joint with Brad Luen 17 April 2012 Seismological Society of America Annual Meeting San Diego, CA Quake Physics versus
More informationSYSM 6303: Quantitative Introduction to Risk and Uncertainty in Business Lecture 4: Fitting Data to Distributions
SYSM 6303: Quantitative Introduction to Risk and Uncertainty in Business Lecture 4: Fitting Data to Distributions M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu
More information1 Degree distributions and data
1 Degree distributions and data A great deal of effort is often spent trying to identify what functional form best describes the degree distribution of a network, particularly the upper tail of that distribution.
More informationGenerative classifiers: The Gaussian classifier. Ata Kaban School of Computer Science University of Birmingham
Generative classifiers: The Gaussian classifier Ata Kaban School of Computer Science University of Birmingham Outline We have already seen how Bayes rule can be turned into a classifier In all our examples
More informationNaive Bayes classification
Naive Bayes classification Christos Dimitrakakis December 4, 2015 1 Introduction One of the most important methods in machine learning and statistics is that of Bayesian inference. This is the most fundamental
More informationDoes Unlabeled Data Help?
Does Unlabeled Data Help? Worst-case Analysis of the Sample Complexity of Semi-supervised Learning. Ben-David, Lu and Pal; COLT, 2008. Presentation by Ashish Rastogi Courant Machine Learning Seminar. Outline
More informationSTAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015
STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis
More informationHypothesis Testing Chap 10p460
Hypothesis Testing Chap 1p46 Elements of a statistical test p462 - Null hypothesis - Alternative hypothesis - Test Statistic - Rejection region Rejection Region p462 The rejection region (RR) specifies
More informationQualifying Exam in Machine Learning
Qualifying Exam in Machine Learning October 20, 2009 Instructions: Answer two out of the three questions in Part 1. In addition, answer two out of three questions in two additional parts (choose two parts
More informationDefinition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution.
Hypothesis Testing Definition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution. Suppose the family of population distributions is indexed
More information