REVEAL. Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008

Size: px
Start display at page:

Download "REVEAL. Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008"

Transcription

1 REVEAL Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008 Presented to Program Officers: Drs. John Tague and Keith Davidson Undersea Signal Processing Team, Office of Naval Research Penn State Team: Faculty: R. Lee Culver 1 and Nirmal K. Bose 2 Students: Colin W. Jemmott 3, Jeremy Joseph 3, Brett Bissinger 2, and Alex Sell 3 1 Applied Research Lab, 2 Department of Electrical Engineering, and 3 Penn State University, State College, PA Contact info: rlc5@psu.edu 16 September 2008 REVEAL Overview and Progress 1

2 REVEAL Long Range Goals Develop a signal processing structure that exploits environmental knowledge by incorporating signal and noise predictions. Use this SP structure to develop improved detectors and classifiers which remain robust to variable and random signal and noise. No specific system application, but focus on passive sonar and frequency 1 khz. Train the future generation of ocean acousticians and signal processors. 16 September 2008 REVEAL Overview and Progress 2

3 REVEAL Project focus Since FY05, the project goal has been to work at the interface between OA and SP in order to apply and transition OA products to SP algorithms. Ocean acoustics REVEAL focus Sonar signal processing transition 16 September 2008 REVEAL Overview and Progress 3

4 REVEAL approach Ocean acoustic models and knowledge Compute signal and noise parameter statistics (also called prior statistics or training data) Passive beamformed sonar data M-ary detector or classifier decision Estimated Ocean Detector (composite Likelihood Ratio) Kullback-Leibler divergence (et. al.) Bayesian (histogram) filter 16 September 2008 REVEAL Overview and Progress 4

5 Classification using discriminant functions Signal parameters Discriminant functions 16 September 2008 REVEAL Overview and Progress 5

6 Typical problem The signal is affected by propagation through the ocean, and we have knowledge and models for the oceanic properties and processes that affect acoustic propagation. Our approach is to use Monte Carlo simulation to obtain many realizations of the signal from statistically-valid realizations of the environment in order to classify the signal source. p 1 (s(θ)) pdf of signal from near-surface source (H 1 ) Source near the surface Receive array p 2 (s(θ)) pdf of signal from near-bottom source (H 2 ) Source near the bottom 16 September 2008 REVEAL Overview and Progress 6

7 Composite LR Consider observation r = s( θ ) + n. A composite Likelihood Ratio (LR) incorporates statistical knowledge of random parameter θ : ( r) ( H1) ( ) p r Λ = = [ H, ] ( H ) pr θ pθ dθ 1 1 H 2 H 2, H 2 [ ] ( ) p r pr θ pθ dθ Since the noise is additive, the likelihood function is the pdf of the noise: ( ) [ θ] = ( θ) θ = ( θ) pr H, pr s H, p r s H If the noise is Gaussian, the likelihood function is then : 1 1 n 1 1 pn ( r s( θ) H, θ) = exp r s ( θ ) σ 2σ September 2008 REVEAL Overview and Progress 7 2

8 Composite LR (cont) and the Likelihood Ratio (LR) is : 2 ( r s1 ) exp 2 p( θ H1 ) dθ 2σ 1 Λ ( r) = 2 ( r s2 ) exp p 2 ( θ H2 ) dθ 2σ 2 The Estimator-Correlator (EC) provides an expression for the LR in the more general case where the noise pdf belongs to the exponential class. Jeff Ballard formulated the EC for Gaussian signals in FY07 and sinusoids in FY Schwartz, S. C., The Estimator-Correlator for Discrete-Time Problems, IEEE Trans. on Inf. Theory, Vol. 23, No. 1, Jan 1977, pp Ballard and Culver, The Estimated Signal Parameter Detector: Incorporating signal parameter statistics in the signal processor, submitted to JOE (2008). 16 September 2008 REVEAL Overview and Progress 8

9 Signal Parameter pdf ( 1 ( θ) ) p pθ H 1 Signal Parameter pdf Estimated Ocean Detector (EOD) p Conditional Moment Function Received Signal, r ( r θ ) 1( ) h r h 2( r ) G G B B 2 Noise pdf ( r ) ( r ) ( r ) ( r ) ln c 1 c 2 - H Σ ( ) > 1 r - Noise only data < ln η H 2 H 1 H 2 ( p ( θ )) p θ H September 2008 REVEAL Overview and Progress 9

10 Neglecting noise When the noise is neglibly small, the likelihood function becomes ( r) ( H, θ) = ( θ) H, θ δ ( θ) p r 1 p r s 1 r s and the Likelihood Ratio is then p r s( θ) H 1, θ p s( θ) H 1 dθ p s( θ ) H 1 p r s( θ) H 2, θ p s( θ) H 2 dθ p s ( θ ) H2 Λ = We have made this assumption in applying the composite LR to the 1996 Strait of Gibraltar and Swellex-96 data, respectively. 1. Culver, R. L. and H. J. Camin, Dependence of probabilistic acoustic signal models on statistical ocean environmental models, submitted to JASA (2008). 2. Jemmott, C.W., R. L. Culver, and N. K. Bose, Passive sonar depth classification using model based amplitude statistics, (in preparation). 16 September 2008 REVEAL Overview and Progress 10

11 The Bayes Filter The Bayes filter is an alternative to the LR in which we use Bayes rule ( H, θ) ( H θ) = (,H θ) = ( H, θ) ( θ) p r p p r p r p r i i i i to convert the likelihood function to the posterior pdf ( θ,h i) p( θ Hi) p( r θ ) p r = ( r θ ) H,. We select the hypothesis with the highest posterior probability. The histogram filter is the discrete implementation of the Bayesian filter. Colin will compare a recursive histogram filter to the LR receiver. p i 16 September 2008 REVEAL Overview and Progress 11

12 Distance measures The Kullback-Leibler divergence (among others) provides a measure of the distance between two multidimensional surfaces, e.g. pdfs. Using a distance measure to classify signals: Predict signal parameter pdfs for difference classes Estimate signal parameters from observations; compute signal parameter pdfs from observations Pick the class whose pdf is closest to the observed signal parameter pdf Brett will present his work on this approach 16 September 2008 REVEAL Overview and Progress 12

13 Noise whitening The EOD requires that the noise pdf belong to the exponential class. Not necessarily Gaussian. How to whiten or decorrelate? So-called higher order whitening has been investigated in the image processing literature. Whitening is closely related to distance measures and to compressive sampling. Dr. Bose will present his work. 1. J. Gluckman, Higher order whitening of natural images, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, N.K. Bose, U. Srinivas and R.L. Culver (2008). Wavelength diversity based infrared super-resolution and condition-based maintenance, INSIGHT, British Institute for Non-Destructive Testing, Vol. 50, No September 2008 REVEAL Overview and Progress 13

14 Predicting signal parameter pdfs Rough surface PE acoustic propagation model obtained from Rosenberg (APL/JHU); based upon Range-dependent Acoustic Model (RAM), but adds capability for acoustic propagation with time- and spatially-varying rough surface. We want to determine if this simulation can predict surface interaction effects on signal frequency and amplitude pdfs. Jeremy Joseph will present his work. 16 September 2008 REVEAL Overview and Progress 14

15 REVEAL FY08 Talks Acoust. Soc. Am. Nov 07 (New Orleans) Culver: Likelihood func. & signal param. pdfs for sonar signal processing Joseph: Effect of rough surface on received signals Apr 08 PSU College of Engr. Research Symposium (CERS) Jemmott: Passive sonar source classification based on received signal amplitude variation statistics Acoust. Soc. Am. June - July 08 (Paris) Bissinger: Application of statistical methods in uw signal classification Joseph: Effects of volume and boundary variability on the statistics of received signal frequency Culver: Detection and classification using the Estimated Ocean Detector Distributed Detection and Est. Workshop (VA Tech, July 08) Bissinger: Statistical Distance Based Signal Classification Culver: Detection and classification using the Estimated Ocean Detector Jemmott: Passive Sonar Model-Based Source Location Classification Joseph: Effect of rough surface on received signals 16 September 2008 REVEAL Overview and Progress 15

16 REVEAL FY08 Papers N.K. Bose, U. Srinivas and R.L. Culver (2008). Wavelength diversity based infrared super-resolution and condition-based maintenance, INSIGHT, British Institute for Non-Destructive Testing, Vol. 50, No. 8. J.A. Ballard and R. L. Culver (2008). The Estimated Signal Parameter Detector: Incorporating signal parameter statistics into the signal processor submitted to IEEE J. Oceanographic Engr. Dec 07 Comments received June 08 Manuscript revised and resubmitted Sept 08 R.L. Culver and H.J. Camin (Dependence of probabilistic acoustic signal models on statistical ocean environmental models submitted to J. Acoust. Soc. Am. Nov 07 Comments received May 08 Manuscript under revision; will re-submit by 30 Sep September 2008 REVEAL Overview and Progress 16

17 Planned FY09 Talks and Papers Present at Asilomar (Oct 08), CISS (Mar 09), and UASP (Oct 09) signal processing meetings. Present as ASA (Nov 09) and (May 09) Publish papers on: depth classification using the Swellex-96 data higher order whitening application of distance measure to uw acoustics predicting uw acoustic signal parameter statistics 16 September 2008 REVEAL Overview and Progress 17

18 REVEAL Project FY09 Plans Apply classifiers to South Florida range data Bottom-mounted line arrays Surface ships and a towed, submerged source Move from high SNR to moderate SNR cases Incorporate noise whitening for EOD Robustness of distance measures and histogram filter Address correlation in extracted parameter values Incorporate the rough surface RAM simulation into the signal processing architecture 16 September 2008 REVEAL Overview and Progress 18

19 REVEAL Project Ocean environment ocean models In-situ measurements environ. realizations acoustic propagation model Monte Carlo simulation signal realizations sonar data noise pdf Classifier Detection/classification decision Signal parameter pdf s (MaxEnt method) 16 September 2008 REVEAL Overview and Progress 19

20 REVEAL Project Ocean environment ocean models In-situ measurements environ. realizations acoustic propagation model Monte Carlo simulation signal realizations sonar data noise pdf Colin and Brett Classifier Detection/classification decision Signal parameter pdf s (MaxEnt method) 16 September 2008 REVEAL Overview and Progress 20

21 REVEAL Project Ocean environment ocean models In-situ measurements environ. realizations acoustic propagation model Monte Carlo simulation signal realizations sonar data noise pdf Dr. Bose Classifier Detection/classification decision Signal parameter pdf s (MaxEnt method) 16 September 2008 REVEAL Overview and Progress 21

22 REVEAL Project Ocean environment ocean models In-situ measurements environ. realizations Jeremy acoustic propagation model Monte Carlo simulation signal realizations sonar data noise pdf Classifier Detection/classification decision Signal parameter pdf s (MaxEnt method) 16 September 2008 REVEAL Overview and Progress 22

Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012

Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012 Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood

More information

An Invariance Property of the Generalized Likelihood Ratio Test

An Invariance Property of the Generalized Likelihood Ratio Test 352 IEEE SIGNAL PROCESSING LETTERS, VOL. 10, NO. 12, DECEMBER 2003 An Invariance Property of the Generalized Likelihood Ratio Test Steven M. Kay, Fellow, IEEE, and Joseph R. Gabriel, Member, IEEE Abstract

More information

SYDE 372 Introduction to Pattern Recognition. Probability Measures for Classification: Part I

SYDE 372 Introduction to Pattern Recognition. Probability Measures for Classification: Part I SYDE 372 Introduction to Pattern Recognition Probability Measures for Classification: Part I Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 Why use probability

More information

Classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Topics Discriminant functions Logistic regression Perceptron Generative models Generative vs. discriminative

More information

Expectation Propagation for Approximate Bayesian Inference

Expectation Propagation for Approximate Bayesian Inference Expectation Propagation for Approximate Bayesian Inference José Miguel Hernández Lobato Universidad Autónoma de Madrid, Computer Science Department February 5, 2007 1/ 24 Bayesian Inference Inference Given

More information

Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

More information

Part 1: Expectation Propagation

Part 1: Expectation Propagation Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud

More information

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Detection Theory Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Outline Neyman-Pearson Theorem Detector Performance Irrelevant Data Minimum Probability of Error Bayes Risk Multiple

More information

Fundamentals of Statistical Signal Processing Volume II Detection Theory

Fundamentals of Statistical Signal Processing Volume II Detection Theory Fundamentals of Statistical Signal Processing Volume II Detection Theory Steven M. Kay University of Rhode Island PH PTR Prentice Hall PTR Upper Saddle River, New Jersey 07458 http://www.phptr.com Contents

More information

GAMINGRE 8/1/ of 7

GAMINGRE 8/1/ of 7 FYE 09/30/92 JULY 92 0.00 254,550.00 0.00 0 0 0 0 0 0 0 0 0 254,550.00 0.00 0.00 0.00 0.00 254,550.00 AUG 10,616,710.31 5,299.95 845,656.83 84,565.68 61,084.86 23,480.82 339,734.73 135,893.89 67,946.95

More information

Yellow Sea Thermohaline and Acoustic Variability

Yellow Sea Thermohaline and Acoustic Variability Yellow Sea Thermohaline and Acoustic Variability Peter C Chu, Carlos J. Cintron Naval Postgraduate School, USA Steve Haeger Naval Oceanographic Office, USA Yellow Sea Bottom Sediment Chart Four Bottom

More information

Naïve Bayes classification

Naïve Bayes classification Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss

More information

Bayesian Regression Linear and Logistic Regression

Bayesian Regression Linear and Logistic Regression When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we

More information

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,

More information

PATTERN RECOGNITION AND MACHINE LEARNING

PATTERN RECOGNITION AND MACHINE LEARNING PATTERN RECOGNITION AND MACHINE LEARNING Chapter 1. Introduction Shuai Huang April 21, 2014 Outline 1 What is Machine Learning? 2 Curve Fitting 3 Probability Theory 4 Model Selection 5 The curse of dimensionality

More information

Optimum Passive Beamforming in Relation to Active-Passive Data Fusion

Optimum Passive Beamforming in Relation to Active-Passive Data Fusion Optimum Passive Beamforming in Relation to Active-Passive Data Fusion Bryan A. Yocom Applied Research Laboratories The University of Texas at Austin Final Project EE381K-14 Multidimensional Digital Signal

More information

What does Bayes theorem give us? Lets revisit the ball in the box example.

What does Bayes theorem give us? Lets revisit the ball in the box example. ECE 6430 Pattern Recognition and Analysis Fall 2011 Lecture Notes - 2 What does Bayes theorem give us? Lets revisit the ball in the box example. Figure 1: Boxes with colored balls Last class we answered

More information

OPTIMAL ADAPTIVE TRANSMIT BEAMFORMING FOR COGNITIVE MIMO SONAR IN A SHALLOW WATER WAVEGUIDE

OPTIMAL ADAPTIVE TRANSMIT BEAMFORMING FOR COGNITIVE MIMO SONAR IN A SHALLOW WATER WAVEGUIDE OPTIMAL ADAPTIVE TRANSMIT BEAMFORMING FOR COGNITIVE MIMO SONAR IN A SHALLOW WATER WAVEGUIDE Nathan Sharaga School of EE Tel-Aviv University Tel-Aviv, Israel natyshr@gmail.com Joseph Tabrikian Dept. of

More information

2D Image Processing (Extended) Kalman and particle filter

2D Image Processing (Extended) Kalman and particle filter 2D Image Processing (Extended) Kalman and particle filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz

More information

Bayesian Learning (II)

Bayesian Learning (II) Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP

More information

Acoustic Scattering from a Poro-Elastic Sediment

Acoustic Scattering from a Poro-Elastic Sediment Acoustic Scattering from a Poro-Elastic Sediment Marcia J. Isakson 1, Nicholas P. Chotiros 1 1 Applied Research Laboratories, The University of Texas, 10000 Burnet Rd., Austin, TX 78713 {misakson,chotiros}@arlut.utexas.edu

More information

STONY BROOK UNIVERSITY. CEAS Technical Report 829

STONY BROOK UNIVERSITY. CEAS Technical Report 829 1 STONY BROOK UNIVERSITY CEAS Technical Report 829 Variable and Multiple Target Tracking by Particle Filtering and Maximum Likelihood Monte Carlo Method Jaechan Lim January 4, 2006 2 Abstract In most applications

More information

Sensitivity Considerations in Compressed Sensing

Sensitivity Considerations in Compressed Sensing Sensitivity Considerations in Compressed Sensing Louis L. Scharf, 1 Edwin K. P. Chong, 1,2 Ali Pezeshki, 2 and J. Rockey Luo 2 1 Department of Mathematics, Colorado State University Fort Collins, CO 8523,

More information

Robust Range-rate Estimation of Passive Narrowband Sources in Shallow Water

Robust Range-rate Estimation of Passive Narrowband Sources in Shallow Water Robust Range-rate Estimation of Passive Narrowband Sources in Shallow Water p. 1/23 Robust Range-rate Estimation of Passive Narrowband Sources in Shallow Water Hailiang Tao and Jeffrey Krolik Department

More information

Learning features by contrasting natural images with noise

Learning features by contrasting natural images with noise Learning features by contrasting natural images with noise Michael Gutmann 1 and Aapo Hyvärinen 12 1 Dept. of Computer Science and HIIT, University of Helsinki, P.O. Box 68, FIN-00014 University of Helsinki,

More information

Naïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability

Naïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish

More information

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1 EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle

More information

A Variance Modeling Framework Based on Variational Autoencoders for Speech Enhancement

A Variance Modeling Framework Based on Variational Autoencoders for Speech Enhancement A Variance Modeling Framework Based on Variational Autoencoders for Speech Enhancement Simon Leglaive 1 Laurent Girin 1,2 Radu Horaud 1 1: Inria Grenoble Rhône-Alpes 2: Univ. Grenoble Alpes, Grenoble INP,

More information

Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department

Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department Decision-Making under Statistical Uncertainty Jayakrishnan Unnikrishnan PhD Defense ECE Department University of Illinois at Urbana-Champaign CSL 141 12 June 2010 Statistical Decision-Making Relevant in

More information

BAYESIAN PROCESSOR OF ENSEMBLE (BPE): PRIOR DISTRIBUTION FUNCTION

BAYESIAN PROCESSOR OF ENSEMBLE (BPE): PRIOR DISTRIBUTION FUNCTION BAYESIAN PROCESSOR OF ENSEMBLE (BPE): PRIOR DISTRIBUTION FUNCTION Parametric Models and Estimation Procedures Tested on Temperature Data By Roman Krzysztofowicz and Nah Youn Lee University of Virginia

More information

A Modified Baum Welch Algorithm for Hidden Markov Models with Multiple Observation Spaces

A Modified Baum Welch Algorithm for Hidden Markov Models with Multiple Observation Spaces IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, VOL. 9, NO. 4, MAY 2001 411 A Modified Baum Welch Algorithm for Hidden Markov Models with Multiple Observation Spaces Paul M. Baggenstoss, Member, IEEE

More information

Introduction to Bayesian Inference

Introduction to Bayesian Inference University of Pennsylvania EABCN Training School May 10, 2016 Bayesian Inference Ingredients of Bayesian Analysis: Likelihood function p(y φ) Prior density p(φ) Marginal data density p(y ) = p(y φ)p(φ)dφ

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

More information

Bayesian Decision Theory

Bayesian Decision Theory Bayesian Decision Theory Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Bayesian Decision Theory Bayesian classification for normal distributions Error Probabilities

More information

Bayesian Decision and Bayesian Learning

Bayesian Decision and Bayesian Learning Bayesian Decision and Bayesian Learning Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208 http://www.eecs.northwestern.edu/~yingwu 1 / 30 Bayes Rule p(x ω i

More information

Probability Map Building of Uncertain Dynamic Environments with Indistinguishable Obstacles

Probability Map Building of Uncertain Dynamic Environments with Indistinguishable Obstacles Probability Map Building of Uncertain Dynamic Environments with Indistinguishable Obstacles Myungsoo Jun and Raffaello D Andrea Sibley School of Mechanical and Aerospace Engineering Cornell University

More information

Marine and Physical Acoustics Division Overview

Marine and Physical Acoustics Division Overview Marine and Physical Acoustics Division Overview David L. Bradley 30 April 2013 Center for Acoustics and Vibration ORGANIZATION AND STAFFING Acoustics Division Ocean Acoustics Department Marine Bioacoustics

More information

Variational inference

Variational inference Simon Leglaive Télécom ParisTech, CNRS LTCI, Université Paris Saclay November 18, 2016, Télécom ParisTech, Paris, France. Outline Introduction Probabilistic model Problem Log-likelihood decomposition EM

More information

Parametric Techniques Lecture 3

Parametric Techniques Lecture 3 Parametric Techniques Lecture 3 Jason Corso SUNY at Buffalo 22 January 2009 J. Corso (SUNY at Buffalo) Parametric Techniques Lecture 3 22 January 2009 1 / 39 Introduction In Lecture 2, we learned how to

More information

COURSE INTRODUCTION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception

COURSE INTRODUCTION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception COURSE INTRODUCTION COMPUTATIONAL MODELING OF VISUAL PERCEPTION 2 The goal of this course is to provide a framework and computational tools for modeling visual inference, motivated by interesting examples

More information

Modifying Voice Activity Detection in Low SNR by correction factors

Modifying Voice Activity Detection in Low SNR by correction factors Modifying Voice Activity Detection in Low SNR by correction factors H. Farsi, M. A. Mozaffarian, H.Rahmani Department of Electrical Engineering University of Birjand P.O. Box: +98-9775-376 IRAN hfarsi@birjand.ac.ir

More information

Probabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016

Probabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016 Probabilistic classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Probabilistic approach Bayes decision theory Generative models Gaussian Bayes classifier

More information

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING Kaitlyn Beaudet and Douglas Cochran School of Electrical, Computer and Energy Engineering Arizona State University, Tempe AZ 85287-576 USA ABSTRACT The problem

More information

Parametric Techniques

Parametric Techniques Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure

More information

Efficient Likelihood-Free Inference

Efficient Likelihood-Free Inference Efficient Likelihood-Free Inference Michael Gutmann http://homepages.inf.ed.ac.uk/mgutmann Institute for Adaptive and Neural Computation School of Informatics, University of Edinburgh 8th November 2017

More information

A Bound on Mean-Square Estimation Error Accounting for System Model Mismatch

A Bound on Mean-Square Estimation Error Accounting for System Model Mismatch A Bound on Mean-Square Estimation Error Accounting for System Model Mismatch Wen Xu RD Instruments phone: 858-689-8682 email: wxu@rdinstruments.com Christ D. Richmond MIT Lincoln Laboratory email: christ@ll.mit.edu

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

Learning with Noisy Labels. Kate Niehaus Reading group 11-Feb-2014

Learning with Noisy Labels. Kate Niehaus Reading group 11-Feb-2014 Learning with Noisy Labels Kate Niehaus Reading group 11-Feb-2014 Outline Motivations Generative model approach: Lawrence, N. & Scho lkopf, B. Estimating a Kernel Fisher Discriminant in the Presence of

More information

Sensor Tasking and Control

Sensor Tasking and Control Sensor Tasking and Control Sensing Networking Leonidas Guibas Stanford University Computation CS428 Sensor systems are about sensing, after all... System State Continuous and Discrete Variables The quantities

More information

ESTIMATING INDEPENDENT-SCATTERER DENSITY FROM ACTIVE SONAR REVERBERATION

ESTIMATING INDEPENDENT-SCATTERER DENSITY FROM ACTIVE SONAR REVERBERATION Proceedings of the Eigth European Conference on Underwater Acoustics, 8th ECUA Edited by S. M. Jesus and O. C. Rodríguez Carvoeiro, Portugal 12-15 June, 2006 ESTIMATING INDEPENDENT-SCATTERER DENSITY FROM

More information

UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS

UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS F. C. Nicolls and G. de Jager Department of Electrical Engineering, University of Cape Town Rondebosch 77, South

More information

Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment

Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment Acoustical Society of America Meeting Fall 2005 Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment Vivek Varadarajan and Jeffrey Krolik Duke University Department

More information

Waveguide invariant analysis for modeling time frequency striations in a range dependent environment.

Waveguide invariant analysis for modeling time frequency striations in a range dependent environment. Waveguide invariant analysis for modeling time frequency striations in a range dependent environment. Alexander Sell Graduate Program in Acoustics Penn State University Work supported by ONR Undersea Signal

More information

BLIND SEPARATION OF INSTANTANEOUS MIXTURES OF NON STATIONARY SOURCES

BLIND SEPARATION OF INSTANTANEOUS MIXTURES OF NON STATIONARY SOURCES BLIND SEPARATION OF INSTANTANEOUS MIXTURES OF NON STATIONARY SOURCES Dinh-Tuan Pham Laboratoire de Modélisation et Calcul URA 397, CNRS/UJF/INPG BP 53X, 38041 Grenoble cédex, France Dinh-Tuan.Pham@imag.fr

More information

Engineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers

Engineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers Engineering Part IIB: Module 4F0 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers Phil Woodland: pcw@eng.cam.ac.uk Michaelmas 202 Engineering Part IIB:

More information

QUANTIZATION FOR DISTRIBUTED ESTIMATION IN LARGE SCALE SENSOR NETWORKS

QUANTIZATION FOR DISTRIBUTED ESTIMATION IN LARGE SCALE SENSOR NETWORKS QUANTIZATION FOR DISTRIBUTED ESTIMATION IN LARGE SCALE SENSOR NETWORKS Parvathinathan Venkitasubramaniam, Gökhan Mergen, Lang Tong and Ananthram Swami ABSTRACT We study the problem of quantization for

More information

Bayesian Methods in Positioning Applications

Bayesian Methods in Positioning Applications Bayesian Methods in Positioning Applications Vedran Dizdarević v.dizdarevic@tugraz.at Graz University of Technology, Austria 24. May 2006 Bayesian Methods in Positioning Applications p.1/21 Outline Problem

More information

A Unifying View of Image Similarity

A Unifying View of Image Similarity ppears in Proceedings International Conference on Pattern Recognition, Barcelona, Spain, September 1 Unifying View of Image Similarity Nuno Vasconcelos and ndrew Lippman MIT Media Lab, nuno,lip mediamitedu

More information

EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER

EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER Zhen Zhen 1, Jun Young Lee 2, and Abdus Saboor 3 1 Mingde College, Guizhou University, China zhenz2000@21cn.com 2 Department

More information

Human Pose Tracking I: Basics. David Fleet University of Toronto

Human Pose Tracking I: Basics. David Fleet University of Toronto Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated

More information

Lecture 5: Bayes pt. 1

Lecture 5: Bayes pt. 1 Lecture 5: Bayes pt. 1 D. Jason Koskinen koskinen@nbi.ku.dk Photo by Howard Jackman University of Copenhagen Advanced Methods in Applied Statistics Feb - Apr 2016 Niels Bohr Institute 2 Bayes Probabilities

More information

Bayesian X-ray Computed Tomography using a Three-level Hierarchical Prior Model

Bayesian X-ray Computed Tomography using a Three-level Hierarchical Prior Model L. Wang, A. Mohammad-Djafari, N. Gac, MaxEnt 16, Ghent, Belgium. 1/26 Bayesian X-ray Computed Tomography using a Three-level Hierarchical Prior Model Li Wang, Ali Mohammad-Djafari, Nicolas Gac Laboratoire

More information

10. Composite Hypothesis Testing. ECE 830, Spring 2014

10. Composite Hypothesis Testing. ECE 830, Spring 2014 10. Composite Hypothesis Testing ECE 830, Spring 2014 1 / 25 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve unknown parameters

More information

BROADBAND MIMO SONAR SYSTEM: A THEORETICAL AND EXPERIMENTAL APPROACH

BROADBAND MIMO SONAR SYSTEM: A THEORETICAL AND EXPERIMENTAL APPROACH BROADBAND MIMO SONAR SYSTM: A THORTICAL AND XPRIMNTAL APPROACH Yan Pailhas a, Yvan Petillot a, Chris Capus a, Keith Brown a a Oceans Systems Lab., School of PS, Heriot Watt University, dinburgh, Scotland,

More information

Probabilistic Graphical Networks: Definitions and Basic Results

Probabilistic Graphical Networks: Definitions and Basic Results This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Le Song Machine Learning I CSE 6740, Fall 2013 Naïve Bayes classifier Still use Bayes decision rule for classification P y x = P x y P y P x But assume p x y = 1 is fully factorized

More information

Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference

Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference Shunsuke Horii Waseda University s.horii@aoni.waseda.jp Abstract In this paper, we present a hierarchical model which

More information

Introduction to Statistical Inference

Introduction to Statistical Inference Structural Health Monitoring Using Statistical Pattern Recognition Introduction to Statistical Inference Presented by Charles R. Farrar, Ph.D., P.E. Outline Introduce statistical decision making for Structural

More information

Machine Learning Overview

Machine Learning Overview Machine Learning Overview Sargur N. Srihari University at Buffalo, State University of New York USA 1 Outline 1. What is Machine Learning (ML)? 2. Types of Information Processing Problems Solved 1. Regression

More information

Lecture : Probabilistic Machine Learning

Lecture : Probabilistic Machine Learning Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning

More information

Introduction to Machine Learning

Introduction to Machine Learning Outline Introduction to Machine Learning Bayesian Classification Varun Chandola March 8, 017 1. {circular,large,light,smooth,thick}, malignant. {circular,large,light,irregular,thick}, malignant 3. {oval,large,dark,smooth,thin},

More information

THE problem of phase noise and its influence on oscillators

THE problem of phase noise and its influence on oscillators IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 54, NO. 5, MAY 2007 435 Phase Diffusion Coefficient for Oscillators Perturbed by Colored Noise Fergal O Doherty and James P. Gleeson Abstract

More information

Chapter 3: Maximum-Likelihood & Bayesian Parameter Estimation (part 1)

Chapter 3: Maximum-Likelihood & Bayesian Parameter Estimation (part 1) HW 1 due today Parameter Estimation Biometrics CSE 190 Lecture 7 Today s lecture was on the blackboard. These slides are an alternative presentation of the material. CSE190, Winter10 CSE190, Winter10 Chapter

More information

Sound Source Tracking Using Microphone Arrays

Sound Source Tracking Using Microphone Arrays Sound Source Tracking Using Microphone Arrays WANG PENG and WEE SER Center for Signal Processing School of Electrical & Electronic Engineering Nanayang Technological Univerisy SINGAPORE, 639798 Abstract:

More information

MIT Spring 2016

MIT Spring 2016 MIT 18.655 Dr. Kempthorne Spring 2016 1 MIT 18.655 Outline 1 2 MIT 18.655 Decision Problem: Basic Components P = {P θ : θ Θ} : parametric model. Θ = {θ}: Parameter space. A{a} : Action space. L(θ, a) :

More information

Intro. ANN & Fuzzy Systems. Lecture 15. Pattern Classification (I): Statistical Formulation

Intro. ANN & Fuzzy Systems. Lecture 15. Pattern Classification (I): Statistical Formulation Lecture 15. Pattern Classification (I): Statistical Formulation Outline Statistical Pattern Recognition Maximum Posterior Probability (MAP) Classifier Maximum Likelihood (ML) Classifier K-Nearest Neighbor

More information

Linear Classification: Probabilistic Generative Models

Linear Classification: Probabilistic Generative Models Linear Classification: Probabilistic Generative Models Sargur N. University at Buffalo, State University of New York USA 1 Linear Classification using Probabilistic Generative Models Topics 1. Overview

More information

Virtual Array Processing for Active Radar and Sonar Sensing

Virtual Array Processing for Active Radar and Sonar Sensing SCHARF AND PEZESHKI: VIRTUAL ARRAY PROCESSING FOR ACTIVE SENSING Virtual Array Processing for Active Radar and Sonar Sensing Louis L. Scharf and Ali Pezeshki Abstract In this paper, we describe how an

More information

Accelerated MRI Image Reconstruction

Accelerated MRI Image Reconstruction IMAGING DATA EVALUATION AND ANALYTICS LAB (IDEAL) CS5540: Computational Techniques for Analyzing Clinical Data Lecture 15: Accelerated MRI Image Reconstruction Ashish Raj, PhD Image Data Evaluation and

More information

Sparse Sensing for Statistical Inference

Sparse Sensing for Statistical Inference Sparse Sensing for Statistical Inference Model-driven and data-driven paradigms Geert Leus, Sundeep Chepuri, and Georg Kail ITA 2016, 04 Feb 2016 1/17 Power networks, grid analytics Health informatics

More information

Pattern Recognition. Parameter Estimation of Probability Density Functions

Pattern Recognition. Parameter Estimation of Probability Density Functions Pattern Recognition Parameter Estimation of Probability Density Functions Classification Problem (Review) The classification problem is to assign an arbitrary feature vector x F to one of c classes. The

More information

Scalable robust hypothesis tests using graphical models

Scalable robust hypothesis tests using graphical models Scalable robust hypothesis tests using graphical models Umamahesh Srinivas ipal Group Meeting October 22, 2010 Binary hypothesis testing problem Random vector x = (x 1,...,x n ) R n generated from either

More information

Sequential Bayesian Estimation of the Probability of Detection for Tracking

Sequential Bayesian Estimation of the Probability of Detection for Tracking 2th International Conference on Information Fusion Seattle, WA, USA, July 6-9, 2009 Sequential Bayesian Estimation of the Probability of Detection for Tracking Kevin G. Jamieson Applied Physics Lab University

More information

Stat 535 C - Statistical Computing & Monte Carlo Methods. Arnaud Doucet.

Stat 535 C - Statistical Computing & Monte Carlo Methods. Arnaud Doucet. Stat 535 C - Statistical Computing & Monte Carlo Methods Arnaud Doucet Email: arnaud@cs.ubc.ca 1 Suggested Projects: www.cs.ubc.ca/~arnaud/projects.html First assignement on the web: capture/recapture.

More information

Multivariate statistical methods and data mining in particle physics

Multivariate statistical methods and data mining in particle physics Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general

More information

Ch 4. Linear Models for Classification

Ch 4. Linear Models for Classification Ch 4. Linear Models for Classification Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Department of Computer Science and Engineering Pohang University of Science and echnology 77 Cheongam-ro,

More information

Jayalath Ekanayake Jonas Tappolet Harald Gall Abraham Bernstein. Time variance and defect prediction in software projects: additional figures

Jayalath Ekanayake Jonas Tappolet Harald Gall Abraham Bernstein. Time variance and defect prediction in software projects: additional figures Jayalath Ekanayake Jonas Tappolet Harald Gall Abraham Bernstein TECHNICAL REPORT No. IFI-2.4 Time variance and defect prediction in software projects: additional figures 2 University of Zurich Department

More information

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions DD2431 Autumn, 2014 1 2 3 Classification with Probability Distributions Estimation Theory Classification in the last lecture we assumed we new: P(y) Prior P(x y) Lielihood x2 x features y {ω 1,..., ω K

More information

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis

More information

Detection in reverberation using space time adaptive prewhiteners

Detection in reverberation using space time adaptive prewhiteners Detection in reverberation using space time adaptive prewhiteners Wei Li,,2 Xiaochuan Ma, Yun Zhu, Jun Yang,,2 and Chaohuan Hou Institute of Acoustics, Chinese Academy of Sciences 2 Graduate University

More information

Advanced statistical methods for data analysis Lecture 1

Advanced statistical methods for data analysis Lecture 1 Advanced statistical methods for data analysis Lecture 1 RHUL Physics www.pp.rhul.ac.uk/~cowan Universität Mainz Klausurtagung des GK Eichtheorien exp. Tests... Bullay/Mosel 15 17 September, 2008 1 Outline

More information

Lecture 6: Model Checking and Selection

Lecture 6: Model Checking and Selection Lecture 6: Model Checking and Selection Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de May 27, 2014 Model selection We often have multiple modeling choices that are equally sensible: M 1,, M T. Which

More information

Active Sonar Target Classification Using Classifier Ensembles

Active Sonar Target Classification Using Classifier Ensembles International Journal of Engineering Research and Technology. ISSN 0974-3154 Volume 11, Number 12 (2018), pp. 2125-2133 International Research Publication House http://www.irphouse.com Active Sonar Target

More information

Fusion of Decisions Transmitted Over Fading Channels in Wireless Sensor Networks

Fusion of Decisions Transmitted Over Fading Channels in Wireless Sensor Networks Fusion of Decisions Transmitted Over Fading Channels in Wireless Sensor Networks Biao Chen, Ruixiang Jiang, Teerasit Kasetkasem, and Pramod K. Varshney Syracuse University, Department of EECS, Syracuse,

More information

Overview. Probabilistic Interpretation of Linear Regression Maximum Likelihood Estimation Bayesian Estimation MAP Estimation

Overview. Probabilistic Interpretation of Linear Regression Maximum Likelihood Estimation Bayesian Estimation MAP Estimation Overview Probabilistic Interpretation of Linear Regression Maximum Likelihood Estimation Bayesian Estimation MAP Estimation Probabilistic Interpretation: Linear Regression Assume output y is generated

More information

σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =

σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) = Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,

More information

computation of the algorithms it is useful to introduce some sort of mapping that reduces the dimension of the data set before applying signal process

computation of the algorithms it is useful to introduce some sort of mapping that reduces the dimension of the data set before applying signal process Optimal Dimension Reduction for Array Processing { Generalized Soren Anderson y and Arye Nehorai Department of Electrical Engineering Yale University New Haven, CT 06520 EDICS Category: 3.6, 3.8. Abstract

More information

ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters

ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters D. Richard Brown III Worcester Polytechnic Institute 26-February-2009 Worcester Polytechnic Institute D. Richard Brown III 26-February-2009

More information

Bayesian Dropout. Tue Herlau, Morten Morup and Mikkel N. Schmidt. Feb 20, Discussed by: Yizhe Zhang

Bayesian Dropout. Tue Herlau, Morten Morup and Mikkel N. Schmidt. Feb 20, Discussed by: Yizhe Zhang Bayesian Dropout Tue Herlau, Morten Morup and Mikkel N. Schmidt Discussed by: Yizhe Zhang Feb 20, 2016 Outline 1 Introduction 2 Model 3 Inference 4 Experiments Dropout Training stage: A unit is present

More information

A Monte Carlo Sequential Estimation for Point Process Optimum Filtering

A Monte Carlo Sequential Estimation for Point Process Optimum Filtering 2006 International Joint Conference on Neural Networks Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada July 16-21, 2006 A Monte Carlo Sequential Estimation for Point Process Optimum Filtering

More information