Fundamentals of Statistical Signal Processing Volume II Detection Theory

Size: px
Start display at page:

Download "Fundamentals of Statistical Signal Processing Volume II Detection Theory"

Transcription

1 Fundamentals of Statistical Signal Processing Volume II Detection Theory Steven M. Kay University of Rhode Island PH PTR Prentice Hall PTR Upper Saddle River, New Jersey

2 Contents 1 Introduction Detection Theory in Signal Processing The Detection Problem The Mathematical Detection Problem Hierarchy of Detection Problems Role of Asymptotics Some Notes to the Reader 15 2 Summary of Important PDFs Introduction Fundamental Probability Density Functions and Properties Gaussian (Normal) Chi-Squared (Central) Chi-Squared (Noncentral) F (Central) F (Noncentral) Rayleigh Rician Quadratic Forms of Gaussian Random Variables Asymptotic Gaussian PDF Monte Carlo Performance Evaluation 36 2A Number of Required Monte Carlo Trials 45 2B Normal Probability Paper 47 2C MATLAB Program to Compute Gaussian Right-Tail Probability and its Inverse 50 2D MATLAB Program to Compute Central and Noncentral \ 2 Right- Tail Probability 52 2E MATLAB Program for Monte Carlo Computer Simulation 58 vn

3 Vlll 3 Statistical Decision Theory I Introduction Summary Neyman-Pearson Theorem Receiver Operating Characteristics Irrelevant Data Minimum Probability of Error Bayes Risk Multiple Hypothesis Testing 81 ЗА Neyman-Pearson Theorem 89 3B Minimum Bayes Risk Detector - Binary Hypothesis 90 3C Minimum Bayes Risk Detector - Multiple Hypotheses 92 4 Deterministic Signals Introduction Summary Matched Filters Development of Detector Performance of Matched Filter Generalized Matched Filters Performance of Generalized Matched Filter Multiple Signals Binary Case Performance for Binary Case M-ary Case Linear Model Signal Processing Examples 125 4A Reduced Form of the Linear Model Random Signals Introduction Summary Estimator-Correlator Linear Model Estimator-Correlator for Large Data Records General Gaussian Detection Signal Processing Example Tapped Delay Line Channel Model 169 5A Detection Performance of the Estimator-Correlator 183

4 CONTENTS ix 6 Statistical Decision Theory II Introduction Summary Summary of Composite Hypothesis Testing Composite Hypothesis Testing Composite Hypothesis Testing Approaches Bayesian Approach Generalized Likelihood Ratio Test Performance of GLRT for Large Data Records Equivalent Large Data Records Tests Locally Most Powerful Detectors Multiple Hypothesis Testing 221 6A Asymptotically Equivalent Tests - No Nuisance Parameters 232 6B Asymptotically Equivalent Tests - Nuisance Parameters 235 6C Asymptotic PDF of GLRT 239 6D Asymptotic Detection Performance of LMP Test 241 6E Alternate Derivation of Locally Most Powerful Test 243 6F Derivation of Generalized ML Rule Deterministic Signals with Unknown Parameters Introduction Summary Signal Modeling and Detection Performance Unknown Amplitude GLRT Bayesian Approach Unknown Arrival Time Sinusoidal Detection Amplitude Unknown Amplitude and Phase Unknown Amplitude, Phase, and Frequency Unknown Amplitude, Phase, Frequency, and Arrival Time Unknown Classical Linear Model Signal Processing Examples 279 7A Asymptotic Performance of the Energy Detector 297 7B Derivation of GLRT for Classical Linear Model 299

5 X 8 Random Signals with Unknown Parameters Introduction Summary Incompletely Known Signal Covariance Large Data Record Approximations Weak Signal Detection Signal Processing Example 315 8A Derivation of PDF for Periodic Gaussian Random Process Unknown Noise Parameters Introduction Summary General Considerations White Gaussian Noise Known Deterministic Signal Random Signal with Known PDF Deterministic Signal with Unknown Parameters Random Signal with Unknown PDF Parameters Colored WSS Gaussian Noise Known Deterministic Signals Deterministic Signals with Unknown Parameters Signal Processing Example 358 9A Derivation of GLRT for Classical Linear Model for a 1 Unknown B Rao Test for General Linear Model with Unknown Noise Parameters 375 9C Asymptotically Equivalent Rao Test for Signal Processing Example NonGaussian Noise Introduction Summary NonGaussian Noise Characteristics Known Deterministic Signals Deterministic Signals with Unknown Parameters Signal Processing Example A Asymptotic Performance of NP Detector for Weak Signals B Rao Test for Linear Model Signal with IID NonGaussian Noise

6 CONTENTS xi 11 Summary of Detectors Introduction Detection Approaches Linear Model Choosing a Detector Other Approaches and Other Texts Model Change Detection Introduction Summary Description of Problem Extensions to the Basic Problem Multiple Change Times Signal Processing Examples Maneuver Detection Time Varying PSD Detection A General Dynamic Programming Approach to Segmentation B MATLAB Program for Dynamic Programming Complex/Vector Extensions, and Array Processing Introduction Summary Known PDFs Matched Filter Generalized Matched Filter Estimator-Correlator PDFs with Unknown Parameters Deterministic Signal Random Signal Vector Observations and PDFs General Covariance Matrix Scaled Identity Matrix Uncorrelated from Temporal Sample to Sample Uncorrelated from Spatial Sample to Sample Detectors for Vector Observations Known Deterministic Signal in CWGN Known Deterministic Signal and General Noise Covariance. 495

7 XII CONTENTS Known Deterministic Signal in Temporally Uncorrelated Noise Known Deterministic Signal in Spatially Uncorrelated Noise Random Signal in CWGN Deterministic Signal with Unknown Parameters in CWGN Estimator-Correlator for Large Data Records Signal Processing Examples Active Sonar/Radar Broadband Passive Sonar A PDF of GLRT for Complex Linear Model 526 Al Review of Important Concepts 529 Al.l Linear and Matrix Algebra 529 Al.1.1 Definitions 529 Al.1.2 Special Matrices 531 Al. 1.3 Matrix Manipulation and Formulas 533 Al.1.4 Theorems 535 Al. 1.5 Eigendecompostion of Matrices 536 Al. 1.6 Inequalities 537 A 1.2 Random Processes and Time Series Modeling 537 A Random Process Characterization 538 Al.2.2 Gaussian Random Process 540 Al.2.3 Time Series Models 541 A2 Glossary of Symbols and Abbreviations (Vols. I & II) 545

Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

More information

DETECTION theory deals primarily with techniques for

DETECTION theory deals primarily with techniques for ADVANCED SIGNAL PROCESSING SE Optimum Detection of Deterministic and Random Signals Stefan Tertinek Graz University of Technology turtle@sbox.tugraz.at Abstract This paper introduces various methods for

More information

Outline. Detection Theory. NonGaussian NoiseCharacteristics. Example: Kurtosis: Chapter 10: NonGaussian Noise Reza Meraji Dec. 14, 2010.

Outline. Detection Theory. NonGaussian NoiseCharacteristics. Example: Kurtosis: Chapter 10: NonGaussian Noise Reza Meraji Dec. 14, 2010. Outline Detection Theory Chapter 10: NonGaussian Noise Reza Meraji Dec. 14, 2010 NonGuassian noise characteristics Kurtosis Laplacian pdf Generalized gaussian pdf Detection: Known deterministic signals

More information

Detecting Parametric Signals in Noise Having Exactly Known Pdf/Pmf

Detecting Parametric Signals in Noise Having Exactly Known Pdf/Pmf Detecting Parametric Signals in Noise Having Exactly Known Pdf/Pmf Reading: Ch. 5 in Kay-II. (Part of) Ch. III.B in Poor. EE 527, Detection and Estimation Theory, # 5c Detecting Parametric Signals in Noise

More information

Estimation, Detection, and Identification CMU 18752

Estimation, Detection, and Identification CMU 18752 Estimation, Detection, and Identification CMU 18752 Graduate Course on the CMU/Portugal ECE PhD Program Spring 2008/2009 Instructor: Prof. Paulo Jorge Oliveira pjcro @ isr.ist.utl.pt Phone: +351 21 8418053

More information

Detection & Estimation Lecture 1

Detection & Estimation Lecture 1 Detection & Estimation Lecture 1 Intro, MVUE, CRLB Xiliang Luo General Course Information Textbooks & References Fundamentals of Statistical Signal Processing: Estimation Theory/Detection Theory, Steven

More information

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Detection Theory Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Outline Neyman-Pearson Theorem Detector Performance Irrelevant Data Minimum Probability of Error Bayes Risk Multiple

More information

Lessons in Estimation Theory for Signal Processing, Communications, and Control

Lessons in Estimation Theory for Signal Processing, Communications, and Control Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL

More information

Detection & Estimation Lecture 1

Detection & Estimation Lecture 1 Detection & Estimation Lecture 1 Intro, MVUE, CRLB Xiliang Luo General Course Information Textbooks & References Fundamentals of Statistical Signal Processing: Estimation Theory/Detection Theory, Steven

More information

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

More information

F2E5216/TS1002 Adaptive Filtering and Change Detection. Course Organization. Lecture plan. The Books. Lecture 1

F2E5216/TS1002 Adaptive Filtering and Change Detection. Course Organization. Lecture plan. The Books. Lecture 1 Adaptive Filtering and Change Detection Bo Wahlberg (KTH and Fredrik Gustafsson (LiTH Course Organization Lectures and compendium: Theory, Algorithms, Applications, Evaluation Toolbox and manual: Algorithms,

More information

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING Kaitlyn Beaudet and Douglas Cochran School of Electrical, Computer and Energy Engineering Arizona State University, Tempe AZ 85287-576 USA ABSTRACT The problem

More information

Detection Theory. Composite tests

Detection Theory. Composite tests Composite tests Chapter 5: Correction Thu I claimed that the above, which is the most general case, was captured by the below Thu Chapter 5: Correction Thu I claimed that the above, which is the most general

More information

ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters

ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters D. Richard Brown III Worcester Polytechnic Institute 26-February-2009 Worcester Polytechnic Institute D. Richard Brown III 26-February-2009

More information

Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn

Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation

More information

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis

More information

Detection theory. H 0 : x[n] = w[n]

Detection theory. H 0 : x[n] = w[n] Detection Theory Detection theory A the last topic of the course, we will briefly consider detection theory. The methods are based on estimation theory and attempt to answer questions such as Is a signal

More information

Asymptotically Optimal Detection of Low Probability of Intercept Signals using Distributed Sensors

Asymptotically Optimal Detection of Low Probability of Intercept Signals using Distributed Sensors Asymptotically Optimal Detection of Low Probability of Intercept Signals using Distributed Sensors 1 Naresh Vankayalapati, Steven Kay Dept. of Electrical, Computer and Biomedical Engineering University

More information

APPENDIX 1 NEYMAN PEARSON CRITERIA

APPENDIX 1 NEYMAN PEARSON CRITERIA 54 APPENDIX NEYMAN PEARSON CRITERIA The design approaches for detectors directly follow the theory of hypothesis testing. The primary approaches to hypothesis testing problem are the classical approach

More information

EIE6207: Estimation Theory

EIE6207: Estimation Theory EIE6207: Estimation Theory Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: Steven M.

More information

Fundamental Probability and Statistics

Fundamental Probability and Statistics Fundamental Probability and Statistics "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are

More information

Introduction to Statistical Inference

Introduction to Statistical Inference Structural Health Monitoring Using Statistical Pattern Recognition Introduction to Statistical Inference Presented by Charles R. Farrar, Ph.D., P.E. Outline Introduce statistical decision making for Structural

More information

ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals

ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Matched Filter Generalized Matched Filter Signal

More information

STONY BROOK UNIVERSITY. CEAS Technical Report 829

STONY BROOK UNIVERSITY. CEAS Technical Report 829 1 STONY BROOK UNIVERSITY CEAS Technical Report 829 Variable and Multiple Target Tracking by Particle Filtering and Maximum Likelihood Monte Carlo Method Jaechan Lim January 4, 2006 2 Abstract In most applications

More information

NOISE can sometimes improve nonlinear signal processing

NOISE can sometimes improve nonlinear signal processing 488 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 59, NO. 2, FEBRUARY 2011 Noise Benefits in Quantizer-Array Correlation Detection and Watermark Decoding Ashok Patel, Member, IEEE, and Bart Kosko, Fellow,

More information

ECE 3800 Probabilistic Methods of Signal and System Analysis

ECE 3800 Probabilistic Methods of Signal and System Analysis ECE 3800 Probabilistic Methods of Signal and System Analysis Dr. Bradley J. Bazuin Western Michigan University College of Engineering and Applied Sciences Department of Electrical and Computer Engineering

More information

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY PREFACE xiii 1 Difference Equations 1.1. First-Order Difference Equations 1 1.2. pth-order Difference Equations 7

More information

Statistical Signal Processing Detection, Estimation, and Time Series Analysis

Statistical Signal Processing Detection, Estimation, and Time Series Analysis Statistical Signal Processing Detection, Estimation, and Time Series Analysis Louis L. Scharf University of Colorado at Boulder with Cedric Demeure collaborating on Chapters 10 and 11 A TT ADDISON-WESLEY

More information

TARGET DETECTION WITH FUNCTION OF COVARIANCE MATRICES UNDER CLUTTER ENVIRONMENT

TARGET DETECTION WITH FUNCTION OF COVARIANCE MATRICES UNDER CLUTTER ENVIRONMENT TARGET DETECTION WITH FUNCTION OF COVARIANCE MATRICES UNDER CLUTTER ENVIRONMENT Feng Lin, Robert C. Qiu, James P. Browning, Michael C. Wicks Cognitive Radio Institute, Department of Electrical and Computer

More information

An Invariance Property of the Generalized Likelihood Ratio Test

An Invariance Property of the Generalized Likelihood Ratio Test 352 IEEE SIGNAL PROCESSING LETTERS, VOL. 10, NO. 12, DECEMBER 2003 An Invariance Property of the Generalized Likelihood Ratio Test Steven M. Kay, Fellow, IEEE, and Joseph R. Gabriel, Member, IEEE Abstract

More information

Testing Statistical Hypotheses

Testing Statistical Hypotheses E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

More information

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review

More information

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 8, AUGUST For convenience and ease of presentation only, we assume that

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 8, AUGUST For convenience and ease of presentation only, we assume that IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 8, AUGUST 2006 3411 Statistical and Information-Theoretic Analysis of Resolution in Imaging Morteza Shahram, Member, IEEE, and Peyman Milanfar, Senior

More information

Use of the likelihood principle in physics. Statistics II

Use of the likelihood principle in physics. Statistics II Use of the likelihood principle in physics Statistics II 1 2 3 + Bayesians vs Frequentists 4 Why ML does work? hypothesis observation 5 6 7 8 9 10 11 ) 12 13 14 15 16 Fit of Histograms corresponds This

More information

Lecture 8: Information Theory and Statistics

Lecture 8: Information Theory and Statistics Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and Estimation I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 22, 2015

More information

CFAR DETECTION OF SPATIALLY DISTRIBUTED TARGETS IN K- DISTRIBUTED CLUTTER WITH UNKNOWN PARAMETERS

CFAR DETECTION OF SPATIALLY DISTRIBUTED TARGETS IN K- DISTRIBUTED CLUTTER WITH UNKNOWN PARAMETERS CFAR DETECTION OF SPATIALLY DISTRIBUTED TARGETS IN K- DISTRIBUTED CLUTTER WITH UNKNOWN PARAMETERS N. Nouar and A.Farrouki SISCOM Laboratory, Department of Electrical Engineering, University of Constantine,

More information

Lecture 8: Information Theory and Statistics

Lecture 8: Information Theory and Statistics Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang

More information

ECE531 Lecture 4b: Composite Hypothesis Testing

ECE531 Lecture 4b: Composite Hypothesis Testing ECE531 Lecture 4b: Composite Hypothesis Testing D. Richard Brown III Worcester Polytechnic Institute 16-February-2011 Worcester Polytechnic Institute D. Richard Brown III 16-February-2011 1 / 44 Introduction

More information

Irr. Statistical Methods in Experimental Physics. 2nd Edition. Frederick James. World Scientific. CERN, Switzerland

Irr. Statistical Methods in Experimental Physics. 2nd Edition. Frederick James. World Scientific. CERN, Switzerland Frederick James CERN, Switzerland Statistical Methods in Experimental Physics 2nd Edition r i Irr 1- r ri Ibn World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI CONTENTS

More information

HANDBOOK OF APPLICABLE MATHEMATICS

HANDBOOK OF APPLICABLE MATHEMATICS HANDBOOK OF APPLICABLE MATHEMATICS Chief Editor: Walter Ledermann Volume VI: Statistics PART A Edited by Emlyn Lloyd University of Lancaster A Wiley-Interscience Publication JOHN WILEY & SONS Chichester

More information

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary ECE 830 Spring 207 Instructor: R. Willett Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we saw that the likelihood

More information

Composite Hypotheses and Generalized Likelihood Ratio Tests

Composite Hypotheses and Generalized Likelihood Ratio Tests Composite Hypotheses and Generalized Likelihood Ratio Tests Rebecca Willett, 06 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve

More information

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)? ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we

More information

Problem Set 2. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 30

Problem Set 2. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 30 Problem Set 2 MAS 622J/1.126J: Pattern Recognition and Analysis Due: 5:00 p.m. on September 30 [Note: All instructions to plot data or write a program should be carried out using Matlab. In order to maintain

More information

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference

More information

Two results in statistical decision theory for detecting signals with unknown distributions and priors in white Gaussian noise.

Two results in statistical decision theory for detecting signals with unknown distributions and priors in white Gaussian noise. Two results in statistical decision theory for detecting signals with unknown distributions and priors in white Gaussian noise. Dominique Pastor GET - ENST Bretagne, CNRS UMR 2872 TAMCIC, Technopôle de

More information

Use of Wijsman's Theorem for the Ratio of Maximal Invariant Densities in Signal Detection Applications

Use of Wijsman's Theorem for the Ratio of Maximal Invariant Densities in Signal Detection Applications Use of Wijsman's Theorem for the Ratio of Maximal Invariant Densities in Signal Detection Applications Joseph R. Gabriel Naval Undersea Warfare Center Newport, Rl 02841 Steven M. Kay University of Rhode

More information

Outline. Random Variables. Examples. Random Variable

Outline. Random Variables. Examples. Random Variable Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,

More information

A Simple Example Binary Hypothesis Testing Optimal Receiver Frontend M-ary Signal Sets Message Sequences. possible signals has been transmitted.

A Simple Example Binary Hypothesis Testing Optimal Receiver Frontend M-ary Signal Sets Message Sequences. possible signals has been transmitted. Introduction I We have focused on the problem of deciding which of two possible signals has been transmitted. I Binary Signal Sets I We will generalize the design of optimum (MPE) receivers to signal sets

More information

EIE6207: Maximum-Likelihood and Bayesian Estimation

EIE6207: Maximum-Likelihood and Bayesian Estimation EIE6207: Maximum-Likelihood and Bayesian Estimation Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak

More information

Practical Statistics

Practical Statistics Practical Statistics Lecture 1 (Nov. 9): - Correlation - Hypothesis Testing Lecture 2 (Nov. 16): - Error Estimation - Bayesian Analysis - Rejecting Outliers Lecture 3 (Nov. 18) - Monte Carlo Modeling -

More information

Generalizing Stochastic Resonance by the Transformation Method

Generalizing Stochastic Resonance by the Transformation Method Generalizing Stochastic Resonance by the Transformation Method Steven Kay Dept. of Electrical, Computer, and Biomedical Engineering University of Rhode Island Kingston, RI 0288 40-874-5804 (voice) 40-782-6422

More information

A SIRV-CFAR Adaptive Detector Exploiting Persymmetric Clutter Covariance Structure

A SIRV-CFAR Adaptive Detector Exploiting Persymmetric Clutter Covariance Structure A SIRV-CFAR Adaptive Detector Exploiting Persymmetric Clutter Covariance Structure Guilhem Pailloux 2 Jean-Philippe Ovarlez Frédéric Pascal 3 and Philippe Forster 2 ONERA - DEMR/TSI Chemin de la Hunière

More information

Antonio De Maio, Maria S. Greco, and Danilo Orlando. 1.1 Historical Background and Terminology Symbols Detection Theory 6

Antonio De Maio, Maria S. Greco, and Danilo Orlando. 1.1 Historical Background and Terminology Symbols Detection Theory 6 Contents 1 Introduction to Radar Detection 1 Antonio De Maio, Maria S. Greco, and Danilo Orlando 1.1 Historical Background and Terminology 1 1.2 Symbols 5 1.3 Detection Theory 6 1.3.1 Signal and Interference

More information

Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization

Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras 1 Motivation Recall: Discrete filter Discretize the

More information

This is the published version of a paper presented at IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 2013.

This is the published version of a paper presented at IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 2013. http://www.diva-portal.org This is the published version of a paper presented at IEEE International Instrumentation and Measurement Technology Conference (IMTC), 3. Citation for the original published

More information

Estimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction

Estimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction Estimation Tasks Short Course on Image Quality Matthew A. Kupinski Introduction Section 13.3 in B&M Keep in mind the similarities between estimation and classification Image-quality is a statistical concept

More information

Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis

Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis Lecture 3 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,

More information

Lecture 7 Introduction to Statistical Decision Theory

Lecture 7 Introduction to Statistical Decision Theory Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7

More information

Physics 403. Segev BenZvi. Classical Hypothesis Testing: The Likelihood Ratio Test. Department of Physics and Astronomy University of Rochester

Physics 403. Segev BenZvi. Classical Hypothesis Testing: The Likelihood Ratio Test. Department of Physics and Astronomy University of Rochester Physics 403 Classical Hypothesis Testing: The Likelihood Ratio Test Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Bayesian Hypothesis Testing Posterior Odds

More information

Lecture 22: Error exponents in hypothesis testing, GLRT

Lecture 22: Error exponents in hypothesis testing, GLRT 10-704: Information Processing and Learning Spring 2012 Lecture 22: Error exponents in hypothesis testing, GLRT Lecturer: Aarti Singh Scribe: Aarti Singh Disclaimer: These notes have not been subjected

More information

Statistical Methods in Particle Physics Lecture 1: Bayesian methods

Statistical Methods in Particle Physics Lecture 1: Bayesian methods Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan

More information

10. Composite Hypothesis Testing. ECE 830, Spring 2014

10. Composite Hypothesis Testing. ECE 830, Spring 2014 10. Composite Hypothesis Testing ECE 830, Spring 2014 1 / 25 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve unknown parameters

More information

Primer on statistics:

Primer on statistics: Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood

More information

Lecture 7. Union bound for reducing M-ary to binary hypothesis testing

Lecture 7. Union bound for reducing M-ary to binary hypothesis testing Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing

More information

Testing Statistical Hypotheses

Testing Statistical Hypotheses E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........

More information

Statistics for the LHC Lecture 2: Discovery

Statistics for the LHC Lecture 2: Discovery Statistics for the LHC Lecture 2: Discovery Academic Training Lectures CERN, 14 17 June, 2010 indico.cern.ch/conferencedisplay.py?confid=77830 Glen Cowan Physics Department Royal Holloway, University of

More information

Optimal Time Division Multiplexing Schemes for DOA Estimation of a Moving Target Using a Colocated MIMO Radar

Optimal Time Division Multiplexing Schemes for DOA Estimation of a Moving Target Using a Colocated MIMO Radar Optimal Division Multiplexing Schemes for DOA Estimation of a Moving Target Using a Colocated MIMO Radar Kilian Rambach, Markus Vogel and Bin Yang Institute of Signal Processing and System Theory University

More information

Probability and Stochastic Processes

Probability and Stochastic Processes Probability and Stochastic Processes A Friendly Introduction Electrical and Computer Engineers Third Edition Roy D. Yates Rutgers, The State University of New Jersey David J. Goodman New York University

More information

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory Statistical Inference Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory IP, José Bioucas Dias, IST, 2007

More information

Detection and Estimation Theory

Detection and Estimation Theory ESE 524 Detection and Estimation Theory Joseph A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University 2 Urbauer

More information

ECE531 Lecture 13: Sequential Detection of Discrete-Time Signals

ECE531 Lecture 13: Sequential Detection of Discrete-Time Signals ECE531 Lecture 13: Sequential Detection of Discrete-Time Signals D. Richard Brown III Worcester Polytechnic Institute 30-Apr-2009 Worcester Polytechnic Institute D. Richard Brown III 30-Apr-2009 1 / 32

More information

(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.

(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1. Problem 1 (21 points) An economist runs the regression y i = β 0 + x 1i β 1 + x 2i β 2 + x 3i β 3 + ε i (1) The results are summarized in the following table: Equation 1. Variable Coefficient Std. Error

More information

Lecture 7 Random Signal Analysis

Lecture 7 Random Signal Analysis Lecture 7 Random Signal Analysis 7. Introduction to Probability 7. Amplitude Distributions 7.3 Uniform, Gaussian, and Other Distributions 7.4 Power and Power Density Spectra 7.5 Properties of the Power

More information

Detection and Estimation Theory

Detection and Estimation Theory ESE 524 Detection and Estimation Theory Joseph A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University 2 Urbauer

More information

Parameter Estimation

Parameter Estimation 1 / 44 Parameter Estimation Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay October 25, 2012 Motivation System Model used to Derive

More information

covariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of

covariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of Index* The Statistical Analysis of Time Series by T. W. Anderson Copyright 1971 John Wiley & Sons, Inc. Aliasing, 387-388 Autoregressive {continued) Amplitude, 4, 94 case of first-order, 174 Associated

More information

Real time detection through Generalized Likelihood Ratio Test of position and speed readings inconsistencies in automated moving objects

Real time detection through Generalized Likelihood Ratio Test of position and speed readings inconsistencies in automated moving objects Real time detection through Generalized Likelihood Ratio Test of position and speed readings inconsistencies in automated moving objects Sternheim Misuraca, M. R. Degree in Electronics Engineering, University

More information

Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment

Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment Acoustical Society of America Meeting Fall 2005 Passive Sonar Detection Performance Prediction of a Moving Source in an Uncertain Environment Vivek Varadarajan and Jeffrey Krolik Duke University Department

More information

Particle Filters. Outline

Particle Filters. Outline Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical

More information

Some Contributions to Radar Detection Theory

Some Contributions to Radar Detection Theory University of Rhode Island DigitalCommons@URI Open Access Dissertations 2017 Some Contributions to Radar Detection Theory Zhenghan Zhu University of Rhode Island, zhu.zhenghan1987@gmail.com Follow this

More information

Hypothesis testing:power, test statistic CMS:

Hypothesis testing:power, test statistic CMS: Hypothesis testing:power, test statistic The more sensitive the test, the better it can discriminate between the null and the alternative hypothesis, quantitatively, maximal power In order to achieve this

More information

Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation Ashok Patel and Bart Kosko

Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation Ashok Patel and Bart Kosko IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 12, DECEMBER 2010 1005 Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation Ashok Patel and Bart Kosko Abstract A new theorem shows that

More information

p 1 p 0 (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- cant method.

p 1 p 0 (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- cant method. 80 CHAP. 2 SOLUTION OF NONLINEAR EQUATIONS f (x) = 0 y y = f(x) (p, 0) p 2 p 1 p 0 x (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- Figure 2.16 cant method. Secant Method The

More information

ECE531 Lecture 2a: A Mathematical Model for Hypothesis Testing (Finite Number of Possible Observations)

ECE531 Lecture 2a: A Mathematical Model for Hypothesis Testing (Finite Number of Possible Observations) ECE531 Lecture 2a: A Mathematical Model for Hypothesis Testing (Finite Number of Possible Observations) D. Richard Brown III Worcester Polytechnic Institute 26-January-2011 Worcester Polytechnic Institute

More information

Signal Detection Basics - CFAR

Signal Detection Basics - CFAR Signal Detection Basics - CFAR Types of noise clutter and signals targets Signal separation by comparison threshold detection Signal Statistics - Parameter estimation Threshold determination based on the

More information

Lecture Notes 4 Vector Detection and Estimation. Vector Detection Reconstruction Problem Detection for Vector AGN Channel

Lecture Notes 4 Vector Detection and Estimation. Vector Detection Reconstruction Problem Detection for Vector AGN Channel Lecture Notes 4 Vector Detection and Estimation Vector Detection Reconstruction Problem Detection for Vector AGN Channel Vector Linear Estimation Linear Innovation Sequence Kalman Filter EE 278B: Random

More information

ADAPTIVE FILTER THEORY

ADAPTIVE FILTER THEORY ADAPTIVE FILTER THEORY Fourth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada Front ice Hall PRENTICE HALL Upper Saddle River, New Jersey 07458 Preface

More information

ECE531 Lecture 2b: Bayesian Hypothesis Testing

ECE531 Lecture 2b: Bayesian Hypothesis Testing ECE531 Lecture 2b: Bayesian Hypothesis Testing D. Richard Brown III Worcester Polytechnic Institute 29-January-2009 Worcester Polytechnic Institute D. Richard Brown III 29-January-2009 1 / 39 Minimizing

More information

Asymptotic Analysis of the Generalized Coherence Estimate

Asymptotic Analysis of the Generalized Coherence Estimate IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 49, NO. 1, JANUARY 2001 45 Asymptotic Analysis of the Generalized Coherence Estimate Axel Clausen, Member, IEEE, and Douglas Cochran, Senior Member, IEEE Abstract

More information

Chapter 2 Signal Processing at Receivers: Detection Theory

Chapter 2 Signal Processing at Receivers: Detection Theory Chapter Signal Processing at Receivers: Detection Theory As an application of the statistical hypothesis testing, signal detection plays a key role in signal processing at receivers of wireless communication

More information

REVEAL. Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008

REVEAL. Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008 REVEAL Receiver Exploiting Variability in Estimated Acoustic Levels Project Review 16 Sept 2008 Presented to Program Officers: Drs. John Tague and Keith Davidson Undersea Signal Processing Team, Office

More information

3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE

3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE 3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE 3.0 INTRODUCTION The purpose of this chapter is to introduce estimators shortly. More elaborated courses on System Identification, which are given

More information

Flat Rayleigh fading. Assume a single tap model with G 0,m = G m. Assume G m is circ. symmetric Gaussian with E[ G m 2 ]=1.

Flat Rayleigh fading. Assume a single tap model with G 0,m = G m. Assume G m is circ. symmetric Gaussian with E[ G m 2 ]=1. Flat Rayleigh fading Assume a single tap model with G 0,m = G m. Assume G m is circ. symmetric Gaussian with E[ G m 2 ]=1. The magnitude is Rayleigh with f Gm ( g ) =2 g exp{ g 2 } ; g 0 f( g ) g R(G m

More information

STA414/2104 Statistical Methods for Machine Learning II

STA414/2104 Statistical Methods for Machine Learning II STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements

More information

Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws. Symeon Chatzinotas February 11, 2013 Luxembourg

Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws. Symeon Chatzinotas February 11, 2013 Luxembourg Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws Symeon Chatzinotas February 11, 2013 Luxembourg Outline 1. Random Matrix Theory 1. Definition 2. Applications 3. Asymptotics 2. Ensembles

More information

UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS

UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS UNIFORMLY MOST POWERFUL CYCLIC PERMUTATION INVARIANT DETECTION FOR DISCRETE-TIME SIGNALS F. C. Nicolls and G. de Jager Department of Electrical Engineering, University of Cape Town Rondebosch 77, South

More information

Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition

Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition Karl-Rudolf Koch Introduction to Bayesian Statistics Second, updated and enlarged Edition With 17 Figures Professor Dr.-Ing., Dr.-Ing.

More information

CO-OPERATION among multiple cognitive radio (CR)

CO-OPERATION among multiple cognitive radio (CR) 586 IEEE SIGNAL PROCESSING LETTERS, VOL 21, NO 5, MAY 2014 Sparse Bayesian Hierarchical Prior Modeling Based Cooperative Spectrum Sensing in Wideb Cognitive Radio Networks Feng Li Zongben Xu Abstract This

More information

Introduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p.

Introduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p. Preface p. xiii Acknowledgment p. xix Introduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p. 4 Bayes Decision p. 5

More information