Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department

Size: px
Start display at page:

Download "Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department"

Transcription

1 Decision-Making under Statistical Uncertainty Jayakrishnan Unnikrishnan PhD Defense ECE Department University of Illinois at Urbana-Champaign CSL June 2010

2 Statistical Decision-Making Relevant in several contexts Receiver design for communication systems Sensor networks for environment-monitoring and failure detection Drug-testing Based on probabilistic model for observations Well-studied problem but questions still remain Uncertain statistical knowledge 2

3 Statistics in Detection Example: Likelihood ratio test for binary hypotheses Hˆ = I { L ( X ) > τ } requires knowledge of likelihood ratio function LX ( ) = p1 ( X ) p ( X ) 0 3

4 Imperfect Statistics in Detection Often perfect statistical knowledge is not available e.g., fault-onset detection intrusion i detection ti anomaly detection Robust change detection ti Universal hypothesis testing spam filteringi h th i t ti primary detection and dynamic spectrum access for cognitive radio How to cope with uncertain statistics? Focus on iid i.i.d. observations Online learning 4

5 Outline Robust Quickest Change Detection Designing for worst-case guarantees minimax optimality Universal Hypothesis Testing Partial knowledge helps Universal Hypothesis Testing Model Uncertainty 5

6 Outline Robust Quickest Change Detection Designing for worst-case guarantees minimax optimality 6

7 Quickest Change Detection Single observation sequence Stopping time τ at which change is declared Tradeoff between Detection delay Frequency of false alarms Applications: process monitoring, quality control 7

8 Lorden Criterion Change-point modeled as deterministic Minimize worst-case delay subject to bound on expected time to false alarm Minimize i i WDD( τ ) subject to E ( τ ) B Eν 0 + where WDD( τ ) = supess sup E[( τ λ + 1) X,, ] λ 1 1 X λ 1 8

9 Lorden Criterion Change-point modeled as deterministic Minimize worst-case delay subject to bound on expected time to false alarm Minimize i i WDD( τ ) subject to E ( τ ) B Eν 0 + where WDD( τ ) = supess sup E[( τ λ + 1) X,, ] λ 1 CUSUM stopping rule is optimal τ ν ( X ) n 1 i C = inf{ n 1: max 1 k n η} i= k ν ( X i ) 0 ( ) 1 X λ 1 9

10 Uncertain Statistics Most known results assume pre-change and post-change distributions and are known Often ν 0 and ν 1 are not completely known in applications ν 0 ν 1 10

11 Example1: Infrastructure Monitoring Post-fault distribution is uncertain 11

12 Example 2: Intrusion Detection Post-intrusion system behavior is uncertain e.g. network security 12

13 Robust Change Detection Suppose ν 0 and ν 1 are known to be in uncertainty t classes of densities P0 and P1 Minimax robust formulation Minimize i i worst-case delay among all distributions from and P P0 1 subject to uniform bound on expected time to false alarm under all possible distributions from min sup WDD( τ ) ν P, ν P s.t. inf E ( τ ) B ν PP 0 0 ν 0 P 0 13

14 Solution via LFDs Approach: identify least favorable distributions (LFDs) under a stochastic ordering condition [Veeravalli et al.1994] Like Huber s approach to robust hypothesis testing [Huber 1965] 14

15 Solution via LFDs Approach: identify least favorable distributions (LFDs) under a stochastic ordering condition [Veeravalli et al.1994] Like Huber s approach to robust hypothesis testing [Huber 1965] For random variables we denote X X 1 2 if P ( X t ) P ( X t ) for all t 1 2 JSB condition: For ( ν, ν ) P P let * ν1(.) L (.) = ν (.) * * * * we need ( L ( X)) ν ( L ( X)) ν and ( L ( X)) ν ( L ( X)) ν ε E.g. -contamination classes, total variation and Prohorov distance neighborhoods h 0 15

16 Solution via LFDs Under JSB and some other regularity conditions the optimal stopping rule designed with respect to LFDs solves robust problem Example: P P Can easily show that LFDs are 0 1 = { N (0,1)} = { N( θ,1):0.1 θ 3} * ν 0 = N 0 (0,1) * ν 1 = N (0.1,1) 16

17 Cost of Robustness 17

18 Comparison with GLR test A benchmark scheme: CUSUM based on Generalized Likelihood Ratio (GLR test) t) τ n ν ( ) X 1 i = inf{ n 1: max sup k n η } ν P ν ( X ) GLR 1 i= k 1 1 Asymptotically ti as good as CUSUM with known distributions in exponential families Often too complex to implement 0 i Robust CUSUM admits simple recursion 18

19 Robust test vs GLR test 19

20 Other Criteria for Optimality Pollak criterion: Alternate definition for delay SRP stopping rule is asymptotically optimal Bayesian criterion: i Change-point modeled d as geometric random variable Minimize average delay subject to probability of false alarm constraint Shiryaev test is optimal 20

21 Other Criteria for Optimality Pollak criterion: Alternate definition for delay SRP stopping rule is asymptotically optimal Bayesian criterion: i Change-point modeled d as geometric random variable Minimize average delay subject to probability of false alarm constraint Shiryaev test is optimal Robust tests designed for LFDs are optimal 21

22 Outline Universal Hypothesis Testing Partial knowledge helps 22

23 Universal Hypothesis Testing Given a sequence of i.i.d. observations X1, X 2,, X n test t whether they were drawn according to a modeled distribution Null H : X ~ p 0 i 0 Alternate H 1: X i ~ p p 0, p unknown Applications: anomaly detection, spam filtering i etc. p 0 23

24 Hoeffding s Universal Test Hoeffding test is optimal in error-exponent sense: Hˆ = I { D( p p ) > τ} Uses Kullback-Leibler divergence as test statistic n 0 { q : D ( q p ) τ } 0 p 0 N n 2 24

25 Hoeffding s Universal Test Hoeffding test is optimal in error-exponent sense: Hˆ = I { D( p p ) > τ} Uses Kullback-Leibler divergence as test statistic τ Select for target false alarm probability via n Sanov s Theorem in Large Deviations (error-exponents) p = P ( Hˆ 0) exp( nτ ) FA p 0 0 Weak convergence under p 0 2 nd ( pn p ) χ N d. 2 0 n N 1 n 2 25

26 Error exponents are inaccurate Alphabet size, N = 20 26

27 Large Alphabet Regime Hoeffding test performs poorly for large (alphabet size) suffers from high bias and variance N N 1 E p [ D( p 0 n p0)] 2n N 1 Var p [ D( p 0 n p0)] 2 2n N n 2 27

28 Large Alphabet Regime Hoeffding test performs poorly for large (alphabet size) suffers from high bias and variance N N 1 E p [ D( p 0 n p0)] 2n N 1 Var p [ D( p 0 n p0)] 2 2n Can do better if we have partial information about alternate hypothesis N n 2 28

29 Mismatched Test Mismatched test uses mismatched divergence instead of KL divergence Hˆ = I { D MM ( p p ) > τ } introduced as a lower bound to KL divergence n 0 MM test is equivalent to replacing with ML estimate t from a family { } i.e., it is a GLRT p θ 0 ˆ θ ML 0 p n MM D ( p p ) = D ( p p ) n n 29

30 Mismatched Test properties + Addresses high variance issues MM )] d E p [ D ( p 0 n p0 2n d Var 2n MM p [ D ( p 0 n p0)] 2 d where θ - However, sub-optimal in error-exponent sense + Optimal when alternate distribution lies in { p θ } 30

31 Mismatched Test properties + Addresses high variance issues MM )] d E p [ D ( p 0 n p0 2n d Var 2n MM p [ D ( p 0 n p0)] 2 d where θ - However, sub-optimal in error-exponent sense + Optimal when alternate distribution lies in { p θ } Partial knowledge of unknown alternate distribution can give substantial performance improvement for large alphabets 31

32 Performance comparison N = 19, n = 40 32

33 Outline Universal Hypothesis Testing under Model Uncertainty 33

34 Uncertain Null Hypothesis Consider following hypothesis testing problem H : X ~ p, for any p P 0 i H1 : Xi ~ q, for anyq P A robust universal formulation Relevant when null hypothesis distribution is uncertain Pandit and Meyn studied this when is P = { p: p ( x) ψ i ( x ) = 0}, 1 i d x P p 0 34

35 Robust Hoeffding Test Robust Hoeffding test ˆ ROB H = I { D ( p P ) > τ} ROB where D ( q P): = inf D( q p) n p P { q : D ( q p ) τ } 0 p 0 35

36 Robust Hoeffding Test Robust Hoeffding test ˆ ROB H = I { D ( p P ) > τ} ROB where D ( q P): = inf D( q p) n p P { q : D ( q p ) τ } 0 ROB { q: D ( q P ) τ} P p 0 36

37 Robust Hoeffding Test Robust Hoeffding test ˆ ROB H = I { D ( p P ) > τ} ROB where D ( q P): = inf D( q p) Guarantees exponential decay of worst-case false alarm probability bilit n p P max H ˆ 0) exp( nτ P ( P ) p P p - Error-exponents not good indicator of error probability 37

38 Weak Convergence Result Can interpret robust divergence as a mismatched divergence Yields weak convergence result under p ROB 2 nd ( p P ) χ n d. 2 n d where dp d gives better approximation for false alarm probability Similar robust Kolmogorov-Smirnov test for continuous distributions p 38

39 Kolmogorov-Smirnov Test Universal hypothesis test for continuous alphabet H : X ~ F 0 i 0 KS test statistic where D = F x F x n 0 sup n ( ) ( ) x n 1 Fn( x) = I{ Xi x} n n i = 1 Thresholds set using weak convergence of Problem of overfitting for large n D n 39

40 Robust KS Test unknown from uncertainty class F 0 F + = { F : F ( x) F( x) F ( x), x} F ( x) 0 40

41 Robust KS Test unknown from uncertainty class F 0 F + = { F : F ( x) F( x) F ( x), x} F ( x ) F ( x) 0 F + ( x) 41

42 Robust KS Test Uncertainty class via stochastic ordering F + = { F : F ( x) F( x) F ( x), x} Modified test statistic E = min sup F ( x ) F ( x ) n F FF n x We obtain weak convergence results for that n are useful for setting thresholds E n 42

43 Conclusion Various approaches to coping with uncertainty Robust change detection: Designing for LFDs guarantees minimax optimality Universal hypothesis testing: Partial knowledge improves performance Dynamic spectrum access: Online learning 43

44 Conclusion Various approaches to coping with uncertainty Robust change detection: Designing for LFDs guarantees minimax optimality Universal hypothesis testing: Partial knowledge improves performance Dynamic spectrum access: Online learning Extensions Performance analysis of other robust stopping rules Adapting dimensionality d with observation length n Convergence rates of weak convergence results Extending to non - iid i.i.d. setting 44

45 Thank You! 45

46 References J. Unnikrishnan, D. Huang, S. Meyn, A. Surana, and V. V. Veeravalli, Universal and Composite Hypothesis Testing via Mismatched Divergence IEEE Trans. Inf. Theory, revised April J. Unnikrishnan, V. V. Veeravalli, and S. Meyn, Minimax Robust Quickest Change Detection submitted to IEEE Trans. Inf. Theory, revised May, J. Unnikrishnan, S. Meyn, and V. Veeravalli, On Thresholds for Robust Goodness-of-Fit Tests to be presented at IEEE Information Theory Workshop, Dublin, Aug available at illinois edu/~junnikr2 46

Data-Efficient Quickest Change Detection

Data-Efficient Quickest Change Detection Data-Efficient Quickest Change Detection Venu Veeravalli ECE Department & Coordinated Science Lab University of Illinois at Urbana-Champaign http://www.ifp.illinois.edu/~vvv (joint work with Taposh Banerjee)

More information

Sequential Detection. Changes: an overview. George V. Moustakides

Sequential Detection. Changes: an overview. George V. Moustakides Sequential Detection of Changes: an overview George V. Moustakides Outline Sequential hypothesis testing and Sequential detection of changes The Sequential Probability Ratio Test (SPRT) for optimum hypothesis

More information

Least Favorable Distributions for Robust Quickest Change Detection

Least Favorable Distributions for Robust Quickest Change Detection Least Favorable Distributions for Robust Quickest hange Detection Jayakrishnan Unnikrishnan, Venugopal V. Veeravalli, Sean Meyn Department of Electrical and omputer Engineering, and oordinated Science

More information

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 3, MARCH

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 3, MARCH IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 57, NO 3, MARCH 2011 1587 Universal and Composite Hypothesis Testing via Mismatched Divergence Jayakrishnan Unnikrishnan, Member, IEEE, Dayu Huang, Student

More information

Asymptotically Optimal Quickest Change Detection in Distributed Sensor Systems

Asymptotically Optimal Quickest Change Detection in Distributed Sensor Systems This article was downloaded by: [University of Illinois at Urbana-Champaign] On: 23 May 2012, At: 16:03 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954

More information

Statistical Models and Algorithms for Real-Time Anomaly Detection Using Multi-Modal Data

Statistical Models and Algorithms for Real-Time Anomaly Detection Using Multi-Modal Data Statistical Models and Algorithms for Real-Time Anomaly Detection Using Multi-Modal Data Taposh Banerjee University of Texas at San Antonio Joint work with Gene Whipps (US Army Research Laboratory) Prudhvi

More information

Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes

Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes George V. Moustakides INRIA, Rennes, France Outline The change detection problem Overview of existing results Lorden s criterion and

More information

Finding the best mismatched detector for channel coding and hypothesis testing

Finding the best mismatched detector for channel coding and hypothesis testing Finding the best mismatched detector for channel coding and hypothesis testing Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory

More information

CHANGE DETECTION WITH UNKNOWN POST-CHANGE PARAMETER USING KIEFER-WOLFOWITZ METHOD

CHANGE DETECTION WITH UNKNOWN POST-CHANGE PARAMETER USING KIEFER-WOLFOWITZ METHOD CHANGE DETECTION WITH UNKNOWN POST-CHANGE PARAMETER USING KIEFER-WOLFOWITZ METHOD Vijay Singamasetty, Navneeth Nair, Srikrishna Bhashyam and Arun Pachai Kannu Department of Electrical Engineering Indian

More information

Quickest Anomaly Detection: A Case of Active Hypothesis Testing

Quickest Anomaly Detection: A Case of Active Hypothesis Testing Quickest Anomaly Detection: A Case of Active Hypothesis Testing Kobi Cohen, Qing Zhao Department of Electrical Computer Engineering, University of California, Davis, CA 95616 {yscohen, qzhao}@ucdavis.edu

More information

Lecture 8: Information Theory and Statistics

Lecture 8: Information Theory and Statistics Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang

More information

Quickest Changepoint Detection: Optimality Properties of the Shiryaev Roberts-Type Procedures

Quickest Changepoint Detection: Optimality Properties of the Shiryaev Roberts-Type Procedures Quickest Changepoint Detection: Optimality Properties of the Shiryaev Roberts-Type Procedures Alexander Tartakovsky Department of Statistics a.tartakov@uconn.edu Inference for Change-Point and Related

More information

Simultaneous and sequential detection of multiple interacting change points

Simultaneous and sequential detection of multiple interacting change points Simultaneous and sequential detection of multiple interacting change points Long Nguyen Department of Statistics University of Michigan Joint work with Ram Rajagopal (Stanford University) 1 Introduction

More information

Quickest Detection With Post-Change Distribution Uncertainty

Quickest Detection With Post-Change Distribution Uncertainty Quickest Detection With Post-Change Distribution Uncertainty Heng Yang City University of New York, Graduate Center Olympia Hadjiliadis City University of New York, Brooklyn College and Graduate Center

More information

COMPARISON OF STATISTICAL ALGORITHMS FOR POWER SYSTEM LINE OUTAGE DETECTION

COMPARISON OF STATISTICAL ALGORITHMS FOR POWER SYSTEM LINE OUTAGE DETECTION COMPARISON OF STATISTICAL ALGORITHMS FOR POWER SYSTEM LINE OUTAGE DETECTION Georgios Rovatsos*, Xichen Jiang*, Alejandro D. Domínguez-García, and Venugopal V. Veeravalli Department of Electrical and Computer

More information

arxiv: v2 [eess.sp] 20 Nov 2017

arxiv: v2 [eess.sp] 20 Nov 2017 Distributed Change Detection Based on Average Consensus Qinghua Liu and Yao Xie November, 2017 arxiv:1710.10378v2 [eess.sp] 20 Nov 2017 Abstract Distributed change-point detection has been a fundamental

More information

Lecture 8: Information Theory and Statistics

Lecture 8: Information Theory and Statistics Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and Estimation I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 22, 2015

More information

SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN. Tze Leung Lai Haipeng Xing

SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN. Tze Leung Lai Haipeng Xing SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN By Tze Leung Lai Haipeng Xing Technical Report No. 2009-5 April 2009 Department of Statistics STANFORD UNIVERSITY

More information

Entropy, Inference, and Channel Coding

Entropy, Inference, and Channel Coding Entropy, Inference, and Channel Coding Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory NSF support: ECS 02-17836, ITR 00-85929

More information

Early Detection of a Change in Poisson Rate After Accounting For Population Size Effects

Early Detection of a Change in Poisson Rate After Accounting For Population Size Effects Early Detection of a Change in Poisson Rate After Accounting For Population Size Effects School of Industrial and Systems Engineering, Georgia Institute of Technology, 765 Ferst Drive NW, Atlanta, GA 30332-0205,

More information

Scalable robust hypothesis tests using graphical models

Scalable robust hypothesis tests using graphical models Scalable robust hypothesis tests using graphical models Umamahesh Srinivas ipal Group Meeting October 22, 2010 Binary hypothesis testing problem Random vector x = (x 1,...,x n ) R n generated from either

More information

Universal and Composite Hypothesis Testing via Mismatched Divergence

Universal and Composite Hypothesis Testing via Mismatched Divergence Universal and Composite Hypothesis Testing via Mismatched Divergence Jayakrishnan Unnikrishnan, Dayu Huang, Sean Meyn, Amit Surana and Venugopal Veeravalli Abstract arxiv:0909.2234v1 [cs.it] 11 Sep 2009

More information

Large-Scale Multi-Stream Quickest Change Detection via Shrinkage Post-Change Estimation

Large-Scale Multi-Stream Quickest Change Detection via Shrinkage Post-Change Estimation Large-Scale Multi-Stream Quickest Change Detection via Shrinkage Post-Change Estimation Yuan Wang and Yajun Mei arxiv:1308.5738v3 [math.st] 16 Mar 2016 July 10, 2015 Abstract The quickest change detection

More information

SCALABLE ROBUST MONITORING OF LARGE-SCALE DATA STREAMS. By Ruizhi Zhang and Yajun Mei Georgia Institute of Technology

SCALABLE ROBUST MONITORING OF LARGE-SCALE DATA STREAMS. By Ruizhi Zhang and Yajun Mei Georgia Institute of Technology Submitted to the Annals of Statistics SCALABLE ROBUST MONITORING OF LARGE-SCALE DATA STREAMS By Ruizhi Zhang and Yajun Mei Georgia Institute of Technology Online monitoring large-scale data streams has

More information

Lecture 7 Introduction to Statistical Decision Theory

Lecture 7 Introduction to Statistical Decision Theory Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7

More information

Surveillance of BiometricsAssumptions

Surveillance of BiometricsAssumptions Surveillance of BiometricsAssumptions in Insured Populations Journée des Chaires, ILB 2017 N. El Karoui, S. Loisel, Y. Sahli UPMC-Paris 6/LPMA/ISFA-Lyon 1 with the financial support of ANR LoLitA, and

More information

Universal and Composite Hypothesis Testing via Mismatched Divergence

Universal and Composite Hypothesis Testing via Mismatched Divergence 1 Universal and Composite Hypothesis Testing via Mismatched Divergence arxiv:0909.2234v3 [cs.it] 9 Sep 2010 Jayakrishnan Unnikrishnan, Dayu Huang, Sean Meyn, Amit Surana and Venugopal Veeravalli Abstract

More information

Applications of Information Geometry to Hypothesis Testing and Signal Detection

Applications of Information Geometry to Hypothesis Testing and Signal Detection CMCAA 2016 Applications of Information Geometry to Hypothesis Testing and Signal Detection Yongqiang Cheng National University of Defense Technology July 2016 Outline 1. Principles of Information Geometry

More information

X 1,n. X L, n S L S 1. Fusion Center. Final Decision. Information Bounds and Quickest Change Detection in Decentralized Decision Systems

X 1,n. X L, n S L S 1. Fusion Center. Final Decision. Information Bounds and Quickest Change Detection in Decentralized Decision Systems 1 Information Bounds and Quickest Change Detection in Decentralized Decision Systems X 1,n X L, n Yajun Mei Abstract The quickest change detection problem is studied in decentralized decision systems,

More information

A CUSUM approach for online change-point detection on curve sequences

A CUSUM approach for online change-point detection on curve sequences ESANN 22 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges Belgium, 25-27 April 22, i6doc.com publ., ISBN 978-2-8749-49-. Available

More information

Monitoring actuarial assumptions in life insurance

Monitoring actuarial assumptions in life insurance Monitoring actuarial assumptions in life insurance Stéphane Loisel ISFA, Univ. Lyon 1 Joint work with N. El Karoui & Y. Salhi IAALS Colloquium, Barcelona, 17 LoLitA Typical paths with change of regime

More information

Distributed detection of topological changes in communication networks. Riccardo Lucchese, Damiano Varagnolo, Karl H. Johansson

Distributed detection of topological changes in communication networks. Riccardo Lucchese, Damiano Varagnolo, Karl H. Johansson 1 Distributed detection of topological changes in communication networks Riccardo Lucchese, Damiano Varagnolo, Karl H. Johansson Thanks to... 2 The need: detecting changes in topological networks 3 The

More information

Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid

Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid 1 Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid Mehmet Necip Kurt, Yasin Yılmaz, Member, IEEE, and Xiaodong Wang, Fellow, IEEE Abstract For a safe and reliable operation of the

More information

MISMATCHED DIVERGENCE AND UNIVERSAL HYPOTHESIS TESTING DAYU HUANG THESIS

MISMATCHED DIVERGENCE AND UNIVERSAL HYPOTHESIS TESTING DAYU HUANG THESIS c 2009 Dayu Huang MISMATCHED DIVERGENCE AND UNIVERSAL HYPOTHESIS TESTING BY DAYU HUANG THESIS Submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical and

More information

Information Theory and Hypothesis Testing

Information Theory and Hypothesis Testing Summer School on Game Theory and Telecommunications Campione, 7-12 September, 2014 Information Theory and Hypothesis Testing Mauro Barni University of Siena September 8 Review of some basic results linking

More information

Local Differential Privacy

Local Differential Privacy Local Differential Privacy Peter Kairouz Department of Electrical & Computer Engineering University of Illinois at Urbana-Champaign Joint work with Sewoong Oh (UIUC) and Pramod Viswanath (UIUC) / 33 Wireless

More information

Lecture 22: Error exponents in hypothesis testing, GLRT

Lecture 22: Error exponents in hypothesis testing, GLRT 10-704: Information Processing and Learning Spring 2012 Lecture 22: Error exponents in hypothesis testing, GLRT Lecturer: Aarti Singh Scribe: Aarti Singh Disclaimer: These notes have not been subjected

More information

Bayesian Quickest Change Detection Under Energy Constraints

Bayesian Quickest Change Detection Under Energy Constraints Bayesian Quickest Change Detection Under Energy Constraints Taposh Banerjee and Venugopal V. Veeravalli ECE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign, Urbana,

More information

Quantization Effect on the Log-Likelihood Ratio and Its Application to Decentralized Sequential Detection

Quantization Effect on the Log-Likelihood Ratio and Its Application to Decentralized Sequential Detection 1536 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 61, NO. 6, MARCH 15, 2013 Quantization Effect on the Log-Likelihood Ratio Its Application to Decentralized Sequential Detection Yan Wang Yajun Mei Abstract

More information

EXTENDED GLRT DETECTORS OF CORRELATION AND SPHERICITY: THE UNDERSAMPLED REGIME. Xavier Mestre 1, Pascal Vallet 2

EXTENDED GLRT DETECTORS OF CORRELATION AND SPHERICITY: THE UNDERSAMPLED REGIME. Xavier Mestre 1, Pascal Vallet 2 EXTENDED GLRT DETECTORS OF CORRELATION AND SPHERICITY: THE UNDERSAMPLED REGIME Xavier Mestre, Pascal Vallet 2 Centre Tecnològic de Telecomunicacions de Catalunya, Castelldefels, Barcelona (Spain) 2 Institut

More information

Accuracy and Decision Time for Decentralized Implementations of the Sequential Probability Ratio Test

Accuracy and Decision Time for Decentralized Implementations of the Sequential Probability Ratio Test 21 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July 2, 21 ThA1.3 Accuracy Decision Time for Decentralized Implementations of the Sequential Probability Ratio Test Sra Hala

More information

Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

More information

EARLY DETECTION OF A CHANGE IN POISSON RATE AFTER ACCOUNTING FOR POPULATION SIZE EFFECTS

EARLY DETECTION OF A CHANGE IN POISSON RATE AFTER ACCOUNTING FOR POPULATION SIZE EFFECTS Statistica Sinica 21 (2011), 597-624 EARLY DETECTION OF A CHANGE IN POISSON RATE AFTER ACCOUNTING FOR POPULATION SIZE EFFECTS Yajun Mei, Sung Won Han and Kwok-Leung Tsui Georgia Institute of Technology

More information

arxiv:math/ v2 [math.st] 15 May 2006

arxiv:math/ v2 [math.st] 15 May 2006 The Annals of Statistics 2006, Vol. 34, No. 1, 92 122 DOI: 10.1214/009053605000000859 c Institute of Mathematical Statistics, 2006 arxiv:math/0605322v2 [math.st] 15 May 2006 SEQUENTIAL CHANGE-POINT DETECTION

More information

Sequential Change-Point Approach for Online Community Detection

Sequential Change-Point Approach for Online Community Detection Sequential Change-Point Approach for Online Community Detection Yao Xie Joint work with David Marangoni-Simonsen H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology

More information

10-704: Information Processing and Learning Fall Lecture 24: Dec 7

10-704: Information Processing and Learning Fall Lecture 24: Dec 7 0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 24: Dec 7 Note: These notes are based on scribed notes from Spring5 offering of this course. LaTeX template courtesy of

More information

The Method of Types and Its Application to Information Hiding

The Method of Types and Its Application to Information Hiding The Method of Types and Its Application to Information Hiding Pierre Moulin University of Illinois at Urbana-Champaign www.ifp.uiuc.edu/ moulin/talks/eusipco05-slides.pdf EUSIPCO Antalya, September 7,

More information

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary ECE 830 Spring 207 Instructor: R. Willett Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we saw that the likelihood

More information

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis

More information

A Novel Asynchronous Communication Paradigm: Detection, Isolation, and Coding

A Novel Asynchronous Communication Paradigm: Detection, Isolation, and Coding A Novel Asynchronous Communication Paradigm: Detection, Isolation, and Coding The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation

More information

Performance of Certain Decentralized Distributed Change Detection Procedures

Performance of Certain Decentralized Distributed Change Detection Procedures Performance of Certain Decentralized Distributed Change Detection Procedures Alexander G. Tartakovsky Center for Applied Mathematical Sciences and Department of Mathematics University of Southern California

More information

Decentralized Sequential Hypothesis Testing. Change Detection

Decentralized Sequential Hypothesis Testing. Change Detection Decentralized Sequential Hypothesis Testing & Change Detection Giorgos Fellouris, Columbia University, NY, USA George V. Moustakides, University of Patras, Greece Outline Sequential hypothesis testing

More information

Anonymous Heterogeneous Distributed Detection: Optimal Decision Rules, Error Exponents, and the Price of Anonymity

Anonymous Heterogeneous Distributed Detection: Optimal Decision Rules, Error Exponents, and the Price of Anonymity Anonymous Heterogeneous Distributed Detection: Optimal Decision Rules, Error Exponents, and the Price of Anonymity Wei-Ning Chen and I-Hsiang Wang arxiv:805.03554v2 [cs.it] 29 Jul 208 Abstract We explore

More information

Generalized Neyman Pearson optimality of empirical likelihood for testing parameter hypotheses

Generalized Neyman Pearson optimality of empirical likelihood for testing parameter hypotheses Ann Inst Stat Math (2009) 61:773 787 DOI 10.1007/s10463-008-0172-6 Generalized Neyman Pearson optimality of empirical likelihood for testing parameter hypotheses Taisuke Otsu Received: 1 June 2007 / Revised:

More information

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout

More information

Lecture 2: Statistical Decision Theory (Part I)

Lecture 2: Statistical Decision Theory (Part I) Lecture 2: Statistical Decision Theory (Part I) Hao Helen Zhang Hao Helen Zhang Lecture 2: Statistical Decision Theory (Part I) 1 / 35 Outline of This Note Part I: Statistics Decision Theory (from Statistical

More information

Space-Time CUSUM for Distributed Quickest Detection Using Randomly Spaced Sensors Along a Path

Space-Time CUSUM for Distributed Quickest Detection Using Randomly Spaced Sensors Along a Path Space-Time CUSUM for Distributed Quickest Detection Using Randomly Spaced Sensors Along a Path Daniel Egea-Roca, Gonzalo Seco-Granados, José A López-Salcedo, Sunwoo Kim Dpt of Telecommunications and Systems

More information

An Effective Approach to Nonparametric Quickest Detection and Its Decentralized Realization

An Effective Approach to Nonparametric Quickest Detection and Its Decentralized Realization University of Tennessee, Knoxville Trace: Tennessee Research and Creative Exchange Doctoral Dissertations Graduate School 5-2010 An Effective Approach to Nonparametric Quickest Detection and Its Decentralized

More information

Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process

Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University

More information

Universal Outlier Hypothesis Testing

Universal Outlier Hypothesis Testing Universal Outlier Hypothesis Testing Venu Veeravalli ECE Dept & CSL & ITI University of Illinois at Urbana-Champaign http://www.ifp.illinois.edu/~vvv (with Sirin Nitinawarat and Yun Li) NSF Workshop on

More information

Nonparametric Distributed Sequential Detection via Universal Source Coding

Nonparametric Distributed Sequential Detection via Universal Source Coding onparametric Distributed Sequential Detection via Universal Source Coding Jithin K. Sreedharan and Vinod Sharma Department of Electrical Communication Engineering Indian Institute of Science Bangalore

More information

Validation Metrics. Kathryn Maupin. Laura Swiler. June 28, 2017

Validation Metrics. Kathryn Maupin. Laura Swiler. June 28, 2017 Validation Metrics Kathryn Maupin Laura Swiler June 28, 2017 Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC,

More information

Change Detection Algorithms

Change Detection Algorithms 5 Change Detection Algorithms In this chapter, we describe the simplest change detection algorithms. We consider a sequence of independent random variables (y k ) k with a probability density p (y) depending

More information

Dynamic spectrum access with learning for cognitive radio

Dynamic spectrum access with learning for cognitive radio 1 Dynamic spectrum access with learning for cognitive radio Jayakrishnan Unnikrishnan and Venugopal V. Veeravalli Department of Electrical and Computer Engineering, and Coordinated Science Laboratory University

More information

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran

MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING. Kaitlyn Beaudet and Douglas Cochran MULTIPLE-CHANNEL DETECTION IN ACTIVE SENSING Kaitlyn Beaudet and Douglas Cochran School of Electrical, Computer and Energy Engineering Arizona State University, Tempe AZ 85287-576 USA ABSTRACT The problem

More information

Efficient scalable schemes for monitoring a large number of data streams

Efficient scalable schemes for monitoring a large number of data streams Biometrika (2010, 97, 2,pp. 419 433 C 2010 Biometrika Trust Printed in Great Britain doi: 10.1093/biomet/asq010 Advance Access publication 16 April 2010 Efficient scalable schemes for monitoring a large

More information

Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools

Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools Steven X. Ding Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools Springer Notation XIX Part I Introduction, basic concepts and preliminaries 1 Introduction 3 1.1 Basic concepts

More information

Change-point models and performance measures for sequential change detection

Change-point models and performance measures for sequential change detection Change-point models and performance measures for sequential change detection Department of Electrical and Computer Engineering, University of Patras, 26500 Rion, Greece moustaki@upatras.gr George V. Moustakides

More information

Quantifying Stochastic Model Errors via Robust Optimization

Quantifying Stochastic Model Errors via Robust Optimization Quantifying Stochastic Model Errors via Robust Optimization IPAM Workshop on Uncertainty Quantification for Multiscale Stochastic Systems and Applications Jan 19, 2016 Henry Lam Industrial & Operations

More information

The Shiryaev-Roberts Changepoint Detection Procedure in Retrospect - Theory and Practice

The Shiryaev-Roberts Changepoint Detection Procedure in Retrospect - Theory and Practice The Shiryaev-Roberts Changepoint Detection Procedure in Retrospect - Theory and Practice Department of Statistics The Hebrew University of Jerusalem Mount Scopus 91905 Jerusalem, Israel msmp@mscc.huji.ac.il

More information

Quiz 2 Date: Monday, November 21, 2016

Quiz 2 Date: Monday, November 21, 2016 10-704 Information Processing and Learning Fall 2016 Quiz 2 Date: Monday, November 21, 2016 Name: Andrew ID: Department: Guidelines: 1. PLEASE DO NOT TURN THIS PAGE UNTIL INSTRUCTED. 2. Write your name,

More information

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)? ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we

More information

Lessons learned from the theory and practice of. change detection. Introduction. Content. Simulated data - One change (Signal and spectral densities)

Lessons learned from the theory and practice of. change detection. Introduction. Content. Simulated data - One change (Signal and spectral densities) Lessons learned from the theory and practice of change detection Simulated data - One change (Signal and spectral densities) - Michèle Basseville - 4 6 8 4 6 8 IRISA / CNRS, Rennes, France michele.basseville@irisa.fr

More information

9. Model Selection. statistical models. overview of model selection. information criteria. goodness-of-fit measures

9. Model Selection. statistical models. overview of model selection. information criteria. goodness-of-fit measures FE661 - Statistical Methods for Financial Engineering 9. Model Selection Jitkomut Songsiri statistical models overview of model selection information criteria goodness-of-fit measures 9-1 Statistical models

More information

Censoring for Type-Based Multiple Access Scheme in Wireless Sensor Networks

Censoring for Type-Based Multiple Access Scheme in Wireless Sensor Networks Censoring for Type-Based Multiple Access Scheme in Wireless Sensor Networks Mohammed Karmoose Electrical Engineering Department Alexandria University Alexandria 1544, Egypt Email: mhkarmoose@ieeeorg Karim

More information

Decentralized decision making with spatially distributed data

Decentralized decision making with spatially distributed data Decentralized decision making with spatially distributed data XuanLong Nguyen Department of Statistics University of Michigan Acknowledgement: Michael Jordan, Martin Wainwright, Ram Rajagopal, Pravin Varaiya

More information

THRESHOLD LEARNING FROM SAMPLES DRAWN FROM THE NULL HYPOTHESIS FOR THE GENERALIZED LIKELIHOOD RATIO CUSUM TEST

THRESHOLD LEARNING FROM SAMPLES DRAWN FROM THE NULL HYPOTHESIS FOR THE GENERALIZED LIKELIHOOD RATIO CUSUM TEST THRESHOLD LEARNING FROM SAMPLES DRAWN FROM THE NULL HYPOTHESIS FOR THE GENERALIZED LIKELIHOOD RATIO CUSUM TEST C. Hory, A. Kokaram University of Dublin, Trinity College EEE Department College Green, Dublin

More information

Statistical Inference

Statistical Inference Statistical Inference Robert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USA Week 12. Testing and Kullback-Leibler Divergence 1. Likelihood Ratios Let 1, 2, 2,...

More information

Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems

Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Jeremy S. Conner and Dale E. Seborg Department of Chemical Engineering University of California, Santa Barbara, CA

More information

Performance Evaluation and Comparison

Performance Evaluation and Comparison Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Cross Validation and Resampling 3 Interval Estimation

More information

Bandits : optimality in exponential families

Bandits : optimality in exponential families Bandits : optimality in exponential families Odalric-Ambrym Maillard IHES, January 2016 Odalric-Ambrym Maillard Bandits 1 / 40 Introduction 1 Stochastic multi-armed bandits 2 Boundary crossing probabilities

More information

Bayesian Learning (II)

Bayesian Learning (II) Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP

More information

Empirical Likelihood Ratio Test with Distribution Function Constraints

Empirical Likelihood Ratio Test with Distribution Function Constraints PAPER DRAFT Empirical Likelihood Ratio Test with Distribution Function Constraints Yingxi Liu, Student member, IEEE, Ahmed Tewfik, Fellow, IEEE arxiv:65.57v [math.st] 3 Apr 26 Abstract In this work, we

More information

PATTERN RECOGNITION AND MACHINE LEARNING

PATTERN RECOGNITION AND MACHINE LEARNING PATTERN RECOGNITION AND MACHINE LEARNING Chapter 1. Introduction Shuai Huang April 21, 2014 Outline 1 What is Machine Learning? 2 Curve Fitting 3 Probability Theory 4 Model Selection 5 The curse of dimensionality

More information

INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS. Michael A. Lexa and Don H. Johnson

INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS. Michael A. Lexa and Don H. Johnson INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS Michael A. Lexa and Don H. Johnson Rice University Department of Electrical and Computer Engineering Houston, TX 775-892 amlexa@rice.edu,

More information

Distributed Estimation and Detection for Smart Grid

Distributed Estimation and Detection for Smart Grid Distributed Estimation and Detection for Smart Grid Texas A&M University Joint Wor with: S. Kar (CMU), R. Tandon (Princeton), H. V. Poor (Princeton), and J. M. F. Moura (CMU) 1 Distributed Estimation/Detection

More information

Optimal Distributed Detection Strategies for Wireless Sensor Networks

Optimal Distributed Detection Strategies for Wireless Sensor Networks Optimal Distributed Detection Strategies for Wireless Sensor Networks Ke Liu and Akbar M. Sayeed University of Wisconsin-Madison kliu@cae.wisc.edu, akbar@engr.wisc.edu Abstract We study optimal distributed

More information

The information complexity of sequential resource allocation

The information complexity of sequential resource allocation The information complexity of sequential resource allocation Emilie Kaufmann, joint work with Olivier Cappé, Aurélien Garivier and Shivaram Kalyanakrishan SMILE Seminar, ENS, June 8th, 205 Sequential allocation

More information

RATE ANALYSIS FOR DETECTION OF SPARSE MIXTURES

RATE ANALYSIS FOR DETECTION OF SPARSE MIXTURES RATE ANALYSIS FOR DETECTION OF SPARSE MIXTURES Jonathan G. Ligo, George V. Moustakides and Venugopal V. Veeravalli ECE and CSL, University of Illinois at Urbana-Champaign, Urbana, IL 61801 University of

More information

A New Algorithm for Nonparametric Sequential Detection

A New Algorithm for Nonparametric Sequential Detection A New Algorithm for Nonparametric Sequential Detection Shouvik Ganguly, K. R. Sahasranand and Vinod Sharma Department of Electrical Communication Engineering Indian Institute of Science, Bangalore, India

More information

Introduction to Bayesian Statistics

Introduction to Bayesian Statistics Bayesian Parameter Estimation Introduction to Bayesian Statistics Harvey Thornburg Center for Computer Research in Music and Acoustics (CCRMA) Department of Music, Stanford University Stanford, California

More information

Data-Efficient Quickest Change Detection in Minimax Settings

Data-Efficient Quickest Change Detection in Minimax Settings IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 10, OCTOBER 2013 6917 Data-Efficient Quickest Change Detection in Minimax Settings Taposh Banerjee, Student Member, IEEE, and Venugopal V. Veeravalli,

More information

False Discovery Rate Based Distributed Detection in the Presence of Byzantines

False Discovery Rate Based Distributed Detection in the Presence of Byzantines IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS () 1 False Discovery Rate Based Distributed Detection in the Presence of Byzantines Aditya Vempaty*, Student Member, IEEE, Priyadip Ray, Member, IEEE,

More information

EECS564 Estimation, Filtering, and Detection Exam 2 Week of April 20, 2015

EECS564 Estimation, Filtering, and Detection Exam 2 Week of April 20, 2015 EECS564 Estimation, Filtering, and Detection Exam Week of April 0, 015 This is an open book takehome exam. You have 48 hours to complete the exam. All work on the exam should be your own. problems have

More information

ECE531 Lecture 2b: Bayesian Hypothesis Testing

ECE531 Lecture 2b: Bayesian Hypothesis Testing ECE531 Lecture 2b: Bayesian Hypothesis Testing D. Richard Brown III Worcester Polytechnic Institute 29-January-2009 Worcester Polytechnic Institute D. Richard Brown III 29-January-2009 1 / 39 Minimizing

More information

Unsupervised Nonparametric Anomaly Detection: A Kernel Method

Unsupervised Nonparametric Anomaly Detection: A Kernel Method Fifty-second Annual Allerton Conference Allerton House, UIUC, Illinois, USA October - 3, 24 Unsupervised Nonparametric Anomaly Detection: A Kernel Method Shaofeng Zou Yingbin Liang H. Vincent Poor 2 Xinghua

More information

Statistical Measures of Uncertainty in Inverse Problems

Statistical Measures of Uncertainty in Inverse Problems Statistical Measures of Uncertainty in Inverse Problems Workshop on Uncertainty in Inverse Problems Institute for Mathematics and Its Applications Minneapolis, MN 19-26 April 2002 P.B. Stark Department

More information

Change Detection in Multivariate Data

Change Detection in Multivariate Data Change Detection in Multivariate Data Likelihood and Detectability Loss Giacomo Boracchi July, 8 th, 2016 giacomo.boracchi@polimi.it TJ Watson, IBM NY Examples of CD Problems: Anomaly Detection Examples

More information

REPORT DOCUMENTATION PAGE

REPORT DOCUMENTATION PAGE REPORT DOCUMENTATION PAGE Form Approved OMB NO. 0704-088 The public reporting burden for this collection of information is estimated to average hour per response, including the time for reviewing instructions,

More information