Detection and Estimation Chapter 1. Hypothesis Testing

Size: px

Start display at page:

Download "Detection and Estimation Chapter 1. Hypothesis Testing"

Gladys Little
5 years ago
Views:

1 Detection and Estimation Chapter 1. Hypothesis Testing Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, /20

2 Syllabus Homework: 4 problems each week; hand in your homework after one week. 20% Midterm and Final Exames: 50% Project: 30%, choose one topic such as large scale hypothesis testing, quickest detection, robust detection, distributed detection, distributed estimation, large covariance matrix estimation, et al; the topic should be beyond your own research; send me your topic right after the midterm; read the most important books and papers in that topic; apply it in an application you choose by your own; provide a presentation of the survey and your own results before the final. 2/20

3 Textbook The book by H. V. Poor will be the textbook. Other books are for references. 3/20

4 The First Paper on Hypothesis Testing In 1900, K. Pearson published the first paper on hypothesis testing with the long title On the criterion that a given system of deviation from the probable in the case of a correlated system of variables is such that it can be reasonable supposed to have arisen from random sampling. In the paper, he discussed the criterion to judge whether the deviations of a set of samples can be explained by the randomness of sampling. 4/20

5 The First Paper on Hypothesis Testing: Continued Pearson made a mistake in this paper, which was pointed out by an agriculture statistician R. Fisher (Pearson was 65 and Fisher was 32). People in early 1900s considered Fisher as an infant claiming to be taller than his father. 5/20

6 Bayesian Hypothesis Testing We assume that two hypotheses H0 and H1 corresponds to two distributions. The set of observations is denoted by Γ. The goal is to determine whether a set of observations are generated by H0 or H1. A decision rule δ is to divide Γ into Γ0 and Γ1 such that 1, if y Γ1 δ(y ) =. 0, if y Γc1 Cij means the cost incurred by choosing Hi while Hj is true. The conditional risk is defined as Rj (δ) = C1j Pj (Γ1 ) + C0j Pj (Γ0 ), 6/20 j = 0, 1.

7 Bayesian Risk and Decision Rule We assume that H0 and H1 has prior probabilities π0 and π1. Then, the Bayes risk is defined as r (δ) = π0 R0 (δ) + π1 R1 (δ). Thus the optimal decision rule is to minimize the Bayes risk. Theorem: The decision rule minimizing the Bayes risk is to calculate the ) likelihood ratio L(y ) = pp1 (y and make the decision: (y ) 0 δb (y ) = where r is a threshold. 7/20 1, 0, if L(y ) r, if L(y ) < r

8 Bayesian and Frequentist Schools The difference between the Bayesian and Frequentist schools is whether the unknown is fixed or random. In early 20th century, Bayesian statistics were disliked by famous statisticians Neyman and Fisher. But it flourished in late 20th century. The main challenge of the Bayesian school is how to determine the prior distribution. Empirical Bayesian method is kind of a bridge between the two schools. 8/20

9 MiniMax Hypothesis Testing The Bayesian risk based testing requires the prior distributions of the hypotheses. What if we do not know them? If the prior distributions are not known, then we should minimize the worst case, i.e., the maximum risk: δ = arg min max π0 R0 (δ) + (1 π0 )R1 (δ). δ π0 Again, we can prove that the likelihood ratio based testing is optimal for the MiniMax hypotheses testing. 9/20

10 Illustration of the MiniMax Rule 10/20

11 Randomized Decision Rule When if V is not differentiable at the least favorable prior probability πl. 11/20

12 Neyman-Pearson Hypothesis Testing Sometimes, the costs are defined directly on the error probabilities. Type I error (false alarm): H0 is falsely rejected, probability PF. Type II error (missed detection): H1 is falsely rejected, probability PM. Detection probability PD = 1 PM. Neyman-Pearson design criterion: max PD (δ), δ 12/20 s.t. PF (δ) α.

13 Neyman-Pearson Lemma Theorem: Consider the hypothesis with densities p1 and p0. Assume α > 0. Then, the following statements are true: Let δ be any decision rule satisfying PF (δ ) α and δ 0 be any decision rule of the form p1 (y ) > ηp0 (y) 1, 0 γ(y), p1 (y ) = ηp0 (y), (1) δ (y) = 0, p1 (y ) < ηp0 (y) where η and γ(y ) are such that PF (δ 0 ) = α. Then, PD (δ 0 ) PD (δ ). For every α (0, 1), there exists a decision rule δ NP satisfying (1) with γ(y ) = γ0 and PF (δ NP ) = α. Suppose that δ 00 is any α-level Neyman-Pearson decision rule. Then, it must be of the form (1) except possibly on a subset of Γ of zero probability measure. 13/20

Some Gossips Neyman (in Poland) and Pearson (in UK) collaborated between 1926 and 1936.

14 Some Gossips Neyman (in Poland) and Pearson (in UK) collaborated between 1926 and Their paper (98 pages) on the likelihood ratio test was published in But their collaboration did not survive... 14/20

15 Composite Hypothesis Testing In simple hypothesis testing, each hypothesis corresponds to only one distribution. In composite hypothesis testing, each hypothesis may correspond to multiple distributions. Usually we consider a family of distributions indexed by a parameter θ. In the Bayesian setup, we assume θ is random and the corresponding prior probability is known. 15/20

16 Composite Hypothesis Testing: Decomposition In many situations, the parameter space can be decomposed to two disjoint sets Λ0 and Λ1, representing H0 and H1, respectively. The decision rule is still based on likelihood ratio test, where the likelihood is calculated by Z p(y Θ Λj ) = pθ (y)w(θ)dθ. Λj 16/20

17 Example 1 17/20

18 UMP Test For composite hypothesis testing problems in which we do not have a prior distribution for the parameter, the development of hypothesis testing that satisfy precise analytical definitions of optimality is very often an illusive task. We can generalize the Neyman-Pearson test by defining the false alarm and detection probabilities to h i h i PF (δ, θ) = Eθ δ (Y ), θ Λ0 PF (δ, θ) = Eθ δ (Y ), θ Λ1. The ideal case is that the test maximizes PD for every θ while satisfying PF α for every θ. This is called the uniformly most powerful (UMP) test. 18/20

19 Example 2 19/20

20 What If UMP Test Does Not Exist? In many situations, UMP test does not exist. This can be overcome by applying other constraints to eliminate unreasonable tests from consideration. One such condition is unbiasedness. We can try to optimize the derivative of PD (θ0 ) for the locally optimal test. Or we can try generalized likelihood ratio test: maxθ Λ1 pθ (y). maxθ Λ0 pθ (y) 20/20

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis