Problem Set 4 Due Oct, 12
|
|
- Cory Banks
- 5 years ago
- Views:
Transcription
1 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios of estimatio theory. Not all exercises are to be tured i. Oly those with the sig are due o Thursday, October 12 th at the begiig of the class. Although the remaiig exercises are ot graded, you are ecouraged to go through them. We will discuss some of the exercises durig discussio sectios. Please feel free to poit out errors ad otios that eed to be clarified. Notes: I this problem set, uless otherwise stated, we will cosider simple biary hypothesis testig. There are two possible hypotheses H 0 ad H 1. Basically, we wish to test the assumptio H 0 is true agaist H 1 is true. By probability of Type I (or false alarm) error, we mea the probability of acceptig hypothesis H 1 (or rejectig H 0 ) while H 0 is true. Type II (or miss detectio) probability is the probability of acceptig H 0 while H 1 is true. (Thik of a radar system where H 0 is the hypothesis that there is ot target ad H 1 the hypothesis that a target is preset.) Exercise 4.1. I discussio sectio, we have see ( ot really!) that i the Neyma-Pearso test, the probability of Type I ad Type II errors (resp. deoted (α) ad (β)) caot be made both small at the same time. Maily, reducig oe type of error will result to icreasig the other. We have also claimed that oe way to reduce both type of errors was to icrease the the umber of observatios. I this exercise we will verify those claims. Let (X 1,..., X ) be idepedet observatios (or realizatios) of a ormal radom variable with mea µ ad variace σ 2 = 100. We would like to test the hypothesis H 0 : µ = 50 agaist H 1 : µ = µ 1 > 50 for a observatio size = 25. A) Error calculatio: We decide to reject H 0 is X 52 where X = X 1+ +X is the sample mea. 1. Fid the probability of rejectig H 0 as a fuctio of µ. 2. Compute the probability of Type I error (α). 3. Fid the probability of Type II error for (i)µ 1 = 53 ad for (ii) µ 1 = 55. B) Now we cosider that the decisio rule is such that we reject H 0 if X c. 1. Fid the value of c such that the probability of Type I error is α =
2 2. Fid the probability of Type II error β with the ew decisio rule whe µ 1 = Compare the values you get for α ad β to the oes obtaied i part A. C) Repeat part B) with a sample size = 100 ad coclude. A) Error calculatio: (1) Note the the sample mea X is ormal N (µ, σ2 ). Thus the probability of rejectig H 0 is give by P (N (µ, σ2 52 µ ) 52) = P (N (0, 1) σ2 / ) = Q( 52 µ ), µ 50 σ2 / (2) Type I error is the probability of rejectig H 0 give µ = µ 0 = 50. Replacig µ by 50 i the result i part (1) gives: α = Q( ) = /25 (3) Type II error is the probability of acceptig H 0 give that µ = µ 1 > µ 0. It is give by P ( X 52). Similar to part (1) we ca express it as a fuctio of µ ad it is: P (N (µ 1, σ2 ) 52) = P (N (0, 1) 52 µ 1 σ2 / ) = Q( µ 1 52 σ2 / ), usig properties of Q-fuctio For µ 1 = 53 we obtai β = ad for β = µ 1 = 55 we obtai Notice that clearly the probability of Type II error depeds o µ 1. B)-(1) Usig the result i (2) of part A, we are lookig for c such that α = P ( X c µ = µ 0 ) = 0.05 Agai give that µ = µ 0 = 50, X N (µ, σ 2 ). Thus we wat c such that Q( c 50 σ2 / ) = c c = σ 2 /Q 1 (0.05) + 50 Lookig at the table we have c = (2) Usig part (A)-(3) with the modified decisio rule, we obtai β = Q( µ σ2 / ) 4-2
3 With µ 1 = 55, we obtai β = (3) Comparig with the results of part (A), we otice that with the chage of the decisio rule, α is reduced from to 0.05, but β is icreased from to (C) We will just use the results i part (B) with = 100. (1) c = σ 2 /Q 1 (0.05) + 50, σ 2 = 100, = 100 Usig tables, we obtai c = (2) Settig µ 1 = 55, we obtai β = Q( µ σ2 / ) = (3)Notice that with sample size = 100, both α ad β have decreased from their respective origial values of ad whe = 25. Exercise 4.2. ML, MAP, ad Neyma-Pearso We cosider a biary commuicatio system where at each time oe of the two symbols s = 0 or s = 1 is trasmitted. Our two hypotheses are H 0 : s = 0 was trasmitted H 1 : s = 1 was trasmitted The commuicatio chael adds oise N (0, 1) ad the received sigal is x = s +. At some time istat, we observe that x = 0.6. (a) Usig the maximum likelihood test, determie which sigal was set. Compute α. (b) Suppose ow that the prior probability of s is give by P (s = 0) = 2 3. Repeat part (a) usig the MAP test. (c) Now suppose that we require that α = Usig Neyma-Pearso test, determie which sigal was set. What is the correspodig β. Hit (1) Because of the symmetry of the ML detectio, the threshold of the decisio is 1/2. Sice x = 0.6, we will decide s = 1. (2) Writig the MAP rule, we see that the threshold is ow equal to > x ad we will decide s = 0. (3) Derivig the NP test we fid that the threshold is equal to > x ad hece we will decide s = 0. Try to compute the correspodig errors. Exercise 4.3. : Bayes Test I the Bayes Test, each evet (D i, H j ) has a associated cost C ij, where (D i, H j ) is the evet that hypothesis H i is accepted while H j is true (for i, j = 0, 1). The average cost or Bayes risk is defied as C = C 00 P (D 0 ; H 0 ) + C 01 P (D 0 ; H 1 ) + C 10 P (D 1 ; H 0 ) + C 11 P (D 1 ; H 1 ) 4-3
4 The test that miimizes the average cost is called the Bayes test ad ca be expressed i terms of the likelihood ratio Λ(X) H 1 η = (C 10 C 00 )P (H 0 ) H 0 (C 01 C 11 )P (H 1 ) Now cosider a biary decisio problem with the followig coditioal pdf s: The costs are give as follows: f(x H 0 ) = 1 2 exp{ x } f(x H 1 ) = exp{ 2 x } C 00 = C 11 = 0 C 01 = 2 C 10 = 1 (a) Determie the Bayes test if P (H 0 ) = 2 3 ad the associated Bayes risk. (b) Repeat part (a) for P (H 0 ) = 1 2 (a) The likelihood ratio is give by: The Bayes test is give by Λ(x) = f(x H 1) 2e 2 x f(x H 1 ) = e x 2e x H 1 H 0 (1 0) 2 3 (2 0) 1 3 Takig the logarithm is both sides we obtai: = 2e x The associated Bayes risk is x H 0 H 1 l( 1 2 ) = ad P (D 0 H 1 ) = givig C = 0.5. (b) For P (H 0 ) = 1 2 C = 2P (D 0 H 1 )P (H 1 ) + P (D 1 H 0 )P (H 0 ) e 2x dx + P (D 1 H 0 ) = e 2x dx = 2 e x dx = e 2x dx = 0.25 e x dx = 0.5 we have equal priors ad the log-likelihood ratio is equal to x H 0 H 1 l( 1 4 ) =
5 Followig the same steps as i part (a), we obtai P (D 0 ; H 1 ) = 2 P (D 1 ; H 0 ) = e 2x dx = e x dx = 0.75 C = 1 ( ) = Exercise 4.4. The coditioal pmf of Y give X is give by the table below. X Y Fid the the estimate of ˆX based o Y that miimizes P ( ˆX = 1 X = 1) subject to the costrait P ( ˆX = 1 X = 1) β, for β (0, 1). Hit See solutio i 126 otes exercise Exercise 4.5. We observe (Y 1,..., Y ), idepedet realizatios of the same radom variable Y. Y depeds o a parameter θ i the followig way: whe θ = 0, Y is uiformly distributed i [ 1, 1]; whe θ = 1 Y is uiform i [0, 2]. Suppose that we would like to guess the value of θ based o our observatios. (a) Fid a sufficiet statistic. (b) We do ot kow the prior distributio of θ, but whe θ = 0, we would like to detect it correctly at least 95% of the time. Give this costrait, fid the test that miimizes the error probability whe θ = 1. (a) Let Y mi = mi(y 1,..., Y ) ad Y max = max(y 1,..., Y ). We ca easily argue that (Y mi, Y max ) is a sufficiet statistic. I fact if Y mi < 0 the we kow that the Y i s must have come from the U( 1, 1) radom variable. Similarly if Y max > 1 the we kow that the Y i s must have come from the U(0, 2) radom variable. If 0 Y mi Y max 1 the the Y i s are uiformly distributed i (Y mi, Y max ) whether θ = 0 or θ = 1. (b) I this part we wat to miimize a Type II error give some costrait. This is solved by the NP test. We ve already give the decisio rule if Y mi < 0 or Y max > 1. If 0 Y mi Y max 1 we decide θ = 0 with probability γ ad γ is chose such that the Type I is equal to α. 4-5
6 Give that θ = 0 there is a error oly if 0 Y i 1, i = 1,..., ad we choose ˆθ = 1. This happes with probability P (error θ = 0) = (1 γ)p (Y i [0, 1], i = 1,..., θ = 0) = (1 γ) P (Y i [0, 1] θ = 0) = (1 γ)( 1 2 ) We wat this error probability to be at most equal to α. (1 γ)( 1 2 ) = α γ = max(0, 1 2 α) where we have take the maximum because γ must be i [0, 1]. Note that if 1 2 α < 0 ( > log 2 (1/α)), the we always choose ˆθ = 1 wheever 0 Y mi Y max 1. Ituitively, this meas that if we have eough observatios (such that costrait for Type I error is met: P (error θ = 0) = 2 α), we always choose ˆθ = 1 i the ucertaity regio. The correspodig Type error is zero which is miimal. Exercise 4.6. We cosider a biary detectio problem where we observe (X 1,..., X ) idepedet realizatios of a ormal radom variable with mea µ ad variace σ 2. We would like to test the hypothesis H 0 : µ = µ 0 agaist the H 1 : µ > µ o. Suppose that the ull hypothesis is false ad that the true value of the mea is µ = µ o + δ, for δ > 0. What is the miimum umber of observatios if we wat the Type II probability of error to be at most β? We cosider that the Type I error probability is equal to α. We will use the NP test to fid the threshold first ad after we will compute the correspodig β. Note that the sample mea X = X 1+X 2 + +X is a sufficiet statistic for the detectio problem ad it has a ormal N (µ 0, σ 2 /) give H 0. The decisio rule will be to choose H 0 if X c ad choose H 1 otherwise; for some threshold c. The probability of error give H = H 0 is P r(error H 0 ) = α = P (N (µ 0, σ 2 /) > c) = P (N (0, 1) > c µ 0 σ2 / ) = Q( c µ 0 σ2 / ) Thus we obtai c = σ 2 /Q 1 (α) + µ 0 4-6
7 Now we write the Type II probability of error: P (error H 1 ) = P (N (µ 0 + δ, σ 2 /) < c) = P (N (0, 1) < c µ 0 δ σ2 / ) = P (N (0, 1) < = P (N (0, 1) < σ2 /Q 1 (α) + µ 0 µ 0 δ ) σ2 / σ2 /Q 1 (α) δ ) σ2 / = 1 P (N (0, 1) > σ2 /Q 1 (α) δ ) σ2 / = 1 Q( = Q( σ2 /Q 1 (α) δ ) σ2 / σ2 /Q 1 (α) δ ) β σ2 / Sice Q is a strictly decreasig fuctio the miimum for which the last equatio is satisfied verifies: Q 1 (α) + δ σ2 / Q 1 (β) σ δ2 (Q 1 (α) + Q 1 (β)) 2 σ δ 2 (Q 1 (α) + Q 1 (β)) 2 Exercise 4.7. Miimum Probability of Error Cosider a Bayes test with the followig costs C 00 = C 11 = 0 C 01 = 1 C 10 = 1 What is the average cost? (you should recogize this cost). What is the Bayes test? (this is a test that you already kow) Hit It should ot be difficult to see that this correspods to the MAP test. Exercise 4.8. Let X ad Y be two idepedet radom variables with the same distributio U(0, 1). Fid the MMSE of X 3 give X + Y. 4-7
8 We kow that the MMSE is give by E[X 3 X + Y ]. So we oly eed to compute the distributio of X give X + Y = z. Cosiderig the cases where 0 z < 1 ad 2 z 1 we obtai { 1 f X X+Y (x z) = 1 z [0 x 1] if 0 z < z [z 1 x 1] if 2 z 1 Now we oly eed to compute E[X 3 X + Y ] ad see that it is give by: { 1 E[X 3 4 X + Y ] = (x + y)3 if 0 z < 1 1 [1 (x + 4(2 x y) y)4 ] if 2 z 1 Exercise 4.9. Show that the mea square estimatio error i the LLSE case is greater or equal to the mea square estimatio error i the MMSE case. Note: First argue that this is true (usig what you have leared so far) ad the use exercise 4.5 (Gallager) for the proof (for bous). MMSE is computig the estimate that miimizes the error over all fuctios of the observatios. LLSE just cosiders the set of liear fuctios that is a subset of the set all fuctios. Sice if A B the mi B mi A we have that the LLSE estimatio error is larger tha the MMSE estimator. It is iterestig to compute the variace of the estimates. Try this exercise ad you will fid that MMSE estimator has a larger variace. Ask yourself why. By goig through the steps of Gallager 4.5 we see that E[e 2 llse] = E[e 2 mmse] + E[( ˆX mmse ˆX llse ) 2 ] which shows that the MMSE estimatio error is less tha the LLSE estimatio error. Exercise : (U)Biased estimator: Let (X 1,..., X 7 ) deote a radom sample from a populatio havig mea µ ad variace σ 2. Cosider the followig estimators of µ: ˆΘ = X X 7 7 Θ = 2X 1 X 6 + X 4 2 (a) Is either estimator ubiased? (b) Which estimator is best? I what sese is it best? (c) Show that the estimator of σ 2 defied by S 2 = 1 (X i X) 2 is biased (where X is the sample mea). 4-8
9 (a) Both estimators are ubiased because they have mea µ. (b) The first estimator has less variace...i this sese it is better tha the secod. (c) It is well kow that the ubiased estimator of the variace is S 2 = 1 1 Try to prove it if your are still ot coviced. (X i X) 2 Exercise Defie (X 1,..., X ) as observed values of iid B(m, p), where m is assumed to be kow ad p ukow. Determie the maximum-likelihood estimator of p. Assume that the X i s are idepedet. We wat to fid ˆp give by ˆp = armgax p (P (X 1 = x 1,..., X = x ; p)) [ ] = armgax p P (X i = x i ; p) (idepedece) [ ( ) ] m = armgax p p x i (1 p) m x i = armgax p [ x 1 p P x i (1 p) m P x i ( ) ] m [ = armgax p p P x i (1 p) m P ] x i (bcse terms i productio are costat) ( = armgax p [log p P x i (1 p) m P )] x i (bcse log is icreasig fuctio) ] = armgax p [( x i ) log(p) + (m x i ) log(1 p) Now takig the derivative of ( x i) log(p) + (m x i) log(1 p) with respect to p ad settig it equal to zero, we obtai Exercise Gallager s ote: Exercise 3.10 x 1 ˆp = x 1 + x x m 4-9
10 (a) The LLR is give by: Λ(Y ) = log The MAP decisio rule chooses H = H 1 for ( ) fy H (y 1) f Y H (y 0) (y 1)2 = = 2σ 2 + 8y σ 2 Λ(Y ) log( p 0 ) p 1 8y + 24 log( p 0 ) 2σ 2 p 1 y 3 σ2 4 log(p 0 p 1 ) (y 5)2 2σ 2 Similarly we choose H = H 0 if y Θ where Θ = 3 σ2 log( p 0 4 p 1 ). The probability of error give H = H 0 is give by: Similar computatio gives that: P r(e H = H 0 ) = P (y Θ H = H 0 ) = P (N (5, σ 2 ) Θ) = P (N (0, 1) Θ 5 σ ) = 1 P (N (0, 1) > Θ 5 σ ) = 1 Q( Θ 5 σ ) = Q( 5 Θ σ ) P r(e H = H 1 ) = P (y > Θ H = H 1 ) = P (N (1, σ 2 ) > Θ) = Q( Θ 1 σ ) (b) The secod observatio Y 2 ca be writte as Y 2 = Y 1 + Z 2 where Z 2 is idepedet to both H ad Z 1. Writig the log-likelihood ratio of the pair (Y 1, Y 2 ) we will see that it is simply equal to the oe computed i (a). Thus the decisio rule ad the estimator errors will be the same as the 4-10
11 oes i part (a). (This secod observatio will ot help to detect H sice it is idepedet to H, ad it wo t help reduce the oise Z 1 either). (c) Addig a oisy versio of the observatio will ot improve our estimatio. I fact Y 1 is sufficiet to estimate H. (d) Note that i our discussio i part (b) the distributio of Z 2 was ot relevat. Usig the same argumets as i (b) we see that the decisio rule ad estimatio errors will be the same as i part (a). (e) Now if Z 1 U(0, 1), the give H = H 0, Y U(5, 6) ad give H = H 1 Y U(1, 2). Sice these two itervals do ot overlap, we ca always detect H without error. Thus the decisio rule will be { H0 if y [5, 6] H = H 0 if y [1, 2] ad the probability of error will be zero. 4-11
Lecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationTable 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab
Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationLecture 7: October 18, 2017
Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationTopic 18: Composite Hypotheses
Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationComposite Hypotheses
Composite Hypotheses March 25-27, 28 For a composite hypothesis, the parameter space Θ is divided ito two disjoit regios, Θ ad Θ 1. The test is writte H : Θ versus H 1 : Θ 1 with H is called the ull hypothesis
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationLecture Notes 15 Hypothesis Testing (Chapter 10)
1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationChapter 13: Tests of Hypothesis Section 13.1 Introduction
Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More informationf(x i ; ) L(x; p) = i=1 To estimate the value of that maximizes L or equivalently ln L we will set =0, for i =1, 2,...,m p x i (1 p) 1 x i i=1
Parameter Estimatio Samples from a probability distributio F () are: [,,..., ] T.Theprobabilitydistributio has a parameter vector [,,..., m ] T. Estimator: Statistic used to estimate ukow. Estimate: Observed
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationIntroduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 3
Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 3 (This versio August 17, 014) 015 Pearso Educatio, Ic. Stock/Watso
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationPower and Type II Error
Statistical Methods I (EXST 7005) Page 57 Power ad Type II Error Sice we do't actually kow the value of the true mea (or we would't be hypothesizig somethig else), we caot kow i practice the type II error
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationSTA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:
STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio
More informationKurskod: TAMS11 Provkod: TENB 21 March 2015, 14:00-18:00. English Version (no Swedish Version)
Kurskod: TAMS Provkod: TENB 2 March 205, 4:00-8:00 Examier: Xiagfeg Yag (Tel: 070 2234765). Please aswer i ENGLISH if you ca. a. You are allowed to use: a calculator; formel -och tabellsamlig i matematisk
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationMOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.
XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationApril 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE
April 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE TERRY SOO Abstract These otes are adapted from whe I taught Math 526 ad meat to give a quick itroductio to cofidece
More informationChapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationLast Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationMath 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationtests 17.1 Simple versus compound
PAS204: Lecture 17. tests UMP ad asymtotic I this lecture, we will idetify UMP tests, wherever they exist, for comarig a simle ull hyothesis with a comoud alterative. We also look at costructig tests based
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationCSE 527, Additional notes on MLE & EM
CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be
More informationNYU Center for Data Science: DS-GA 1003 Machine Learning and Computational Statistics (Spring 2018)
NYU Ceter for Data Sciece: DS-GA 003 Machie Learig ad Computatioal Statistics (Sprig 208) Brett Berstei, David Roseberg, Be Jakubowski Jauary 20, 208 Istructios: Followig most lab ad lecture sectios, we
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationData Analysis and Statistical Methods Statistics 651
Data Aalysis ad Statistical Methods Statistics 651 http://www.stat.tamu.edu/~suhasii/teachig.html Suhasii Subba Rao Review of testig: Example The admistrator of a ursig home wats to do a time ad motio
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationA quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population
A quick activity - Cetral Limit Theorem ad Proportios Lecture 21: Testig Proportios Statistics 10 Coli Rudel Flip a coi 30 times this is goig to get loud! Record the umber of heads you obtaied ad calculate
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationMath 140 Introductory Statistics
8.2 Testig a Proportio Math 1 Itroductory Statistics Professor B. Abrego Lecture 15 Sectios 8.2 People ofte make decisios with data by comparig the results from a sample to some predetermied stadard. These
More informationChapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationEmpirical Process Theory and Oracle Inequalities
Stat 928: Statistical Learig Theory Lecture: 10 Empirical Process Theory ad Oracle Iequalities Istructor: Sham Kakade 1 Risk vs Risk See Lecture 0 for a discussio o termiology. 2 The Uio Boud / Boferoi
More informationLecture 9: Hierarchy Theorems
IAS/PCMI Summer Sessio 2000 Clay Mathematics Udergraduate Program Basic Course o Computatioal Complexity Lecture 9: Hierarchy Theorems David Mix Barrigto ad Alexis Maciel July 27, 2000 Most of this lecture
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More information6 Sample Size Calculations
6 Sample Size Calculatios Oe of the major resposibilities of a cliical trial statisticia is to aid the ivestigators i determiig the sample size required to coduct a study The most commo procedure for determiig
More information2. The volume of the solid of revolution generated by revolving the area bounded by the
IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More information