Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the exam. Please justify all your steps carefully. Otherwise o poits will be give. Grade Table (for gradig use oly, please leave empty) Questio Poits Score 0 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 0 0 Total: 00
Probability ad Statistics Secod Sessio Exam - Page of 9 09.0.08 Iformatios. Read this carefully. Please justify all your statemets carefully. Explai the steps of your reasoig. Otherwise o poits will be give. You are expected to write full seteces whe givig your aswer. DO NOT WRITE with red or gree pes. DO NOT WRITE with a pecil. Your aswers should be readable. Write your ame o all the sheets you ited to had i before the ed of the exam. GOOD LUCK
Probability ad Statistics Secod Sessio Exam - Page 3 of 9 09.0.08. (0 poits) Coditioal Probabilities (a) (4 poits) Let (Ω, A, P) be a probability space ad A, B A with P(A) = 0.6 ad P(B) = 0.7. Show that P(B A) 0.5. (b) (6 poits) Suppose we are give three differet dice. The first die is fair, i.e. the probability to obtai a six is. The secod ad the third die are biased. The 6 probability to obtai a six with the secod die is ad the probability to obtai a six with the third die is. Suppose that oe of the three dice is chose at radom ad tossed. Moreover, suppose that we get a six. Calculate the probability that the fair die was chose. Solutio (a) We calculate which leads to Therefore we get P(A c B c ) P(A c ) + P(B c ) = 0.4 + 0.3 = 0.7, P(A B) = P(A c B c ) 0.7 = 0.3. P(B A) = P(A B) P(A) 0.3 0.6 = 0.5. (b) We defie the followig four evets: A := first die was chose, B := secod die was chose, C := third die was chose, 6 := we obtaied a six, i.e. we have to calculate P(A 6 ). We kow from the problem formulatio that P(A) = P(B) = P(C) = 3 ad that P( 6 A) = 6, P( 6 B) = ad P( 6 C) =. We apply Bayes Theorem to get P( 6 A) P(A) P(A 6 ) = P( 6 A) P(A) + P( 6 B) P(B) + P( 6 C) P(C) = = 6 3 + 6 3 6 + + 6 = 0. + 3 3
Probability ad Statistics Secod Sessio Exam - Page 4 of 9 09.0.08. (0 poits) Desity, Expectatio, Variace ad Covariace Let X be a real-valued radom variable with desity { c(4 x f(x) = ) if 0 x, 0 else, for some costat c > 0. (a) ( poits) Calculate c. (b) ( poits) Calculate E[X]. (c) ( poits) Calculate Var(X). Let Y be aother real-valued radom variable with E[Y ] = 5 0, Var(Y ) = ad 4 80 Cov(X, Y ) =. 4 (d) ( poits) Calculate E[XY ]. (e) ( poits) Calculate Var(X + Y ). Solutio (a) We calculate f(x) dx = 0 ( c(4 x ) dx = c 8 8 ) = c 6 3 3. Sice f is the desity of a radom variable, it has to itegrate to. Hece, we get (b) To get E[X], we compute E[X] = xf(x) dx = (c) I order to calculate Var(X), we eed E[X ] = Hece, we get (d) We calculate (e) We calculate x f(x) dx = 0 0 c = 3 6. x 3 6 (4 x ) dx = 3 6 x 3 6 (4 x ) dx = 3 6 Var(X) = E[X ] E[X] = 4 5 ( 3 4 0 0 4x x 3 dx = 3 6 (8 4) = 3 4. 4x x 4 dx = 3 6 ) = 64 80 45 80 = 9 80. E[XY ] = Cov(X, Y ) + E[X]E[Y ] = 4 + 3 4 5 4 = 9 6. Var(X + Y ) = Var(X) + Var(Y ) + Cov(X, Y ) = 9 80 + 0 80 + 4 =. ( 3 3 3 ) = 4 5 5.
Probability ad Statistics Secod Sessio Exam - Page 5 of 9 09.0.08 3. (0 poits) Chebyshev Iequality (a) (4 poits) Let g : R [0, ) be a icreasig fuctio. Let c R such that g(c) > 0. Let X be a real-valued radom variable. Show that P(X c) E[g(X)]. g(c) Note that you ca NOT use that X has a probability mass fuctio or a desity. Hit: Use that the probability of a evet ca be writte as the expectatio of the idicator fuctio of this evet. (b) ( poits) Let X be a real-valued radom variable ad c > 0 a costat. result i (a) to show that P( X E[X] c) Var(X) c. Use the Now let X,..., X 50 be idepedet, idetically Poisso distributed radom variables with parameter λ = ad defie S = 50 50 i= X i. (c) (4 poits) Usig the result i (b), give the smallest possible value of c > 0 such that Solutio P( S E[S] c) 0.0. (a) Sice g is positive, icreasig ad g(c) > 0, we get {X c} g(x) g(c) almost surely. Hece we ca coclude P(X c) = E[ {X c} ] E [ ] g(x) g(c) = E[g(X)]. g(c) (b) We defie Y := X E[X] ad g(x) := x as a fuctio from [0, ) to [0, ). The g is a icreasig fuctio with g(x) > 0 for all x > 0 ad Hece we get E[g(Y )] = E[(X E[X]) )] = Var(X). P( X E[X] c) = P(Y c) E[g(Y )] g(c) = Var(X) c, where for the iequality we used the result i (a).
Probability ad Statistics Secod Sessio Exam - Page 6 of 9 09.0.08 (c) Let i {,..., 50}. Sice X i Poi(), we have Var(X i ) =. Moreover, sice X,..., X 50 are idepedet, we have ( ) 50 ( ) 50 Var(S) = Var X i = Var(X i ) = 50 50 50 = 5. Usig the result i (b), we get i= i= P( S E[S] c) Var(S) c = 5c. Now 5c 0.0 c 4, from which we ca coclude that c =.
Probability ad Statistics Secod Sessio Exam - Page 7 of 9 09.0.08 4. (0 poits) Covergece i Probability ad Almost Sure Covergece (a) (3 poits) Let (X ) be idepedet, Beroulli distributed radom variables with P(X = ) = ad P(X = 0) =, for all. Show that X coverges to 0 i probability, as. (b) (7 poits) Let (Y ) be idepedet, idetically uiformly distributed radom variables o [0, ]. Defie M := mi{y,..., Y } for all. Show that M coverges to 0 almost surely, as. Hit: Use the first Borel-Catelli Lemma to show that for a arbitrary y (0, ) we have lim M y almost surely. Solutio (a) For ɛ, we get For 0 < ɛ <, we get P( X 0 > ɛ) = P(X > ɛ) = 0, for all. P( X 0 > ɛ) = P(X > ɛ) = P(X = ) = 0. (b) Let y (0, ) ad. The, usig the i.i.d.-property of Y,..., Y ad the defiitio of the distributio fuctio of a uiform distributio o [0, ], we calculate P(M > y) = P(mi{Y,..., Y } > y) = P(Y > y,..., Y > y) = = P(Y > y) = ( y). P(Y i > y) i= Usig this result, we get P(M > y) = = ( y) = = y ( y) = y <. Due to the first Borel-Catelli Lemma, we get P(M > y for ifiitely may ) = 0. This implies that lim M y almost surely. Sice y was chose arbitrarily i (0, ), we ca coclude that M coverges to 0 almost surely, as.
Probability ad Statistics Secod Sessio Exam - Page 8 of 9 09.0.08 5. (0 poits) Radom Vector ad Coditioal Distributio Let X ad Y be two idepedet, expoetially distributed radom variables with parameter λ =. Let us defie T = X + Y. (a) ( poit) What is the joit desity of the radom vector (X, Y )? (b) (5 poits) Calculate the joit desity of the radom vector (X, T ). (c) ( poits) Use the result obtaied i (b) to calculate the margial desity of T. (d) ( poits) Calculate the coditioal desity of X give T =. Solutio (a) Sice X ad Y both have a Expoetial()-distributio, their correspodig desities f X ad f Y are f X (x) = f Y (x) = e x {x 0}. Moreover, sice X ad Y are idepedet, the joit desity f X,Y vector (X, Y ) is give by (b) We defie The ( ) X = B X + Y f X,Y (x, y) = f X (x)f Y (y) = e (x+y) {x,y 0}. B := ( ) X, B = Y ( ) 0. ( ) 0 ad det(b) =. of the radom By the Trasformatio Theorem, we get for the joit desity f X,T of the radom vector (X, T ) ( ( )) x f X,T (x, t) = det(b) f X,Y B t = f X,Y (x, t x) = e (x+t x) {x,t x 0} = e t {t x 0}. (c) We calculate the margial desity f T of T usig the joit desity f X,T foud i (b): f T (t) = f X,T (x, t) dx = e t {t x 0} dx = t 0 e t {t 0} dx = t e t {t 0}. (d) We calculate the coditioal desity f X T = of X T = usig the joit desity f X,T foud i (b) ad the margial desity f T foud i (c): f X T = (x ) = f X,T (x, ) f T () = e { x 0} e = { x 0}.
Probability ad Statistics Secod Sessio Exam - Page 9 of 9 09.0.08 6. (0 poits) Joit Distributio ad Jese s Iequality Peter has two possibilities to go to work: Either he ca walk the short distace or he ca take the bus. The bus stop is o his way to the office. Every morig Peter arrives at a radom time betwee 7:0 ad 7:0 at the bus stop. Likewise, also the bus arrives at the stop at a radom time betwee 7:0 ad 7:0. Let X Ui([0, 0]) model the arrival time of Peter at the bus stop i miutes after 7:00 ad Y Ui([0, 0]) model the arrival time of the bus at the bus stop i miutes after 7:00. Moreover, we assume that the arrival time of Peter ad the arrival time of the bus at the bus stop are idepedet, i.e. that X ad Y are idepedet. (a) (5 poits) Suppose that whe Peter arrives at the bus stop, he waits for at most five miutes: If the bus arrives withi these five miutes, he takes the bus, otherwise he walks to the office. Calculate the probability that Peter takes the bus. (b) (5 poits) Show that E [ ] X Y > without calculatig it. Ca we coclude that, o average, Peter arrives at the bus stop later tha the bus? Solutio (a) Sice X ad Y are idepedet, uiformly o [0, 0] distributed radom variables, the joit desity f X,Y of (X, Y ) is give by f X,Y (x, y) = 0 {x [0,0]} 0 {y [0,0]} = 00 {x,y [0,0]}. Peter takes the bus if ad oly if X Y X + 5. Thus, we calculate P( Peter takes the bus ) = P(X Y X + 5) = = 0 x+5 0 x 5 x+5 0 = 00 x 5 0 = 5 00 + 00 = 50 00 + 5 00 = 3 8. f X,Y (x, y) dydx 00 dydx + 0 0 5 x 0 00 dydx (x + 5 x) dx + (0 x) dx 00 5 [00 ] (400 5) (b) Note that Y is o-determiistic, takes values i [0, 0] ad that the fuctio x /x is strictly covex o [0, 0]. Hece we ca apply Jese s iequality to get E [ ] [ ] X = E[X] E > E[X] Y Y E[Y ] =,
Probability ad Statistics Secod Sessio Exam - Page 0 of 9 09.0.08 where we used i the first equality that X ad Y are idepedet ad i the secod equality that X ad Y have the same distributio. We ca ot coclude from this iequality that, o average, Peter arrives at the bus stop later tha the bus. Sice X ad Y have the same distributio, we also have E[X] = E[Y ].
Probability ad Statistics Secod Sessio Exam - Page of 9 09.0.08 7. (0 poits) Posterior Distributio Let X be a ormally distributed radom variable with mea θ, which is ukow, ad variace equal to. Assume that we have a prior distributio for θ, which is the stadard ormal distributio. Moreover, suppose that i a experimet we observe that X = x. (a) (7 poits) Show that the posterior distributio of θ give X = x is the ormal distributio with mea x ad variace. (b) (3 poits) Determie the Maximum a Posteriori estimate of θ give X = x. Solutio (a) The desity f X θ=ϑ of X θ = ϑ is give by The desity f θ of θ is give by f X θ=ϑ (x ϑ) = π e (x ϑ). f θ (ϑ) = π e ϑ. Hece for the joit desity f X,θ of X ad θ we get f X,θ (x, ϑ) = f X θ=ϑ (x ϑ)f θ (ϑ) = π π e (x ϑ) ϑ = π π e (ϑ xϑ+ x ) = π π e (ϑ x) 4 x. The margial desity f X of X is obtaied by itegratig the joit desity f X,θ (x, ϑ) with respect to ϑ: f X (x) = R f X,θ (x, ϑ) dϑ = e 4 x e (ϑ x) dϑ = e 4 x π R π π. Coditioed o X = x, θ has the desity f θ X=x (ϑ x) = f X,θ(x, ϑ) f X (x) = e (ϑ x). π We coclude that θ X = x N ( x, ).
Probability ad Statistics Secod Sessio Exam - Page of 9 09.0.08 (b) The Maximum a Posteriori estimate ˆθ MAP of θ give X = x is defied as ˆθ MAP := arg max f θ X=x(ϑ x) = arg max e (ϑ x). ϑ R ϑ R π Sice (ϑ x) < 0 for all ϑ R \ { x} ad ( x x) = 0, we get ˆθ MAP = x.
Probability ad Statistics Secod Sessio Exam - Page 3 of 9 09.0.08 8. (0 poits) Maximum Likelihood Estimatio Let θ > be ukow. Suppose X is a radom variable with desity f θ give by { (θ + )x θ+ if 0 x, f θ (x) = 0 else. Let (X i ) i be idepedet, idetically distributed radom variables with the same distributio as X. Show that the Maximum Likelihood Esti- (a) (7 poits) Let us cosider X,..., X. mator (MLE) ˆθ MLE (X,..., X ) of θ is ˆθ MLE (X,..., X ) = i= log(x i). (b) (3 poits) For the radom variable X oe ca show that E[log(X)] = θ + ad Var[log(X)] <. Use these two results to show that ˆθ MLE (X,..., X ) give i (a) coverges almost surely to θ, as. Solutio (a) The likelihood fuctio L(θ) is give by L(θ) = f θ (X i ) = i= (θ + )X θ+ i {0 Xi } = (θ + )X θ+ i, i= sice X,..., X oly take values i [0, ]. By takig the logarithm, we get the log-likelihood fuctio [ ] l(θ) = log[l(θ)] = log (θ + )X θ+ i = [log(θ + ) + (θ + )log(x i )]. We the have i= i= i= ˆθ MLE (X,..., X ) = arg max L(θ) = arg max l(θ). θ> θ> We fid the maximum of l(θ) o (, ) by differetiatig l(θ) ad settig the derivative equal to zero. We have θ l(θ) = [ ] [log(θ + ) + (θ + )log(x i )] = θ θ + + log(x i) i= = θ + + log(x i ) i= i=
Probability ad Statistics Secod Sessio Exam - Page 4 of 9 09.0.08 ad θ + + log(x i ) = 0 i= Sice [ ] θ l(θ) = θ θ + + log(x i ) i= θ + = θ = log(x i ) i= i= log(x i). = < 0 for all θ >, (θ + ) l(θ) is strictly cocave o (, ). Therefore the uique root of the log-likelihood fuctio foud above is ideed the locatio of the maximum of the log-likelihood fuctio. Hece we have foud ˆθ MLE (X,..., X ) = i= log(x i). (b) Sice [log(x i )] i are i.i.d. radom variables ad Var[log(X)] < accordig to the hit, we ca apply the Strog Law of Large Numbers to get i= almost surely, as, ad thus almost surely, as. log(x i ) E[log(X)] = θ + ˆθ MLE (X,..., X ) θ+ = θ
Probability ad Statistics Secod Sessio Exam - Page 5 of 9 09.0.08 9. (0 poits) Hypothesis Test Let X,..., X 6 be the aual average temperature of the last six years. Suppose that X,..., X 6 are idepedet, ormally distributed radom variables with ukow mea µ ad variace σ = 3 ad that we made the followig observatios: X = 0, X =, X 3 =, X 4 = 9, X 5 = 3, X 6 =. A meteorologist claims that the expected aual average temperature is 0. We will ivestigate this statemet usig a z-test with sigificace level α = 0.05. To begi with, we defie the ull hypothesis ad the alterative hypothesis as H 0 : µ = 0, H : µ 0. (a) (4 poits) Write dow a test statistic T that depeds o X,..., X 6 ad such that T N (0, ) uder H 0. (b) (4 poits) Let P 0 deote the probability measure uder H 0. Fid q > 0 such that P 0 (T [ q, q ]) = α. Hit: You may use the table for the stadard ormal distributio to read off q i the ed. (c) ( poits) Ca we reject H 0 o the basis of the six observatios 0,,, 9, 3, usig a sigificace level of α = 0.05? Solutio (a) We defie X := 6 6 i= X iid i. Sice X,..., X 6 N ( 0, ) 3 uder H0, we have that X is agai ormally distributed with E [ X] = 6 6 i= uder H 0. If we the defie E[X i ] = 0 ad Var ( X) = 36 6 Var(X i ) = 4 i= T := T (X,..., X 6 ) := X E [ X] Var ( X) = X 0 = X 0, we have that T N (0, ) uder H 0. (b) Let q > 0. Sice the ormal distributio is symmetric, we ca calculate P 0 (T [ q, q]) = P 0 (T q) P 0 (T < q) = P 0 (T q) P 0 (T > q) = P 0 (T q) ( P 0 (T q)) = P 0 (T q). Therefore we ca deduce that P 0 (T [ q, q]) = α P 0 (T q) = α P 0 (T q) = α.
Probability ad Statistics Secod Sessio Exam - Page 6 of 9 09.0.08 If we write Φ for the distributio fuctio of the stadard ormal distributio ad usig that α = 0.05, we get ( q = Φ α ) = Φ (0.975). Checkig the table for the stadard ormal distributio, we fid q =.96. (c) We use the observatios give i the exercise to calculate T (0,,, 9, 3, ) = 66 0 =. 6 Sice > q =.96, we reject H 0 o the basis of the six observatios 0,,, 9, 3, usig a sigificace level of α = 0.05.
Probability ad Statistics Secod Sessio Exam - Page 7 of 9 09.0.08 0. (0 poits) Cofidece Iterval Suppose we have a coi with P( head ) = p, where p (0, ) is ukow ad we toss this coi idepedetly times. For all i, we defie { if we get head i the i-th throw, X i := 0 if we get tail i the i-th throw. Moreover, we defie X := i= X i. Fially, let α (0, ) ad Φ be the stadard ormal distributio fuctio. Use the Cetral Limit Theorem to show that [ ( ) X Φ ( )] α, X + Φ α is a approximate ( α)-cofidece iterval for p. Hit: The cofidece iterval resultig from the Cetral Limit Theorem approximatio will deped o p. Sice a cofidece iterval for p is ot allowed to deped o p, you will have to maximize the iterval with respect to p to get boudaries that do ot deped o p aymore. Solutio We have that X,..., X iid Ber(p), hece E [ X] = i= E[X i ] = p ad Var ( X) = Accordig to the Cetral Limit Theorem, we have Var(X i ) = i= p( p). X E [ X] Var ( X) = X p p( p) N (0, ). Let Z N (0, ) ad q > 0. Sice the ormal distributio is symmetric, we ca calculate Thus, we have { P Z P(Z [ q, q]) = P(Z q) P(Z < q) = P(Z q) P(Z > q) [ Φ ( α = P(Z q) ( P(Z q)) = P(Z q) = Φ(q). ) (, Φ α )]} ( = Φ [Φ α )] = α. Therefore, P X p p( p) ( [ Φ α ) (, Φ α )] α,
Probability ad Statistics Secod Sessio Exam - Page 8 of 9 09.0.08 which leads to [ ( P X Φ α ) p( p) p X + Φ ( α ) ] p( p) α. Sice a cofidece iterval of p is ot allowed to deped o p, we have to maximize p( p) = p p with respect to p. We have p (p p ) = 0 p = ad p (p p ) = < 0. Hece, ( p( ) p) is a strictly cocave fuctio o [0, ] with a uique maximum of = at p =. We coclude that 4 [ ( ) X Φ ( )] α, X + Φ α is a approximate ( α)-cofidece iterval for p.
Probability ad Statistics Secod Sessio Exam - Page 9 of 9 09.0.08 Stadard ormal (cumulative) distributio fuctio. x ) P (X x) = exp ( y dy, for x 0 π 0 3 4 5 6 7 8 9 0.0.5000.5040.5080.50.560.599.539.579.539.5359 0..5398.5438.5478.557.5557.5596.5636.5675.574.5753 0..5793.583.587.590.5948.5987.606.6064.603.64 0.3.679.67.655.693.633.6368.6406.6443.6408.657 0.4.6554.659.668.6664.6700.6736.677.6808.6844.6879 0.5.695.6950.6985.709.7054.7088.73.757.790.74 0.6.757.79.734.7357.7389.74.7454.7486.757.7549 0.7.7580.76.764.7673.7704.7734.7764.7794.783.785 0.8.788.790.7939.7967.7995.803.805.8078.806.833 0.9.859.886.8.838.864.889.835.8340.8365.8389.0.843.8438.846.8485.8508.853.8554.8577.8599.86..8643.8665.8686.8708.879.8749.8770.8790.880.8830..8849.8869.8888.8907.895.8944.896.8980.8997.905.3.903.9049.9066.908.9099.95.93.947.96.977.4.99.907.9.936.95.965.979.99.9306.939.5.933.9345.9357.9370.938.9394.9406.948.949.944.6.945.9463.9474.9484.9495.9505.955.955.9535.9545.7.9554.9564.9573.958.959.9599.9608.966.965.9633.8.964.9649.9656.9664.967.9678.9686.9693.9699.9706.9.973.979.976.973.9738.9744.9750.9756.976.9767.0.9775.97778.9783.9788.9793.9798.98030.98077.984.9869..984.9857.98300.9834.9838.984.9846.98500.98537.98574..9860.98645.98679.9873.98745.98778.98809.98840.98870.98899.3.9898.98956.98983.9900.99036.9906.99086.99.9934.9958.4.9980.990.994.9945.9966.9986.99305.9934.99343.9936.5.99379.99396.9943.99430.99446.9946.99477.9949.99506.9950.6.99534.99547.99560.99573.99585.99598.99609.996.9963.99643.7.99653.99664.99674.99683.99693.9970.997.9970.9978.99736.8.99744.9975.99760.99767.99774.9978.99788.99795.9980.99807.9.9983.9989.9985.9983.99836.9984.99846.9985.99856.9986 3.0.998650.998694.998736.998777.99887.998856.998893.998930.998965.998999 3..99903.999065.999096.9996.99955.99984.999.99938.99964.99989 3..99933.999336.999359.99938.99940.99943.999443.99946.99948.999499 3.3.99957.999534.999550.999566.99958.999596.99960.99964.999638.99965 3.4.999663.999675.999687.999698.999709.99970.999730.999740.999749.999758 3.5.999767.999776.999784.99979.999800.999807.99985.9998.99988.999835 3.6.99984.999847.999853.999858.999864.999869.999874.999879.999883.999888 3.7.99989.999896.999900.999904.999908.9999.99995.99998.9999.99995 3.8.99998.99993.999933.999936.999938.99994.999943.999946.999948.999950 3.9.99995.999954.999956.999958.999959.99996.999963.999964.999966.999967