STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight. Problem. Suppose we geerate a radom variable X i the followig way. First we flip a fair coi. If the coi is heads, take X to have a N µ, σ distributio. If the coi is tails, take X to have a N µ, σ distributio. Fid: a The mea of X b The stadard deviatio of X Solutio. Let Z Beroulli, the X Z N µ, σ, X Z 0 N µ, σ. Note: Questio asks for stadard deviatio, ot the variace be careful a EX E[[E[X Z]] E[Zµ + Zµ ] µ + µ b VarX Var[E[X Z]] + E[Var[X Z]] Var[Zµ + Zµ ] + E[Zσ + Zσ ] Var[µ µ Z + µ ] + σ + σ µ µ VarZ + σ + σ 4 µ µ + σ + σ sdx VarX 4 µ µ + σ + σ Problem. For a collectio of radom variables prove that: Var i a i X i i j a i a j CovX i, X j. Solutio. Note: It is possible to use iductio to prove this give that we are give the formula. However sometimes just proceedig directly from defiitio i this case of variace ca be easier.
stat-36700 homework - solutios Var i a i X i E a i X i i i j i j i j [ a i a j EX i X j E i a i X i ] i j a i a j EXi X j EX i EX j a i a j CovX i, X j Problem 3. Let X N µ, σ. Show that M X t expµt + σ t / a i a j EX i EX j Solutio 3. Here we ote that if Z N 0, ad X N µ, σ the 3 X µ + σz Proof. We proceed as follows: M X t : E X e Xt 3 Note: Here we ca fid a affie i.e. scale ad shift represetatio of X i terms of Z. The advatage is that the MGF of Z is much easier to calculate usig Lemma 0. E Z e σz+µt by defiitio of X e µt M Z σt Usig lemma 0. e µt e σ t Usig lemma 0. e µt+ σ t Problem 4. Suppose that X, X..., X are i.i.d. radom variables with E[X i ] µ ad Var[X i ] σ Let ad Show that a E[Y] µ. b Var[Y] σ. c E[S] σ. Y S X i, i i X i Y Solutio 4. We firstly ote that sice the X i s are IID for all it follows that EX i EX µ ad VarX i VarX σ for all.
stat-36700 homework - solutios 3 a EY µ Proof. We proceed directly 4 : EY E X i i i i µ i µ EX i EX by liearity of expectatio sice X i s are idetically distributed 4 Key poit: We did ot rely o idepedece of X i s here to derive the expectatio of T. Simply usig liearity of Expectatio ad Idetically distributed X i s was eough. Always try ad prove statemets with miimal required assumptios µ b Var[Y] σ. Proof. We firstly ote that 5 VarY Var X i i 5 Key poit: Here we do rely o both idepedece of X i s ad their idetical distributio calculatio to simplify the variace of Y i VarX i by idepedece of X i s i VarX sice X i s are idetically distributed σ i σ σ c E[S] σ Proof. We ote some prelimiary useful idetities to simplify the proof 6. a X i X i i X i X b i X i µ i X i i µ X µ X µ 6 Key poit: This questio shows how to costruct a ubiased estimator of the true variace from the sample data. These are useful idetities to ote dow ad used frequetly i similar proofs
stat-36700 homework - solutios 4 Approach Proof. Assumig the prelimiary useful idetities, we proceed as follows: E E i X i X X i X i i ix i X i i X i }{{} X + X i X + X Xi X i i E i E Xi E X X E X i X by liearity of expectatio sice X i s are i.i.d VarX + EX Var X + E X σ + µ σ + µ σ from parts a b X i X σ by liearity of expectatio ad rescalig by : ES Approach Assumig the prelimiary useful idetities, we proceed with the followig useful decompositio 7 : 7 Key poit: Decompositios of complicated expressios are really isightful. It may lead to a loger proof i this case but ofte the breakdow ca be more isightful ad iterpretable. I comig weeks we will see the useful bias-variace decompositio which is used to measure squared error loss i machie learig so keep this add-subtract µ approach i mid
stat-36700 homework - solutios 5 E E i X i X ix i X i i i i X i µ + µ X add ad subtract µ [ X i µ X i µ X µ + X µ ] expad the square [ X i µ ] i [X i µ X µ] + split ito separate sums [ X i µ ] X µ i i remove X µ outside the sum Which gives us the required result i [X i µ] + E X i µ E X µ + } {{ } } {{ } σ σ from part b usig liearity of expectatio σ σ σ + σ i i i [ X µ ] [ X µ ] E X µ }{{} σ from part b X i X σ by liearity of expectatio ad rescalig by : ES Problem 5. Let X, Y have the uiform distributio o [, ] [, ] Fid the probability that X + Y /. Solutio 5. Claim: this is 9 3 Approach Proof. This is the probability that X, Y fall i the upper triagular regio o the box [, ] [, ] give that they are joitly uiform o the box [, ] [, ]. The required probability is simply the ratio of the area of the upper triagle to the area of the box i this case 8 : P X + Y P Y X 3 9 3 8 Key poit: Try ad draw a picture ad exploit the geometry of the problem to fid the quickest solutio. I this case because of the uiform distributio ad thus uiform volume i 3D, we are simply cocered with relative D areas to get our required probability
stat-36700 homework - solutios 6.5.0 0.5 0.0 0.5.0.5 X + Y.5.0 0.5 0.0 0.5.0.5 Approach Usig itegratio 9 : Notice that f X,Y x, y 4 o [, ] [, ] so that 9 f X,Y x, y dx dy. Note: itegratio approach is loger tha the geometric approach used i part a but more geeral whe the desity is ot uiform. Agai, drawig a picture helps. P X + Y P Y X / / [ 8 + x 8 4 + 3 9 3 / x 8 + x 4 dx ] / dy dx 4 Problem 6. Let X, Y have the uiform distributio o the set {x, y : x + y }. Fid the joit desity fuctio of X, Y. Solutio 6. By defiitio this is the uiform distributio of the closed uit disk circle i R. So we defie the joit desity f X,Y x, y as follows 0 : 0 Key poit: Always remember to write dow the desity for all values x, y R i.e. iclude the 0 desity value for poits outside the uit disk i this case
stat-36700 homework - solutios 7 π if x + y f X,Y x, y 0 otherwise Problem 7. Let F be a cotiuous, strictly icreasig CDF. Let U be a radom variable uiformly distributed o [0, ]. Show that the radom variable Z F U has CDF F. Remark. This result lets us draw samples from ay distributio usig samples from the uiform distributio o [0, ] Solutio 7. Proof. We deote the CDF of Z : F Z z F Z z : PZ z defiitio of CDF of Z P F U z by defiitio of Z PU Fz by ivertibility of F : F U Fz writig i terms of the CDF of U Fz Usig Lemma 0.3 sice U Uif0, So the radom variable Z F U has CDF F as required. Problem 8. Let X be uiformly distributed o [ 5, ]. Let Y X 4. Fid the CDF ad PDF of Y. Solutio 8. F Y y PY y PX 4 y y [0,] P y 4 X y 4 + y,65] P y 4 X Note: Compare this to the approach take i the similar Q8c of HW y 4 y [0,] y 4 6 dx + y,65] y 4 y [0,] 3 y 4 + y,65] 6 + y 4 This implies that the CDF ad PDF are: 6 dx 0 y 0 F Y y 3 y 4 0 < y 6 + y 4 < y 65 y > 65 y 3 4 0 < y f Y y F Y y 4 y 3 4 < y 65 0 otherwise
stat-36700 homework - solutios 8 Appedix: Supportig Lemmas Here we prove some lemmas from the lecture otes that we use repeatedly i the solutios Lemma 0. Scalig ad Shiftig MGF. For a radom variables X ad Y such that Y ax + b we have the followig relatioship betwee the MGFs: M Y t e bt M X at where the MGFs of X ad Y as deoted as M X t ad M Y t respectively Proof. M Y t : E Y e Yt E Y e ax+bt by defiitio of Y E Y e bt e axt e bt E X e atx e bt M X at From which we obtai the required result. Lemma 0. MGF of stadard ormal. For Z N 0, the M Z t e t Proof. M Z t : E Z e Zt e t e zt f Z zdz e zt π e z dz π e z +zt dz π e z t e t dz π e z t dz }{{} the itegral of N t, over it s etire support e t Which is the required result.
stat-36700 homework - solutios 9 Lemma 0.3 CDF of Uif0,. Let U Uif0, ad let F U u deote the CDF of U. The F U u u u [0, ] Proof. F U u : PU u u 0 [t] u 0 u dt by defiitio of Y From which we obtai the required result. Refereces