2016 Final for Advanced Probability for Communications

Size: px

Start display at page:

Download "2016 Final for Advanced Probability for Communications"

Camron Barker
5 years ago
Views:

1 06 Final for Advanced Probability for Communications The number of total points is 00 in this exam.. 8 pt. The Law of the Iterated Logarithm states that for i.i.d. {X i } i with mean 0 and variance, [ ] S n Pr sup n log logn where S n X + + X n. Then for arbitrarily small ɛ>0, what is the iting probability for [ ] S n sup Pr n log logn >ɛ? Justify your ansewr. Hint: S n / n N, wheren is standard normal distributed. Solution. The central it theorems implies that Thus, sup sup S n / n N. [ ] S n Pr n log logn >ɛ [ Pr > + ɛ ] loglogn n [ sup Pr < ɛ ] loglogn n ] sup Pr ΦL. [ n <L Since L can be arbitrarily large, [ ] S n sup Pr n log logn >ɛ. [ +Pr < ɛ ] loglogn n

2 . Define a cdf F recursively via the following four rules. F 0 0 and F ; F x for <x< ; F x/ F x for0 x ; 4 F x F x. By the rules, we obtain F F ; F F, and F F ; F 4 F ; 4 F F ; F 4 ; F 5 F 4 ; F 6 F ; F 7 F ; F 8 4 F. 4 We can continue the above procedure to obtain the F -function values for x k/ for every l and0 k. Based on the setting, the Riemann upper and lower approximations of xdfx with x 0 is given by: L 8 [ ] i i + i 8 [ i + i + xdfx 0 ] i U, where L and U 8 [ i F 8 i + i + F [ F i + F ] i ] i

3 The Riemann upper and lower approximations of 0 xdfxwith x is given by 6 [ ] i i + i L 6 [ i + i + xdfx 0 ] i U, where and 6 L [ i F i + F ] i [ i + i + U ] i Noting that F i+ F i l is either 0 or l, answer the following l questions. a 8 pt. Find the Riemann approximation of xdfx with x 0 l. In other words, determine the upper and lower bounds of the below equation: L l l 0 [ i F xdfx ] i + i F 8 [ i + i + ] i U l. Hint: Based on L and L, deduce the formula of L l. Then deduce the relation between L l and U l.

4 b 8 pt. Determine l L l and l U l and check whether they are equal or not. Solutions. a L and U L + L and U L + L l l l l l andu l L l + l l + l b Apparently, l L l l U l.. a 8 pt. Let {P i } m i and {Q i} m i be positive real numbers satisfying m P i i m Q i. i Prove that for 0 <λ<, m i Q λ i P λ i, and give a necessary and sufficient condition for equality. Hint: Note that m i Qλ i P λ i λ m i P Qi i P i E[fX], where fx x λ and X is a random variable with alphabet { Q, Q,..., Q } m P P P m and distribution Pr[X Q i /P i ]P i. Then apply Jensen s inequality on E[fX]. 4

5 b 8 pt. Continue from a. If λ>orλ<0, does m i Q λ i P λ i still hold? Justify your answer. c 0 pt. General Hölder s inequality Prove that for positive real numbers {a i } m i and {b i} m i, m λ m λ m i a i b a/λ i i b/ λ i, 0 <λ<; i λ m λ i, λ < 0orλ>; m i a/λ i i b/ λ i with equality holding if, and only if, for some constant c, a /λ i cb / λ i for all i. Hint: Use a and b with properly setting P i and Q i. For example, you may set P i b/λ i. Solutions. m i b/λ i a Noting that f x λλ x λ < 0forx>0and0<λ<, we obtain from Jensen s inequality that m λ m λ Q i E[fX] fe[x] P i Q i. P i i Since fx is strictly concave, equality holds if, and only if, X is deterministic, i.e., P i Q i for every i. b f x λλ x λ > 0forx>0andλ<0orλ>. Thus E[fX] fe[x] m i P i Q i P i i λ m λ Q i. i c Setting a /λ i Q i m i a/λ i yields the desired result. and P i b /λ i m i b/λ i 5

6 4. a 8 pt. Suppose that {X n } n is an independent sequence of random variables having the same distribution Pr[X i ] Pr[X i ]. Then Theorem. states that S n N, n where N is standard normal distributed, and S n X + + X n. Find the fourth moment of S n / n using cumulant formulas. Is it asymptotically equal to E[N 4 ]asn goes to infinity? Hint: The cumulants of a zero-mean random variable X are given by C 0 c 0 C 0 c E[X ]Var[X] C 0 c E[X ] C 4 0 c 4 E[X 4 ] E [X ]. Recall why these terms are named cumulants. Also, E[N ] and E[N 4 ]. b 8 pt. Is it always true that for i.i.d. zero-mean and unit variance random variables {X i } i, E[S4 n/n ]E[N 4 ]. If your answer is affirmative, prove it. If not, justify your answer. Hint: Do we assume that X i has finite fourth moment? c 8 pt. For the tandem filtering bank with h τ h τ h n τ n g nτ satisfying G0, G 0 0 and G 0, where Gf isthe Fourier transform of gτ, h τ h τ h n τ 6

7 determine the transfer function n i H if of the iting filter h h h n as n goes to infinity. Hint: Determine the relation between H i f andgf. d 8 pt. Re-defining in c that h i τ λ δτ+ λ n n δτ τ 0for i n, determine the transfer function of the iting filter as n goes to infinity, where δ is the Dirac delta function. Hint: Determine the relation between H i f andgf. Solution. a By the second and fourth cumulants, we know that E[S 4 n] E [S n]n E[X 4 ] E [X ] and This implies E[S n ]ne[x ]. E[] 4 E []+n E[X 4 ] E [X ] n E [X ]+n E[X 4 ] E [X ] ne[x 4 ]+n ne [X ] n n, where the last step follows from E[X 4 ]E[X ]. Thus, [ ] 4 E, n n and is equal to E[N 4 ]. b For i.i.d. zero-mean and unit-variance random variables, E[ 4 ] E [ ]+n E[X 4 ] E [X ] n E [X ]+n E[X 4 ] E [X ] ne[x 4 ]+n ne [X ] ne[x 4 ]+n n. 7

8 Thus, if E[X 4 ] <, E [ n 4 ] However, if E[X 4 ], then E n E[X4 ]+. n [ n 4 ]. c Therefore, the answer to the question is negative. H i f Gf/ n h i τe ıπfτ dτ n g nτe ıπfτ dτ gse ıπfs/ n ds gse ıπf/ ns ds s nτ Thus the transfer function of the filter bank is equal to n f H i f G n n. This implies Gn f n i e n log G n f e s 0 loggfs s s / n e f G fs Gfs s 0 s G0 f G fsgfs [G fs] e G fs s 0 G 0 0 e G 0 f / e f /. 8

9 d H i f λ + λ n n e ıπfτ 0 implies the transfer function of the filter bank is equal to + λe ıπfτ 0 n, n which implies + λe ıπfτ 0 n e λe ıπfτ 0. n 5. a 8 pt. Suppose {X n } n are i.i.d. sequences with each X i {0, } and 0 < Pr[X i ]<. Let Y n X X X n.is{y n } n α-mixing? Justify your answer. b 8 pt. Give an example that a first-order -state Markov sequence not necessarily stationary, irreducible, aperiodic, etc. is not α- mixing. Explain what condition among stationarity, irreducibility and aperiodicity is violated in your example. Hint: Construct a first-order -state Markov sequence that has a very, very strong dependence between X and X n even for large n. Solutions. a Y n Y n X n.thusby Pr[Y n a n Y n a n ]Pr[X n a n a n ] > 0, {Y n } n is a finite-state first-order irreducible aperiodic Markov sequence. Hence, it is α-mixing. b Let the transition probability matrix of the Markov process be [ ] 0. 0

10 Assume the initial probability is Pr[X 0]Pr[X ]. Then Pr[X andx n ] Pr[X ]Pr[X n ] Pr[X ]Pr[X n X ] Pr[X ]Pr[X n ] 4. Hence, this is not an α-mixing process. Since p n ij Pr[X n j X i] {, i j 0, i j {X n } n is not irreducible even though it is aperiodic. 6. a 8 pt. Suppose that µ is a point mass at the origin, and µ{0} σ. Let the characteristic function of random variable X be { ϕ X t exp e itx } R x µdx. Determine the distribution of X. b 8 pt. Re-do a when µ consists of a point mass λx at some x 0. Hint: The characteristic function of a Poisson random variable with parameter λ is E[e ıtx ]exp{λe ıt }. c 0 pt. If an independent triangular array {X n,k } rn k satisfies i. max k rn E[Xn,k ]0; ii. sup n s n <, wheres n r n k E[X n,k ], prove that max ϕxn,k t exp { ϕ Xn,k t } 0, k r n where ϕ Xn,k t E[e ıtx n,k ] is the characteristic function of Xn,k. Hint: See Slide 8-. Solution. 0

11 a b { ϕ X t exp e itx } R x µdx { } exp σ e itx x 0 x { } exp σ ite itx x 0 exp { ıtσ }. Thus X is deterministic with distribution Pr[X σ ]. { ϕ X t exp e itx } R x µdx exp { λ e itx }, Hence, X has the same distribution as xz λ,wherez λ has Poisson distribution with mean λ. c max ϕxn,k t exp { ϕ Xn,k t } k r n r n k r n k r n k ϕxn,k t exp { ϕ Xn,k t } +θn,k t e θ n,kt θ n,k t e θ n,kt r n e t s n / θ n,k t k rn e t s n / max θ n,k t k r n k 0as. θ n,k t

Section 27. The Central Limit Theorem. Po-Ning Chen, Professor. Institute of Communications Engineering. National Chiao Tung University

Section 27. The Central Limit Theorem. Po-Ning Chen, Professor. Institute of Communications Engineering. National Chiao Tung University Section 27 The Central Limit Theorem Po-Ning Chen, Professor Institute of Communications Engineering National Chiao Tung University Hsin Chu, Taiwan 3000, R.O.C. Identically distributed summands 27- Central