Refinement of Two Fundamental Tools in Information Theory
|
|
- Jonathan Chapman
- 5 years ago
- Views:
Transcription
1 Refiemet of Two Fudametal Tools i Iformatio Theory Raymod W. Yeug Istitute of Network Codig The Chiese Uiversity of Hog Kog Joit work with Siu Wai Ho ad Sergio Verdu
2 Discotiuity of Shao s Iformatio Measures q Shao s iformatio measures: H(X), H(X Y), I(X;Y) ad I(X;Y Z). q They are described as cotiuous fuctios [Shao 1948] [Csiszár & Körer 1981] [Cover & Thomas 1991] [McEliece 2002] [Yeug 2002]. q All Shao's iformatio measures are ideed discotiuous everywhere whe radom variables take values from coutably ifiite alphabets [Ho & Yeug 2005]. q e.g., X ca be ay positive iteger. P.2
3 Discotiuity of Etropy q Let P X = {1, 0, 0,...} ad PX = 1,, log log log,...,0,0,.... q As, we have 2 i PX ( i) PX ( i) = log 0. q However, lim H ( X ) =. P.3
4 Discotiuity of Etropy q Theorem 1: For ay c 0 ad ay X takig values from a coutably ifiite alphabet with H(X) <, P X ( PX, PX ) = i PX ( X ) H( X ) + c s.t. V ( i) P ( i) 0 but H X H ( ) X 0 H ( X ) + c H( X ) P.4
5 Discotiuity of Etropy q Theorem 2: For ay c 0 ad ay X takig values from coutably ifiite alphabet with H(X) <, P X P i D( P P ) P i X ( ) s.t. X X = i X ( )log 0 P ( i) but H H ( X ) H( X ) + c ( ) X X H ( X ) + c H( X ) 0 P.5
6 Pisker s iequality D(p q) 1 2 l2 V 2 (p, q) q By Pisker s iequality, covergece w.r.t. implies covergece w.r.t. V(, ). D( ) q Therefore, Theorem 2 implies Theorem 1. P.6
7 Discotiuity of Etropy P.7
8 Discotiuity of Shao s Iformatio Measures q Theorem 3: For ay X, Y ad Z takig values from coutably ifiite alphabet with I(X;Y Z) <, P XYZ but s.t. lim lim I D( PXYZ PXYZ ) ( X ; Y Z ) =. = 0 P.8
9 Discotiuity of Shao s Iformatio Measures Applicatios: chael codig theorem lossless/lossy source codig theorems, etc. Typicality Fao s Iequality Shao s Iformatio Measures P.9
10 To Fid the Capacity of a Commuicatio Chael Alice Chael Bob Capacity C 1 Typicality Capacity C 2 Fao s Iequality P.10
11 O Coutably Ifiite Alphabet Applicatios: chael codig theorem lossless/lossy source codig theorems, etc. Typicality Fao s Iequality Shao s Iformatio Measures discotiuous! P.11
12 Typicality q Weak typicality was first itroduced by Shao [1948] to establish the source codig theorem. q Strog typicality was first used by Wolfowitz [1964] ad the by Berger [1978]. It was further developed ito the method of types by Csiszár ad Körer [1981]. q Strog typicality possesses stroger properties compared with weak typicality. q It ca be used oly for radom variables with fiite alphabet. 12
13 Notatios q Cosider a i.i.d. source {X k, k 1}, where X k takig values from a coutable alphabet X. q Let P = for all k. X P X k q Assume H(P X ) <. q Let X = (X 1, X 2,, X ) q For a sequece x = (x 1, x 2,, x ) X, q N(x; x) is the umber of occurreces of x i x q q(x; x) = -1 N(x; x) ad q Q X = {q(x; x)} is the empirical distributio of x q e.g., x = (1, 3, 2, 1, 1). N(1; x) = 3, N(2; x) = N(3; x) =1 Q X = {3/5, 1/5, 1/5}. 13
14 Weak Typicality q Defiitio (Weak typicality): For ay ε > 0, the weakly typical set W [X]ε with respect to P X is the set of sequeces x = (x 1, x 2,, x ) X such that 1 log P X(x) H(P X ) ε 14
15 Weak Typicality q Defiitio 1 (Weak typicality): For ay ε > 0, the weakly typical set W [X]ε with respect to P X is the set of sequeces x = (x 1, x 2,, x ) X such that D(Q X P X )+ H(Q X ) H(P X ) ε q Note that while H(Q X ) = Empirical etropy Q X x (x)logq X (x) x = Q X (x)log P X (x) 15
16 Asymptotic Equipartitio Property q Theorem 4 (Weak AEP): For ay ε > 0: q 1) If x W [X]ε, the 2 ( H ( X ) + ε ) ( H ( X ) ε ) p( x) 2 q 2) For sufficietly large, q 3) For sufficietly large, { } X > 1 ε Pr W [ X ] ε (1 ε )2 ( H ( X ) ε ) W [ X ] ε 2 ( H ( X ) + ε ) 16
17 Illustratio of AEP X Set of all -sequeces Typical Set of -sequeces: Prob. 1 Uiform distributio 17
18 Strog Typicality q Strog typicality has bee defied i slightly differet forms i the literature. q Defiitio 2 (Strog typicality): For X < ad ay δ > 0, the strogly typical set T [X]δ with respect to P X is the set of sequeces x = (x 1, x 2,, x ) X such that V ( P, Q ) P ( x) q( x; x) δ X = x X the variatioal distace betwee the empirical distributio of the sequece x ad P X is small. X 18
19 Asymptotic Equipartitio Property q Theorem 5 (Strog AEP): For a fiite alphabet X ad ay δ > 0: q 1) If x T [X]δ, the ( H ( X ) + δ ) ( H ( X ) δ ) 2 p( x) 2 q 2) For sufficietly large, { } X > 1 δ Pr T [ X ] δ q 3) For sufficietly large, (1 δ)2 (H (X ) γ ) T [ X ]δ (H (X )+γ ) 2 19
20 Breakdow of Strog AEP q If strog typicality is exteded (i the atural way) to coutably ifiite alphabets, strog AEP o loger holds q Specifically, Property 2 holds but Properties 1 ad 3 do ot hold. P.20
21 Typicality X fiite alphabet Weak Typicality: D( QX PX ) + H ( QX ) H ( PX ) ε Strog Typicality: V ( P X, QX ) δ 21
22 Uified Typicality X coutably ifiite alphabet Weak Typicality: D( QX PX ) + H ( QX ) H ( PX ) ε Strog Typicality: V ( P X, QX ) δ x s.t. D( Q X PX ) is small but H ( Q ) H ( P ) is large X X 22
23 Uified Typicality X coutably ifiite alphabet Weak Typicality: D( QX PX ) + H ( QX ) H ( PX ) ε Strog Typicality: V ( P X, QX ) δ Uified Typicality: D( Q P ) H ( Q ) H ( P ) η. X X + X X 23
24 Uified Typicality q Defiitio 3 (Uified typicality): For ay η > 0, the uified typical set U [X]η with respect to P X is the set of sequeces x = (x 1, x 2,, x ) X such that D(Q X P X )+ H(Q X ) H(P X ) η q Weak Typicality: Strog Typicality: D(Q X P X )+ H(Q X ) H(P X ) ε V(P X,Q X ) δ q Each typicality correspods to a distace measure q Etropy is cotiuous w.r.t. the distace measure iduced by uified typicality 24
25 Asymptotic Equipartitio Property q Theorem 6 (Uified AEP): For ay > 0: q 1) If x U [X]η, the 2 ( H ( X ) + η) ( H ( X ) η ) p( x) 2 q 2) For sufficietly large, q 3) For sufficietly large, { } X > 1 η Pr U [ X ] η (1 η)2 (H (X ) µ ) U [ X ]η (H (X )+µ ) 2 25
26 Uified Typicality q Theorem 7: For ay x X, if x U [X]η, the x W [X]ε ad x T [X]δ, where ε = η ad δ = η 2l 2. 26
27 Uified Joitly Typicality q Cosider a bivariate iformatio source {(X k, Y k ), k 1} where (X k, Y k ) are i.i.d. with geeric distributio P XY. q We use (X, Y) to deote the pair of geeric radom variables. q Let (X, Y) = ((X 1, Y 1 ), (X 2, Y 2 ),, (X, Y )). q For the pair of sequece (x, y), the empirical distributio is Q XY = {q(x,y; x,y)} where q(x,y; x,y) = -1 N(x,y; x,y). 27
28 Uified Joitly Typicality q Defiitio 4 (Uified joitly typicality): For ay η > 0, the uified typical set U [XY]η with respect to P XY is the set of sequeces (x, y) X Y such that D( Q XY + H ( Q X P XY ) ) + H ( Q H ( P X XY ) ) + H ( Q H ( P Y ) XY ) H ( P Y ) η. q This defiitio caot be simplified. 28
29 Coditioal AEP q Defiitio 5: For ay x U [X]η, the coditioal typical set of Y is defied as U [Y X ]η { : (x, y) U [ XY ]η } (x) = y U [Y ]η q Theorem 8: For x U [X]η, if the U [Y X ]η (x) 1, 2 (H (Y X ) ν ) (H (Y X )+ν ) U [Y X ]η (x) 2 where ν 0 as η 0 ad the 29
30 Illustratio of Coditioal AEP 2 H ( X ) x S [ X ] 2 H ( Y ) y S [ Y ] H ( X,Y ) 2 ( x, y ) T [ XY ] P.30
31 Applicatios q Rate-distortio theory q A versio of rate-distortio theorem was proved by strog typicality [Cover & Thomas 1991][Yeug 2008] q It ca be easily geeralized to coutably ifiite alphabet q Multi-source etwork codig q The achievable iformatio rate regio i multisource etwork codig problem was proved by strog typicality [Yeug 2008] q It ca be easily geeralized to coutably ifiite alphabet 31
32 Fao s Iequality q Fao's iequality: For discrete radom variables X ad Y takig values o the same alphabet X = {1, 2, }, let ε = P[ X Y ] = 1 P ( w, w) w X XY q The H ( X Y) ε log( X 1) + h( ε ), where h( x) 1 = xlog + (1 x for 0 < x < 1 ad h(0) = h(1) = 0. 1 x)log 1 x P.32
33 Motivatio 1 q This upper boud o H ( X Y) ε log( X 1) + h( ε) H(X Y ) is ot tight. q For fixed ε ad X, ca always fid X such that H(X Y ) H(X) < ε log( X 1)+ h(ε) q The we ca ask, for fixed P X ad ε, what is max H ( X Y ) < ε log( X 1) + Y : P[ X Y ] = ε P X h( ε ) P.33
34 Motivatio 2 q If X is coutably ifiite, Fao s iequality o loger gives a upper boud o H(X Y). q It is possible that q explaied by the discotiuity of etropy. P X ,,..., ad PY = log log log = q The H(X Y ) = H(X ) but q Uder what coditios ε 0 H(X Y) 0 for coutably ifiite alphabets? H ( X Y) 0 as ε 0 ε = 1 log which ca be { 1,0,0,... } 0 P.34
35 Tight Upper Boud o H(X Y) q Theorem 9: Suppose ε = P[ X Y ] 1 PX (1), the H ( X Y ) ε H ( Q ( PX, ε )) + h( ε ) where the right side is the tight boud depedet o ε ad P X. (This is the simplest of the 3 cases.) q1 q2 q3 q4 ε Q( P X, ε) = { ε q1, ε q2, ε q3, } q Let Φ ( ε ) = ε H ( Q ( P, ε )) h( ε ) X X + P.35
36 Geeralizig Fao s Iequality q Fao's iequality [Fao 1952] gives a upper boud o the coditioal etropy H(X Y) i terms of the error probability ε = Pr{X Y}. q e.g. P X = [0.4, 0.4, 0.1, 0.1] [Fao 1952] H(X Y) [Ho & Verdú 2008] ε P.36
37 Geeralizig Fao s Iequality q e.g., X is a Poisso radom variable with mea equal to 10. q Fao's iequality o loger gives a upper boud o H(X Y). H(X Y) ε P.37
38 Geeralizig Fao s Iequality q e.g. X is a Poisso radom variable with mea equal to 10. q Fao's iequality o loger gives a upper boud o H(X Y). H(X Y) [Ho & Verdú 2008] ε P.38
39 Joit Source-Chael Codig ( S, S2, S 1 k ) Ecoder ( X, X2, X 1 ) Chael S ˆ, Sˆ, Sˆ ) Decoder Y, Y, Y ) ( 1 2 k ( 1 2 k-to- joit source-chael code P.39
40 P.40 Error Probabilities q The average symbol error probability is defied as q The block error probability is defied as = = k i i i k S S k 1 ] ˆ [ 1 P λ )] ˆ, ˆ, ˆ ( ),, ( [ k k k S S S S S S = P µ
41 Symbol Error Rate q Theorem 10: For ay discrete memoryless source ad geeral chael, the rate of a k-to- joit sourcechael code with symbol error probability λ k satisfies k 1 sup I( X X 1 k k H ( S ) Φ S* ; Y ) ( λ ) k where S* is costructed from {S 1, S 2,..., S k } accordig to P P S* S* (1) ( a) = mi = mi j j P Sj a i (1), a 1 = 1PS ( i) i= 1 PS *( i) a j 2. P.41
42 Block Error Rate q Theorem 11: For ay geeral discrete source ad geeral chael, the block error probability µ k of a k- to- joit source-chael code is lower bouded by k Φ 1 S k H ( S ) sup I( X ; Y ) µ k X P.42
43 Iformatio Theoretic Security q Weak secrecy lim 1 I( X ; Y ) = 0 has bee cosidered i [Csiszár & Körer 78, Broadcast chael] ad some semial papers. q [Wyer 75, Wiretap chael I] oly stated that a large value of the equivocatio implies a large value of P ew, where the equivocatio refers to ad P ew meas µ. 1 H ( X Y k ) q It is importat to clarify what exactly weak secrecy implies. P.43
44 Weak Secrecy q E.g., P X = (0.4, 0.4, 0.1, 0.1). H(X) [Fao 1952] H(X Y) [Ho & Verdú 2008] ε = P[X Y] P.44
45 Weak Secrecy q Theorem 12: For ay discrete statioary memoryless source (i.i.d. source) with distributio P X, if q The q Remark: lim 1 I( X ; Y ) = q Weak Secrecy together with the statioary source assumptio is isufficiet to show the maximum error probability. q The proof is based o the tight upper boud o H(X Y) i terms of error probability. 0, lim λ = λmax ad lim µ = 1. P.45
46 Summary Applicatios: chael codig theorem lossless/lossy source codig theorems Typicality Fao s Iequality Weak Typicality Strog Typicality Shao s Iformatio Measures P.46
47 O Coutably Ifiite Alphabet Applicatios: chael codig theorem lossless/lossy source codig theorem Typicality Weak Typicality Shao s Iformatio Measures discotiuous! P.47
48 Uified Typicality Typicality Applicatios: chael codig theorem MSNC/lossy SC theorems Uified Typicality Shao s Iformatio Measures P.48
49 Geeralized Fao s Iequality Applicatios: results o JSCC, IT security MSNC/lossy SC theorems Geeralized Typicality Fao s Iequality Uified Typicality Shao s Iformatio Measures P.49
50 Perhaps... A lot of fudametal research i iformatio theory are still waitig for us to ivestigate. P.50
51 Refereces q S.-W. Ho ad R. W. Yeug, O the Discotiuity of the Shao Iformatio Measures, IEEE Tras. Iform. Theory, vol. 55, o. 12, pp , Dec q S.-W. Ho ad R. W. Yeug, O Iformatio Divergece Measures ad a Uified Typicality, IEEE Tras. Iform. Theory, vol. 56, o. 12, pp , Dec q S.-W. Ho ad S. Verdú, O the Iterplay betwee Coditioal Etropy ad Error Probability, IEEE Tras. Iform. Theory, vol. 56, o. 12, pp , Dec q S.-W. Ho, O the Iterplay betwee Shao's Iformatio Measures ad Reliability Criteria, i Proc IEEE It. Symposium Iform. Theory (ISIT 2009), Seoul, Korea, Jue 28-July 3, q S.-W. Ho, Bouds o the Rates of Joit Source-Chael Codes for Geeral Sources ad Chaels, i Proc IEEE Iform. Theory Workshop (ITW 2009), Taormia, Italy, Oct , P.51
52 Q & A P.52
53 Why Coutably Ifiite Alphabet? q A importat mathematical theory ca provide some isights which caot be obtaied from other meas. q Problems ivolve radom variables takig values from coutably ifiite alphabets. q Fiite alphabet is the special case. q Beefits: tighter bouds, faster coverget rates, etc. q I source codig, the alphabet size ca be very large, ifiite or ukow. P.53
54 Discotiuity of Etropy q Etropy is a measure of ucertaity. q We ca be more ad more sure that a particular evet will happe as time goes, but at the same time, the ucertaity of the whole picture keeps o icreasig. q If oe foud the above statemet couter-ituitive, he/she may have the cocept that etropy is cotiuous rooted i his/her mid. q The limitig probability distributio may ot fully characterize the asymptotic behavior of a Markov chai. P.54
55 Discotiuity of Etropy Suppose a child hides i a shoppig mall where the floor pla is show i the ext slide. I each case, the chace for him to hide i a room is directly proportioal to the size of the room. We are oly iterested i which room the child locates i but ot his exact positio iside a room. Which case do you expect is the easiest to locate the child? P.55
56 Case A 1 blue room + 2 gree rooms Case B 1 blue room + 16 gree rooms Case C 1 blue room gree rooms Case A Case B Case C Case D Case D 1 blue room gree rooms The chace i the blue room The chace i a gree room P.56
57 Discotiuity of Etropy From Case A to Case D, the difficulty is icreasig. By the Shao etropy, the ucertaity is icreasig although the probability of the child beig i the blue room is also icreasig. We ca cotiue to costruct this example ad make the chace i the blue room approachig to 1! The critical assumptio is that the umber of rooms ca be ubouded. So we have see that There is a very sure evet ad large ucertaity of the whole picture ca exist at the same time. Imagie there is a city where everyoe has a ormal life everyday with probability With probability 0.01, however, ay kid of accidet that beyod our imagiatio ca happe. Would you feel a big ucertaity about your life if you were livig i that city? P.57
58 lim 1 I( X ; Y ) = Weak secrecy is isufficiet to show the maximum error probability. Example 1: Let W, V ad X i be biary radom variables. Suppose W ad V are idepedet ad uiform. Let ~ λ ~ µ max max X i W = idepedet ad uiform = 1 max = lim x P X k ( x) = 0.5 ( ) 1 max P ( x) x X 0 = lim V V 1 = 0 = = 3 4 P.58
59 Example 1 Let Y 1 Y 2 Y 3 Y 4... = = = = X 1 X 4 X 9 X 16 X 2 X 3 X 8 X 15 X 5 X 6 X 7 X 14 0 lim lim 1 1 I( X ; Y = 0 k ) X 10 X 11 X 12 X 13 Choose The xˆ lim lim (0,0,,0) = (1,1,,1) µ λ = P[ V = P[ V if if = = Y Y i i = 0 = 1 1 1] = 2 1 1] = 2 i i. < ~ µ 1 4 max ~ < λ = max 3 4 = 1 2 P.59
60 Joit Uified Typicality Ca be chaged to D( Q XY + H ( Q D( Q XY X + H( Q X P XY ) P XY ) ) + H ( Q H ( P X ) + H( Q H ( P X XY ) ) + H ( Q XY ) η. ) H ( P Y ) H( P XY ) H ( P XY ) Y ) η.? As: X Y X Y Q = {q(xy)} D(Q P) << 1 P = {p(xy)} P.60
61 Joit Uified Typicality Ca be chaged to D( Q XY + H ( Q D( Q XY + H ( Q X Y P XY ) P ) XY ) + H ( Q H ( P H ( P X ) + H ( Q Y XY ) ) + H ( Q X ) η. ) H ( P Y ) H ( P X XY ) H ( P )? Y ) η. As: X Y X Y Q = {q(xy)} D(Q P) << 1 P = {p(xy)} P.61
62 Asymptotic Equipartitio Property q Theorem 5 (Cosistecy): For ay (x,y) X Y, if (x,y) U [XY]η, the x U [X]η ad y U [Y]η. q Theorem 6 (Uified JAEP): For ay η > 0: q 1) If (x, y) U [XY]η, the ( H ( XY ) + η) ( H ( XY ) η ) 2 p( x, y) 2 q 2) For sufficietly large, q 3) For sufficietly large, (1 η)2 {( ) } X,Y > 1 η Pr U [ XY ] ( H ( XY ) η ) U η [ XY ] η 2 ( H ( XY ) + η) P.62
On Information Divergence Measures and a Unified Typicality
O Iformatio Divergece Measures ad a Uified Typicality Siu-Wai Ho ad Raymod W. Yeug Abstract Strog typicality, which is more powerful for theorem provig tha weak typicality, ca be applied to fiite alphabets
More informationInformation Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame
Iformatio Theory Tutorial Commuicatio over Chaels with memory Chi Zhag Departmet of Electrical Egieerig Uiversity of Notre Dame Abstract A geeral capacity formula C = sup I(; Y ), which is correct for
More informationUniversal source coding for complementary delivery
SITA2006 i Hakodate 2005.2. p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories,
More informationLecture 15: Strong, Conditional, & Joint Typicality
EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,
More informationAsymptotic Coupling and Its Applications in Information Theory
Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationLecture 6: Source coding, Typicality, and Noisy channels and capacity
15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 6: Source codig, Typicality, ad Noisy chaels ad capacity Jauary 31, 2013 Lecturer: Mahdi Cheraghchi Scribe: Togbo Huag 1 Recap Uiversal
More informationLecture 11: Channel Coding Theorem: Converse Part
EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationInformation Theory and Statistics Lecture 4: Lempel-Ziv code
Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationEE 4TM4: Digital Communications II Information Measures
EE 4TM4: Digital Commuicatios II Iformatio Measures Defiitio : The etropy H(X) of a discrete radom variable X is defied by We also write H(p) for the above quatity. Lemma : H(X) 0. H(X) = x X Proof: 0
More informationLecture 14: Graph Entropy
15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number
More information4.1 Data processing inequality
ECE598: Iformatio-theoretic methods i high-dimesioal statistics Sprig 206 Lecture 4: Total variatio/iequalities betwee f-divergeces Lecturer: Yihog Wu Scribe: Matthew Tsao, Feb 8, 206 [Ed. Mar 22] Recall
More informationEntropies & Information Theory
Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory
More informationLecture 10: Universal coding and prediction
0-704: Iformatio Processig ad Learig Sprig 0 Lecture 0: Uiversal codig ad predictio Lecturer: Aarti Sigh Scribes: Georg M. Goerg Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved
More informationMultiterminal source coding with complementary delivery
Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2006 Seoul, Korea, October 29 November 1, 2006 Multitermial source codig with complemetary delivery Akisato Kimura ad Tomohiko Uyematsu
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationUNIT 2 DIFFERENT APPROACHES TO PROBABILITY THEORY
UNIT 2 DIFFERENT APPROACHES TO PROBABILITY THEORY Structure 2.1 Itroductio Objectives 2.2 Relative Frequecy Approach ad Statistical Probability 2. Problems Based o Relative Frequecy 2.4 Subjective Approach
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationThe method of types. PhD short course Information Theory and Statistics Siena, September, Mauro Barni University of Siena
PhD short course Iformatio Theory ad Statistics Siea, 15-19 September, 2014 The method of types Mauro Bari Uiversity of Siea Outlie of the course Part 1: Iformatio theory i a utshell Part 2: The method
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationSpring Information Theory Midterm (take home) Due: Tue, Mar 29, 2016 (in class) Prof. Y. Polyanskiy. P XY (i, j) = α 2 i 2j
Sprig 206 6.44 - Iformatio Theory Midterm (take home) Due: Tue, Mar 29, 206 (i class) Prof. Y. Polyaskiy Rules. Collaboratio strictly prohibited. 2. Write rigorously, prove all claims. 3. You ca use otes
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationINFORMATION THEORY AND STATISTICS. Jüri Lember
INFORMATION THEORY AND STATISTICS Lecture otes ad exercises Sprig 203 Jüri Lember Literature:. T.M. Cover, J.A. Thomas "Elemets of iformatio theory", Wiley, 99 ja 2006; 2. Yeug, Raymod W. "A first course
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationInequalities for Entropies of Sets of Subsets of Random Variables
Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom
More informationMi-Hwa Ko and Tae-Sung Kim
J. Korea Math. Soc. 42 2005), No. 5, pp. 949 957 ALMOST SURE CONVERGENCE FOR WEIGHTED SUMS OF NEGATIVELY ORTHANT DEPENDENT RANDOM VARIABLES Mi-Hwa Ko ad Tae-Sug Kim Abstract. For weighted sum of a sequece
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationMultiterminal Source Coding with an Entropy-Based Distortion Measure
20 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Multitermial Source Codig with a Etropy-Based Distortio Measure Thomas A. Courtade ad Richard D. Wesel Departmet of Electrical Egieerig Uiversity
More informationL = n i, i=1. dp p n 1
Exchageable sequeces ad probabilities for probabilities 1996; modified 98 5 21 to add material o mutual iformatio; modified 98 7 21 to add Heath-Sudderth proof of de Fietti represetatio; modified 99 11
More informationε-capacity and Strong Converse for Channels with General State
ε-capacity ad Strog Coverse for Chaels with Geeral State Marco Tomamichel Cetre for Quatum Techologies, Natioal Uiversity of Sigapore Email: cqtmarco@us.edu.sg Vicet Y. F. Ta Istitute for Ifocomm Research,
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationThe source coding game with a cheating switcher
The source codig game with a cheatig switcher Hari Palaiyaur hpalaiya@eecsberkeleyedu Cheg Chag cchag@eecsberkeleyedu Departmet of Electrical Egieerig ad Computer Scieces Uiversity of Califoria at Berkeley
More informationENGI Series Page 6-01
ENGI 3425 6 Series Page 6-01 6. Series Cotets: 6.01 Sequeces; geeral term, limits, covergece 6.02 Series; summatio otatio, covergece, divergece test 6.03 Stadard Series; telescopig series, geometric series,
More informationEntropy Bounds for Discrete Random Variables via Coupling
Etropy Bouds for Discrete Radom Variables via Couplig Igal Saso Departmet of Electrical Egieerig Techio, Haifa 3000, Israel E-mail: saso@ee.techio.ac.il Abstract This work provides ew bouds o the differece
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationCooperative Communication Fundamentals & Coding Techniques
3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More information6a Time change b Quadratic variation c Planar Brownian motion d Conformal local martingales e Hints to exercises...
Tel Aviv Uiversity, 28 Browia motio 59 6 Time chage 6a Time chage..................... 59 6b Quadratic variatio................. 61 6c Plaar Browia motio.............. 64 6d Coformal local martigales............
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information6.041/6.431 Spring 2009 Final Exam Thursday, May 21, 1:30-4:30 PM.
6.041/6.431 Sprig 2009 Fial Exam Thursday, May 21, 1:30-4:30 PM. Name: Recitatio Istructor: Questio Part Score Out of 0 2 1 all 18 2 all 24 3 a 4 b 4 c 4 4 a 6 b 6 c 6 5 a 6 b 6 6 a 4 b 4 c 4 d 5 e 5 7
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationRandom Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.
Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)
More informationDifferential Entropy
School o Iormatio Sciece Dieretial Etropy 009 - Course - Iormatio Theory - Tetsuo Asao ad Tad matsumoto Email: {t-asao matumoto}@jaist.ac.jp Japa Advaced Istitute o Sciece ad Techology Asahidai - Nomi
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationAda Boost, Risk Bounds, Concentration Inequalities. 1 AdaBoost and Estimates of Conditional Probabilities
CS8B/Stat4B Sprig 008) Statistical Learig Theory Lecture: Ada Boost, Risk Bouds, Cocetratio Iequalities Lecturer: Peter Bartlett Scribe: Subhrasu Maji AdaBoost ad Estimates of Coditioal Probabilities We
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS Cotets 1. A few useful discrete radom variables 2. Joit, margial, ad
More informationINFINITE SEQUENCES AND SERIES
INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES I geeral, it is difficult to fid the exact sum of a series. We were able to accomplish this for geometric series ad the series /[(+)]. This is
More informationLecture 7: October 18, 2017
Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem
More informationNotes #3 Sequences Limit Theorems Monotone and Subsequences Bolzano-WeierstraßTheorem Limsup & Liminf of Sequences Cauchy Sequences and Completeness
Notes #3 Sequeces Limit Theorems Mootoe ad Subsequeces Bolzao-WeierstraßTheorem Limsup & Limif of Sequeces Cauchy Sequeces ad Completeess This sectio of otes focuses o some of the basics of sequeces of
More informationOn Evaluating the Rate-Distortion Function of Sources with Feed-Forward and the Capacity of Channels with Feedback.
O Evaluatig the Rate-Distortio Fuctio of Sources with Feed-Forward ad the Capacity of Chaels with Feedback. Ramji Vekataramaa ad S. Sadeep Pradha Departmet of EECS, Uiversity of Michiga, A Arbor, MI 4805
More information1 Review and Overview
DRAFT a fial versio will be posted shortly CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #3 Scribe: Migda Qiao October 1, 2013 1 Review ad Overview I the first half of this course,
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationA Note on Matrix Rigidity
A Note o Matrix Rigidity Joel Friedma Departmet of Computer Sciece Priceto Uiversity Priceto, NJ 08544 Jue 25, 1990 Revised October 25, 1991 Abstract I this paper we give a explicit costructio of matrices
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More informationA gentle introduction to Measure Theory
A getle itroductio to Measure Theory Gaurav Chadalia Departmet of Computer ciece ad Egieerig UNY - Uiversity at Buffalo, Buffalo, NY gsc4@buffalo.edu March 12, 2007 Abstract This ote itroduces the basic
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More information5.1 A mutual information bound based on metric entropy
Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local
More information1 Introduction. 1.1 Notation and Terminology
1 Itroductio You have already leared some cocepts of calculus such as limit of a sequece, limit, cotiuity, derivative, ad itegral of a fuctio etc. Real Aalysis studies them more rigorously usig a laguage
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationSymmetric Two-User Gaussian Interference Channel with Common Messages
Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL 680 Email: geg5@illiois.edu Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationTHE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES
THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES Debdeep Pati Idia Statistical Istitute, Kolkata Jue 26, 2006 Abstract The traditioal proof of the strog law of large umbers usig idepedet ad idetically
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information