Symmetric Two-User Gaussian Interference Channel with Common Messages

Size: px
Start display at page:

Download "Symmetric Two-User Gaussian Interference Channel with Common Messages"

Transcription

1 Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX arxiv:40.509v [cs.it] 0 Ja 04 Abstract We cosider symmetric two-user Gaussia iterferece chael with commo messages. We derive a upper boud o the sum capacity, ad show that the upper boud is tight i the low iterferece regime, where the optimal trasmissio scheme is to sed o commo messages ad each receiver treats iterferece as oise. Our result shows that although the availability of commo messages provides a cooperatio opportuity for trasmitters, i the low iterferece regime the presece of commo messages does ot help icrease the sum capacity. I. INTRODUCTION Iterferece chael is a fudametally importat commuicatio model i iformatio theory []. While the exact capacity regio of iterferece chael i the simplest settig with two trasmitter-receiver pairs is still ukow i geeral, recet research efforts have sigificatly improve our uderstadig of the capacity regio. I particular, [] characterizes the capacity regio of two-user Gaussia iterferece chael withi oe bit. The exact capacity regio has also bee derived i certai regimes, e.g. the strog iterferece regime [3], ad the low iterferece regime [4] [6], which show that i the low iterferece regime treatig iterferece as oise is optimal ad achieves the sum capacity. I this paper, we cosider the symmetric two-user Gaussia iterferece chael with commo messages, where each trasmitter wats to sed a private message to its correspodig receiver ad both trasmitters also ited to sed a commo message to both receivers. We derive a upper boud o the sum capacity usig a geie-aided method [], [4], ad show that the upper boud is tight i the low iterferece regime, where the optimal trasmissio scheme is to sed o commo messages ad each receiver treats iterferece as oise. Our result shows that although the availability of commo messages provides a cooperatio opportuity for trasmitters, i the low iterferece regime the presece of commo messages does ot help icrease the sum capacity. A. Orgaizatio This paper is orgaized as follows. We describe the chael model i Sectio II, ad derive a upper boud o the sum capacity i Sectio III. I Sectio IV, we give a atural lower boud o the sum capacity, ad show that the upper boud matches the lower boud i certai low iterferece regime. I Sectio V, we prove that i the low iterferece regime the availability of the commo messages does ot help icrease the sum capacity ad thus treatig iterferece as oise is optimal. Sectio VI cocludes this paper. II. SYSTEM MODEL We cosider a symmetric two-user Gaussia iterferece chael with commo messages. The chael iput-output relatio is give by Y = X + cx + Z Y = X + cx + Z where X i is the sigal set by the ith trasmitter, ad Y i is the sigal received by the ith receiver, Z k is N 0, for k =,, ad E[Xk ] P for k =,. Without loss of geerality, we assume that c 0. There are a set of three idepedet messages W 0, W, W, where W 0 is available at both trasmitters ad iteded for both receivers, W is available at trasmitter oly ad iteded for receiver oly, ad W is available at trasmitter oly ad iteded for receiver oly. We use R i to deote the trasmissio rate for messages W i, for i = 0,,.

2 Fig.. Chael Model III. UPPER BOUND FOR THE SUM CAPACITY I this sectio, we use the geie-added techique to derive a upper boud o the sum rates. Our mai result is give i Theorem. Theorem. The sum rate R 0 + R + R ca be bouded from above as where gp, P is defied as ad gp, P AP, P := R 0 + R + R max gp, P, 3 0 P =P P mi a,a,var Z,Var Z AP,P f P, P, a, Var Z, a, Var Z, 4 { a, a, Var Z, Var Z 0 Var Z a a Var Z Var Z Var Z c P 0 Var Z a a Var Z Var Z Var Z c P 5 ad f P, P, a, Var Z, a, Var Z := 4 log P + c P + c P P P P + c P + c P + P + c P + c P + Var Z cp + a Var Z log c P + a Var Z + P + c P + c P + Var Z cp + a Var Z log c P + a Var Z + 6 Proof: We first prove that R 0 + R + R max gp, P. 7 0 P,P P

3 From Fao s iequality, we have for k =,, ay ɛ > 0, ad sufficietly large R 0 ɛ/3 IW 0 ; Yk 8 = hyk hyk W 0 9 = hyk hyk W 0 hyk Xk, W 0 hyk Xk, W 0 0 = hyk IXk ; Yk W 0 hyk Xk, W 0 Also have from Fao s iequality, we have for k =,, ay ɛ > 0, ad sufficietly large R k ɛ/3 IW k ; Yk IW k ; Yk, W 0 3 = IW k ; Yk W 0 4 IXk ; Yk W 0 5 where 4 follows from the idepedece betwee W k ad W 0, ad 5 follows from the fact that give W 0, W k X k Y k forms a Markov chai. The sum rate R 0 + R + R ɛ = R 0 ɛ/3 + R 0 ɛ/3 + R ɛ/3 + R ɛ/3 6 hy IX ; Y W 0 hy X, W 0 + hy IX ; Y W 0 hy X, W 0 + IX ; Y W 0 + IX ; Y W 0 7 = hy + IX ; Y W 0 hy X, W 0 + hy + IX ; Y W 0 hy X, W 0 8 hy + IX ; Y, U W 0 hy X, W 0 + hy + IX ; Y, U W 0 hy X, W 0 9 = hy + IX ; U W 0 + IX ; Y U, W 0 hy X, W 0 + hy + IX ; U W 0 + IX ; Y U, W 0 hy X, W 0 0 = hy + hu W 0 hu X, W 0 + hy U, W 0 hy X, U, W 0 hy X, W 0 + hy + hu W 0 hu X, W 0 + hy U, W 0 hy X, U, W 0 hy X, W 0 = hu X, W 0 hu X, W 0 + hy + hy + hy U, W 0 + hy U, W 0 + hu W 0 hy X, U, W 0 hy X, W 0 + hu W 0 hy X, U, W 0 hy X, W 0 for ay geie sigals U, U. Motivated by the problem of two-user Gaussia iterferece chael without commo iformatio, we shall choose ad Z ki are i.i.d. N 0, Var Z k ad are correlated with the oise sigal Z ki as Z ki = U ki = cx ki + Z ki 3 a k Z ki + N ki 4 Var Z k

4 Let Note that P k := VarX ki W 0 5 hu k X k, W 0 = hcx k + Z k X k, W 0 = h Z k = log πevar Z k 6 Next, we shall boud from above the rest of the terms o the RHS of i terms of P, P, a, Var Z, a ad Var Z. First, let us cosider hy ad hy. We have hy hy i 7 where 9 is due to the cocavity of the log fuctio. The variace where the cross term log πevary i 8 [ ] log πe VarY i 9 VarY i = VarX i + cx i + Z i 30 = VarX i + cx i + VarZ i 3 = VarX i + c VarX i + ce[x i E[X i ]X i E[X i ]] + VarZ i 3 E[X i E[X i ]X i E[X i ]] = E [E [X i E[X i ]X i E[X i ] W 0 ]] 33 = E [E [X i E[X i ] W 0 ] E [X i E[X i ] W 0 ]] 34 = E [E[X i W 0 ] E[X i ] E[X i W 0 ] E[X i ]] 35 [ E E[X i W 0 ] E[X i ] ] [ E E[X i W 0 ] E[X i ] ] 36 where 34 follows from the idepedece of X i ad X i give W 0, ad 36 follows from the Cauchy-Schwartz iequality. Furthermore, [ E E[X ki W 0 ] E[X ki ] ] [ = E E[X ki W 0 ] ] E[X ki ] 37 = E[Xki] E[X ki ] [ E[Xki] E E[X ki W 0 ] ] 38 = VarX ki VarX ki W 0 39 Substitutig 36 ad 39 ito 3, we may obtai VarY i VarX i + c VarX i + c VarXi VarX i W 0 VarX i VarX i W 0 + VarZ 40 VarX i + c VarX i + c VarX i VarX i W 0 VarX i VarX i W 0 + VarZ P + c P + c P P P P + 4 4

5 where 4 follows from the fact that gx, x, y, y = y x y x 43 is joitly cocave for x y, x y. Hece, we have hy [πe log P + c P + c ] P P P P + ad similarly hy [πe log P + c P + c ] P P P P + Next, we cosider hy U, W 0 ad hy U, W 0. We have hy U, W 0 hy i U i, W 0 46 = hx i + cx i + Z i cx i + Z i, W 0 47 [ log πevarx i + cx i + Z i cx i + Z ] i, W 0 log [ πe ] VarX i + cx i + Z i cx i + Z i, W 0 [ log πe VarXi W 0 + c VarX i W 0 + cvarx i W 0 + a Var Z c VarX i W 0 + Var Z log [πe c VarX i W 0 + c VarX i W 0 + a Var Z c VarX i W 0 + Var Z = log πe P + c P + VarX i W 0 + cp + a Var Z c P + Var Z 5 5 where 5 follows from the fact that gx, y := x + c y + is joitly cocave for x 0, y 0. Similarly, we may also obtai that hy U, W 0 log πe P + c P + cx + a Var Z c x + Var Z cp + a Var Z c P + Var Z 53 54

6 Fially, let us cosider hu W 0 hy X, U, W 0 hy X, W 0 ad hu W 0 hy X, U, W 0 hy X, W 0. We have Assumig that by the coditioal etropy-power iequality, we have hu W 0 hy X, U, W 0 hy X, W 0 = hcx + Z W 0 hx + cx + Z X, cx + Z, W 0 hx + cx + Z X, W 0 55 = hcx + Z W 0 hcx + N W 0 hcx + Z W 0 56 Var Z VarN = a 57 hcx + N W 0 log e hcx + Z W0 + πe a Var Z 58 ad hcx + Z W 0 e log hcx + Z W0 + πe Var Z 59 Substitutig 58 ad 59 ito 56, we may obtai where hu W 0 hy X, U, W 0 hy X, W 0 gt gt := t e log t + πe a Var Z log e t + πe Var Z 60 ad t := hcx + Z W 0 6 hcx i + i W 0 6 = log [πe [ log πevarcx i + Z ] i W 0 [ log πe c VarX i W 0 + Var Z ] c ] VarX i W 0 + Var Z = [πe log c P + Var Z ] The derivative g t = πe a Var Z Var Z e 4t e t + πe a Var Z e t + πe Var Z so gt is a mootoe icreasig fuctio for t [πe 4 log a Var Z Var Z ] 67 Assumig that we have from 66 c P a Var Z Var Z Var Z 68 hu W 0 hy X, U, W 0 hy X, W 0 [ g log πe c P + Var Z ] 69 [ log πe c P + Var Z log πe c P + a log πe c P + ] 70

7 Similarly, assumig that ad Var Z a 7 c P a Var Z Var Z Var Z 7 we have hu W 0 hy X, U, W 0 hy X, W 0 [ log πe c P + Var Z log πe c P + a log πe c P + ]. 73 Therefore, we have R 0 + R + R max gp, P P,P P Next we argue that we oly eed to cosider the case whe P = P. Recall that P k k =, is defied as P k := VarX ki W We show that i this symmetric model, give ay trasmissio scheme, oe ca easily costruct aother trasmissio scheme achievig the same sum rate with P = P. Ideed, suppose i the give trasmissio scheme P P. We costruct aother trasmissio scheme as follows: I the first time block, we use the same code book of the give trasmissio scheme. I the secod time block, sice the chael is symmetric, we ca switch the roles of user ad user ad use the same trasmissio scheme achievig the same sum rate. Hece the ew trasmissio scheme achieves the same sum rate with P = P = P+P. Therefore, we have This completes the proof of Theorem. R 0 + R + R max gp, P P =P P IV. TIGHTNESS OF UPPER BOUND IN THE LOW INTERFERENCE REGIME I this sectio, we first give a lower boud o the sum capacity, ad the show that the upper boud give i Theorem matches the lower boud i the low iterferece regime. A simple codig scheme is that each trasmitter splits the power P ito two parts, oe for commo message M 0 ad oe for the privacy message, ad does chael codig for each message idepedetly, ad each receiver decodes the iteded messages by usig successive iterferece cacellatio. The trasmissio sum rates of this superpositio codig scheme are a atural lower boud for the sum capacity. Lemma. Give P = P = P P 0, the maximum sum rates achieved by the above superpositio codig scheme is where R 0 + R + R log + P + + c P 0 P c + log + P + c P + =IX G, X 0G ; Y G + IX G, Y G X 0G =IX G, X 0G ; Y G + IX G, Y G X 0G, X G = X G + P P X 0G, X G = X G + P P X 0G, Y G = X G + + c P P X 0G + cx G + Z, Y G = X G + + c P P X 0G + cx G + Z, ad X G, X G, X 0G are zero mea Gaussia radom variables with variaces P, P ad. 77

8 Proof: Gaussia radom variables X G, X G ad X 0G correspod to the codebooks for messages W, W ad W 0 i the superpositio codig scheme. The chael iput-output relatio is Y G = X G + + c P P X 0G + cx G + Z Y G = X G + + c P P X 0G + cx G + Z, Treatig iterferece as oise, we ca write dow the expressios of the MAC capacity regio six iequalities i total ad get the maximum sum rates 77 by usig Fourier-Motzki elimiatio. More specifically, give a, b, c, d, e ad f, from the followig iequalities we ca get a tight upper boud for R 0 + R + R via Fourier-Motzki elimiatio: R 0 a 78 R b 79 R 0 + R c 80 R 0 d 8 R e 8 R 0 + R f 83 R 0 + R + R mi{a + b + e, b + d + e, c + e, b + f}. 84 I our MAC chael, we have the implicit coditios a + b c, d + e f ad by symmetry a = d, b = e, c = f. Thus mi{a + b + e, b + d + e, c + e, b + f} = b + f = c + e. This completes our proof. We show that the lower boud 77 ad the upper boud match for all 0 P = P P if the parameters P ad c satisfy certai coditios. More precisely, Theorem 3. Give P ad c, if there exist oegative parameters a ad b such that the the lower boud 77 ad the outer boud 3 match, i.e., gp, P = for 0 P = P P. c + c P = ab 85 c P a b b b 86 a + b. 87 log + P + + c P 0 c + log + P + P c P + Before provig Theorem 3, ote that by settig a = to be a valid solutio of the above costraits, we ca derive a sufficiet coditio uder which the lower boud 77 ad the outer boud match. Corollary 4. If the parameters P ad c satisfy the followig low iterferece coditios: 88 c 4 P + 4c P + 3c + c P 89 ad c + c P, 90 the the lower boud 77 ad the outer boud match. Proof: The idea is very simple. We fid the regio i which a = is a valid solutio of 85, 86 ad 87. Cosider 85 ad 87 ad use the familiar iequality a + b ab, we have Let m := c + c P, so m. From 85 ad 87 we get c + c P = ab a + b. 9 a + m, 9 a

9 thus I 86, let a =, the we get c equivaletly, 4m P a + 4m. 93 m m m, 94 c 4 P + 4c P + 3c + c P. 95 So if P ad c satisfy 89 ad 90, the a = ad b = m are valid solutios of 85, 86 ad 87. Therefore, the lower boud 77 ad the outer boud match due to Theorem 3. A. Proof of Theorem 3 The followig Lemma 5 is a restatemet of 57, 68, 7, 7 i Theorem, which is the so-called useful geie coditio defied i [4]. Lemma 5 says that if the useful geie coditio is satisfied, the the capacity of geie aided chael is achieved by chael iput with Gaussia distributios. To simplify the otatio, defie b k := Var Z k. Lemma 5. Give the coditioal variaces P ad P, if the b a 96 c P a b b b 97 b a 98 c P a b b b 99 R 0 + R + R ɛ IX G; Y G, U G X 0G + IY G ; X G, X 0G +IX G ; Y G, U G X 0G + IY G ; X G, X 0G, 00 where X kg are the zero mea Gaussia radom variables, ad U kg, Y kg are the correspodig Gaussia geie ad output. More specifically, X G := X G + P P X 0G, 0 X G := X G + P P X 0G, 0 where X G, X G ad X 0G are idepedet zero mea Gaussia radom variables with variace P, P ad, respectively. Ad accordigly, Proof: I the proof of Theorem, 9 is The RHS is exactly Y G = X G + cx G + Z 03 Y G = X G + cx G + Z 04 U G = cx G + Z 05 U G = cx G + Z 06 R 0 + R + R ɛ hy + IX ; Y, U W 0 hy X, W 0 +hy + IX ; Y, U W 0 hy X, W 0. IX ; Y, U W 0 + IY ; X, W 0 + IX ; Y, U W 0 + IY ; X, W 0, 07 ad for each term we have derived a outer boud c.f., 4, 5, 54, 70, 70 i the proof of Theorem. It is easy to verify that the derived boud for each term ca be obtaied by replacig every term i the mutual iformatio ad

10 etropy expressios by the correspodig Gaussia radom variables. I this way, we ca equivaletly write the fuctio fp, P, a, Var Z, a, Var Z as IX G; Y G, U G X 0G + IY G ; X G, X 0G + IX G ; Y G, U G X 0G + IY G ; X G, X 0G. 08 This completes the proof. As stated before, Lemma 5 is just a restatemet of the upper boud i Theorem i terms of mutual iformatio amog Gaussia radom variables. The advatage of doig so is that we ca easily compare the lower boud ad outer boud, ad study uder what coditios they match. The followig lemma deals with the so-called smart geie coditio defied i [4]. Uder this coditio, the geie-aided chael sum capacity is same as the oe achieved by superpositio codig ad successive iterferece cacellatio i the geie-free chael. Lemma 6. Give fixed coditioal variaces P = P, if there exist parameters a ad b satisfyig c + c P = ab 09 ad the useful geie coditios i Lemma 5, the the sum capacity gp, P of the geie aided chael is same as the oe achieved by superpositio codig ad successive iterferece cacellatio i the geie-free chael. Proof: We emphasize that here the ier boud ad outer boud are bouds for specific give P ad P, where 0 P = P P. By Lemma 5 ad Lemma, give P = P, the gap betwee outer boud ad ier boud is IX G; U G X 0G, Y G + IX G ; U G X 0G, Y G. 0 If the gap is zero, i.e., outer boud ad ier boud match, each term must be zero sice mutual iformatio is oegative. Ideed, by symmetry we have IX G ; U G X 0G, Y G = IX G ; U G X 0G, Y G. Recall that X G := X G + P P X 0G, X G := X G + P P X 0G, 3 where X G, X G ad X 0G are idepedet zero mea Gaussia radom variables with variace P, P ad, respectively. By defiig X kg := X kg E[X kg X 0G ] = X kkg, 4 we have IX G ; U G X 0G, Y G = 0 IX G ; cx G + Z X 0G, X G + cx G + Z = 0 I X G ; c X G + Z X 0G, X G + c X G + Z = 0 I X G ; c X G + Z X G + c X G + Z = 0, where i the last step we use the fact that X 0G is idepedet of X kg, i.e., X 0G is idepedet of X kkg. The last coditio is equivalet to the Markov Chai coditio: X G X G + c X G + Z c X G + Z. 5 Sice all the radom variables are Gaussia, by the fact that Gaussia radom variables X Y Z if ad oly if CovX, Z = CovX, Y CovY CovY, Z, 6 we get the smart geie coditio c + c P = ab. 7 So far, we have show that give P = P, uder what coditios ier boud ad outer boud match. The ext step is to show for all P ad P, where 0 P = P P, there exist parameters ap ad bp satisfyig both useful geie ad smart geie coditios, ad this will coclude our proof.

11 More specifically, we wat to show that i some low iterferece regime, for ay P P, there exist oegative parameters a ad b we emphasize here a ad b ca be a fuctio of P satisfyig the followig coditios c + c P = ab 8 c P a b b b 9 a + b. 0 It is easy to see that we oly eed to cosider the case that P = P for the above costraits, sice if there exist a ad b satisfyig the coditios for the case P = P, which are exactly equatios 85, 86 ad 87, the it has solutios for all P P as P decreases, we ca fix a ad decrease the value of b to satisfy all the costraits. This completes the proof of Theorem. V. OPTIMA COMMON MESSAGE RATE IN THE LOW INTERFERENCE REGIME I this sectio, we show that i the low iterferece regime defied i 89 ad 90, the optimal power allocatio is to set P 0 to be zero to achieve sum capacity. Lemma 7. If the the sum rate 77 c 4 + c 3 + c P + c + c 0, RP 0 := log + P P c P 0 P P 0 c + log + P P 0 + c P P 0 + is a decreasig fuctio of P 0 ad thus is maximized by settig P 0 to be zero. So Proof: To simplify the otatio, defie ad the derivative of RP 0 is a := P + c 3 b := P + + P c 4 d := c 5 e = + c. 6 RP 0 = logb + dp 0 a P 0 + log b ep 0 a P 0, 7 dr dp 0 = I total there are three poles ad oe zero, which are It is easy to see bd aed bep 0 + abd + b abe. 8 dp 0 + bp 0 aep 0 b p = b d = cp + + P c p = a = P + c > 0, < 0, p 3 = b e = P + + c < p P + +P c [ + z = c + c P + c + c ] 3 c 4 + c + c + c + 3 c P + 4 c + c + 3 c c. 5 p < 0 < P < p 3 < p. 9 Now we oly eed to cosider the value of z. If c is sufficietly small, the z is egative, ad thus dr dp 0 is egative o [max{p, z }, t 3 ], so RP 0 is a mootoically decreasig fuctio o [0, P ], sice p, z < 0 ad t 3 > P. The coditio of z 0 is exactly the iequality. Lastly, we prove that the coditios 89 ad 90 imply. Deote by Γ A the regio of c, P determied by 89 ad 90, ad deote by Γ B the regio of c, P determied by.

12 Theorem 8. Γ A Γ B. Thus, whe 89 ad 90 hold, the optimal rate for the commo message is zero to achieve the sum capacity. Proof: Note that LHS of 89, 90 ad are icreasig fuctios of P. Fix c, ad let c, P A ad c, P B be the poits o the boudary of Γ A ad Γ B, respectively. To show Γ A Γ B, it is sufficiet to show P A P B. Equivaletly, it is sufficiet to show either c 4 P B + 4c P B + 3c + c P B, 30 or Ideed, we will prove c + c P B. 3 c 4 P B + 4c P B + 3c + c P B, 3 c + c P B. 33 First from we get Therefore, P B = c c c 4 + c 3 + c. 34 c + c P B 35 The last step holds obviously, so c + c P B. Next we prove c 4 P B + 4c P B + 3c + c P B. c c c + c + c + 36 c c + c c c + c c c c. 40 c 4 P B + 4c P B + 3c + c P B 4 c c c c + c + c + 4 c c c + c + + c c + 7 c + c + c + c + + 3c + 4c c + c + c c c + c c c c + c + + 4c c c + 7 c c + c 4 4c + 4c 3 c + c + 4c 4 8c 3 + 8c c c + c 4 + c 6 + 4c 3 + 6c 5 c + 4c 4 8c 3 + 8c c c 6 + 6c 5 + 7c 4 4c 3 + 3c c + 47 c6 + 6c 5 + 5c 4 + 0c 3 + 5c + 6c + 48 c 6 + 6c 5 c 4 8c 3 + 3c 0c c c + 4c Sice the last step holds, we have c 4 PB + 4c P B + 3c + c P B. This completes the proof of Theorem 8.

13 VI. CONCLUSION We cosider symmetric two-user Gaussia iterferece chael with commo messages. We derive a upper boud o the sum capacity, ad show that the upper boud is tight i the low iterferece regime, where the optimal trasmissio scheme is to sed o commo messages ad each receiver treats iterferece as oise. Our result shows that although the availability of commo messages provides a cooperatio opportuity for trasmitters, i the low iterferece regime the presece of commo messages does ot help icrease the sum capacity. ACKNOWLEDGMENT We thak Prof. Pramod Viswaath ad Dr. Sreekath Aapureddy for the helpful discussios. Research of Qua Geg was supported i part by Prof. Pramod Viswaath s Natioal Sciece Foudatio grat No. CCF REFERENCES [] T. M. Cover ad J. A. Thomas, Elemets of iformatio theory. Joh Wiley & Sos, 0. [] R. Etki, D. Tse, ad H. Wag, Gaussia iterferece chael capacity to withi oe bit, IEEE Trasactios o Iformatio Theory, vol. 54, o., pp , 008. [3] H. Sato, The capacity of the gaussia iterferece chael uder strog iterferece corresp., IEEE Trasactios o Iformatio Theory, vol. 7, o. 6, pp , 98. [4] V. Aapureddy ad V. Veeravalli, Gaussia iterferece etworks: Sum capacity i the low-iterferece regime ad ew outer bouds o the capacity regio, IEEE Trasactios o Iformatio Theory, vol. 55, o. 7, pp , 009. [5] X. Shag, G. Kramer, ad B. Che, A ew outer boud ad the oisy-iterferece sum rate capacity for gaussia iterferece chaels, IEEE Trasactios o Iformatio Theory, vol. 55, o., pp , 009. [6] A. Motahari ad A. Khadai, Capacity bouds for the gaussia iterferece chael, IEEE Trasactios o Iformatio Theory, vol. 55, o., pp , 009.

arxiv: v1 [cs.it] 13 Jul 2012

arxiv: v1 [cs.it] 13 Jul 2012 O the Sum Capacity of the Discrete Memoryless Iterferece Chael with Oe-Sided Weak Iterferece ad Mixed Iterferece Fagfag Zhu ad Biao Che Syracuse Uiversity Departmet of EECS Syracuse, NY 3244 Email: fazhu{biche}@syr.edu

More information

Lecture 11: Channel Coding Theorem: Converse Part

Lecture 11: Channel Coding Theorem: Converse Part EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information

On the Capacity of Symmetric Gaussian Interference Channels with Feedback

On the Capacity of Symmetric Gaussian Interference Channels with Feedback O the Capacity of Symmetric Gaussia Iterferece Chaels with Feedback La V Truog Iformatio Techology Specializatio Departmet ITS FPT Uiversity, Haoi, Vietam E-mail: latv@fpteduv Hirosuke Yamamoto Dept of

More information

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL Joural of Theoretical ad Applied Iformatio Techology 31 st December 212 Vol 46 No2 25-212 JATIT & LLS All rights reserved ISSN: 1992-8645 wwwatitorg E-ISSN: 1817-3195 SCCESSIVE INTERFERENCE CANCELLATION

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Homework Set #3 - Solutions

Homework Set #3 - Solutions EE 15 - Applicatios of Covex Optimizatio i Sigal Processig ad Commuicatios Dr. Adre Tkaceko JPL Third Term 11-1 Homework Set #3 - Solutios 1. a) Note that x is closer to x tha to x l i the Euclidea orm

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame Iformatio Theory Tutorial Commuicatio over Chaels with memory Chi Zhag Departmet of Electrical Egieerig Uiversity of Notre Dame Abstract A geeral capacity formula C = sup I(; Y ), which is correct for

More information

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients. Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,

More information

Lecture 7: October 18, 2017

Lecture 7: October 18, 2017 Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12

Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12 Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

On Random Line Segments in the Unit Square

On Random Line Segments in the Unit Square O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Self-normalized deviation inequalities with application to t-statistic

Self-normalized deviation inequalities with application to t-statistic Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric

More information

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources Hybrid Codig for Gaussia Broadcast Chaels with Gaussia Sources Rajiv Soudararaja Departmet of Electrical & Computer Egieerig Uiversity of Texas at Austi Austi, TX 7871, USA Email: rajivs@mailutexasedu

More information

Lecture 2. The Lovász Local Lemma

Lecture 2. The Lovász Local Lemma Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio

More information

ON POINTWISE BINOMIAL APPROXIMATION

ON POINTWISE BINOMIAL APPROXIMATION Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece

More information

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

1 Approximating Integrals using Taylor Polynomials

1 Approximating Integrals using Taylor Polynomials Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................

More information

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018) Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black

More information

Inseparability of the Multiple Access Wiretap Channel

Inseparability of the Multiple Access Wiretap Channel Iseparability of the Multiple Access Wiretap Chael Jiawei Xie Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 074 xiejw@umd.edu ulukus@umd.edu Abstract We

More information

Lecture 15: Strong, Conditional, & Joint Typicality

Lecture 15: Strong, Conditional, & Joint Typicality EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,

More information

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,

More information

Notes 5 : More on the a.s. convergence of sums

Notes 5 : More on the a.s. convergence of sums Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series

More information

Inequalities for Entropies of Sets of Subsets of Random Variables

Inequalities for Entropies of Sets of Subsets of Random Variables Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom

More information

Increasing timing capacity using packet coloring

Increasing timing capacity using packet coloring 003 Coferece o Iformatio Scieces ad Systems, The Johs Hopkis Uiversity, March 4, 003 Icreasig timig capacity usig packet colorig Xi Liu ad R Srikat[] Coordiated Sciece Laboratory Uiversity of Illiois e-mail:

More information

MATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1

MATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1 MATH 2: HOMEWORK 6 SOLUTIONS CA PRO JIRADILOK Problem. If s = 2, ad Problem : Rudi, Chapter 3, Problem 3. s + = 2 + s ( =, 2, 3,... ), prove that {s } coverges, ad that s < 2 for =, 2, 3,.... Proof. The

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation ECE 645: Estimatio Theory Sprig 2015 Istructor: Prof. Staley H. Cha Maximum Likelihood Estimatio (LaTeX prepared by Shaobo Fag) April 14, 2015 This lecture ote is based o ECE 645(Sprig 2015) by Prof. Staley

More information

Lecture 12: September 27

Lecture 12: September 27 36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Capacity Theorems for the Finite-State Broadcast Channel with Feedback

Capacity Theorems for the Finite-State Broadcast Channel with Feedback Capacity Theorems for the Fiite-State Broadcast Chael with Feedback Ro abora ad Adrea Goldsmith ept of Electrical Egieerig, Staford Uiversity Abstract We cosider the discrete, time-varyig broadcast chael

More information

Assignment 5: Solutions

Assignment 5: Solutions McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

4. Partial Sums and the Central Limit Theorem

4. Partial Sums and the Central Limit Theorem 1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems

More information

Shannon s noiseless coding theorem

Shannon s noiseless coding theorem 18.310 lecture otes May 4, 2015 Shao s oiseless codig theorem Lecturer: Michel Goemas I these otes we discuss Shao s oiseless codig theorem, which is oe of the foudig results of the field of iformatio

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Lecture 2: Concentration Bounds

Lecture 2: Concentration Bounds CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy

More information

5.1 A mutual information bound based on metric entropy

5.1 A mutual information bound based on metric entropy Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE Geeral e Image Coder Structure Motio Video (s 1,s 2,t) or (s 1,s 2 ) Natural Image Samplig A form of data compressio; usually lossless, but ca be lossy Redudacy Removal Lossless compressio: predictive

More information

Fundamental Theorem of Algebra. Yvonne Lai March 2010

Fundamental Theorem of Algebra. Yvonne Lai March 2010 Fudametal Theorem of Algebra Yvoe Lai March 010 We prove the Fudametal Theorem of Algebra: Fudametal Theorem of Algebra. Let f be a o-costat polyomial with real coefficiets. The f has at least oe complex

More information

The log-behavior of n p(n) and n p(n)/n

The log-behavior of n p(n) and n p(n)/n Ramauja J. 44 017, 81-99 The log-behavior of p ad p/ William Y.C. Che 1 ad Ke Y. Zheg 1 Ceter for Applied Mathematics Tiaji Uiversity Tiaji 0007, P. R. Chia Ceter for Combiatorics, LPMC Nakai Uivercity

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + 62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of

More information

2.2. Central limit theorem.

2.2. Central limit theorem. 36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard

More information

Entropy Rates and Asymptotic Equipartition

Entropy Rates and Asymptotic Equipartition Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

A Partial Decode-Forward Scheme For A Network with N relays

A Partial Decode-Forward Scheme For A Network with N relays A Partial Decode-Forward Scheme For A etwork with relays Yao Tag ECE Departmet, McGill Uiversity Motreal, QC, Caada Email: yaotag2@mailmcgillca Mai Vu ECE Departmet, Tufts Uiversity Medford, MA, USA Email:

More information

Introduction to Probability. Ariel Yadin

Introduction to Probability. Ariel Yadin Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways

More information

Basics of Probability Theory (for Theory of Computation courses)

Basics of Probability Theory (for Theory of Computation courses) Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.

More information

Generalized Semi- Markov Processes (GSMP)

Generalized Semi- Markov Processes (GSMP) Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

A Proof of Birkhoff s Ergodic Theorem

A Proof of Birkhoff s Ergodic Theorem A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed

More information

Problem Set 2 Solutions

Problem Set 2 Solutions CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

A New Achievability Scheme for the Relay Channel

A New Achievability Scheme for the Relay Channel A New Achievability Scheme for the Relay Chael Wei Kag Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 20742 wkag@umd.edu ulukus@umd.edu October 4, 2007

More information

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero? 2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a

More information

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes. Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

The standard deviation of the mean

The standard deviation of the mean Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

University of Manitoba, Mathletics 2009

University of Manitoba, Mathletics 2009 Uiversity of Maitoba, Mathletics 009 Sessio 5: Iequalities Facts ad defiitios AM-GM iequality: For a, a,, a 0, a + a + + a (a a a ) /, with equality iff all a i s are equal Cauchy s iequality: For reals

More information

Review Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =

Review Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = = Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:

More information

Optimally Sparse SVMs

Optimally Sparse SVMs A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but

More information

Overview of Gaussian MIMO (Vector) BC

Overview of Gaussian MIMO (Vector) BC Overview of Gaussia MIMO (Vector) BC Gwamo Ku Adaptive Sigal Processig ad Iformatio Theory Research Group Nov. 30, 2012 Outlie / Capacity Regio of Gaussia MIMO BC System Structure Kow Capacity Regios -

More information

Information Theory and Coding

Information Theory and Coding Sol. Iformatio Theory ad Codig. The capacity of a bad-limited additive white Gaussia (AWGN) chael is give by C = Wlog 2 ( + σ 2 W ) bits per secod(bps), where W is the chael badwidth, is the average power

More information

Discrete Mathematics for CS Spring 2008 David Wagner Note 22

Discrete Mathematics for CS Spring 2008 David Wagner Note 22 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

On the Dispersions of the Discrete Memoryless Interference Channel

On the Dispersions of the Discrete Memoryless Interference Channel O the Dispersios of the Discrete Memoryless Iterferece Chael Sy-Quoc Le, Vicet Y. F. Ta,2, ad Mehul Motai Departmet of ECE, Natioal Uiversity of Sigapore, Sigapore 2 Istitute for Ifocomm Research (I 2

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Spring Information Theory Midterm (take home) Due: Tue, Mar 29, 2016 (in class) Prof. Y. Polyanskiy. P XY (i, j) = α 2 i 2j

Spring Information Theory Midterm (take home) Due: Tue, Mar 29, 2016 (in class) Prof. Y. Polyanskiy. P XY (i, j) = α 2 i 2j Sprig 206 6.44 - Iformatio Theory Midterm (take home) Due: Tue, Mar 29, 206 (i class) Prof. Y. Polyaskiy Rules. Collaboratio strictly prohibited. 2. Write rigorously, prove all claims. 3. You ca use otes

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Supplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting

Supplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting Supplemetary Materials for Statistical-Computatioal Phase Trasitios i Plated Models: The High-Dimesioal Settig Yudog Che The Uiversity of Califoria, Berkeley yudog.che@eecs.berkeley.edu Jiamig Xu Uiversity

More information

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other

More information

Output Analysis and Run-Length Control

Output Analysis and Run-Length Control IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

Largest families without an r-fork

Largest families without an r-fork Largest families without a r-for Aalisa De Bois Uiversity of Salero Salero, Italy debois@math.it Gyula O.H. Katoa Réyi Istitute Budapest, Hugary ohatoa@reyi.hu Itroductio Let [] = {,,..., } be a fiite

More information

Information-based Feature Selection

Information-based Feature Selection Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with

More information

Discrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 18

Discrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 18 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2013 Aat Sahai Lecture 18 Iferece Oe of the major uses of probability is to provide a systematic framework to perform iferece uder ucertaity. A

More information

Application to Random Graphs

Application to Random Graphs A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let

More information

1 Duality revisited. AM 221: Advanced Optimization Spring 2016

1 Duality revisited. AM 221: Advanced Optimization Spring 2016 AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R

More information

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ. 2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For

More information