Lecture 4 Capacity of Wireless Channels

Lecture 4 Capacity of Wireless Channels I-Hsiang Wang ihwang@ntu.edu.tw 3/0, 014

What we have learned So far: looked at specific schemes and techniques Lecture : point-to-point wireless channel - Diversity: combat fading by exploiting inherent diversity - Coding: combat noise, and further exploits degrees of freedom Lecture 3: cellular system - Multiple access: TDMA, CDMA, OFDMA - Interference management: orthogonalization (partial frequency reuse), treat-interference-as-noise (interference averaging)

Information Theory Is there a framework to - Compare all schemes and techniques fairly? - Assert what is the fundamental limit on how much rate can be reliably delivered over a wireless channel? Information theory! - Provides a fundamental limit to (coded) performance - Identifies the impact of channel resources on performance - Suggest novel techniques to communicate over wireless channels Information theory provides the basis for the modern development of wireless communication 3

Historical Perspective G. Marconi C. Shannon 1901 1948 First radio built 100+ years ago Great stride in technology But design was somewhat ad-hoc Information theory: every channel has a capacity Provides a systematic view of all communication problems Engineering meets science New points of view arise 4

Modern View on Multipath Fading Channel quality Time Classical view: fading channels are unreliable - Diversity techniques: average out the variation Modern view: exploit fading to gain spectral efficiency - Thanks to the study on fading channel through the lens of information theory! 5

Plot Use a heuristic argument (geometric) to introduce the capacity of the AWGN channel Discuss the two key resources in the AWGN channel: - Power - Bandwidth The AWGN channel capacity serves as a building block towards fading channel capacity: - Slow fading channel: outage capacity - Fast fading channel: ergodic capacity 6

AWGN Channel Capacity Outline Resources of the AWGN Channel Capacity of some LTI Gaussian Channels Capacity of Fading Channels 7

AWGN Channel Capacity 8

Channel Capacity Capacity := the highest data rate can be delivered reliably over a channel - Reliably Vanishing error probability Before Shannon, it was widely believed that: - to communicate with error probability 0 - data rate must also 0 Repetition coding (with M-level PAM) over N time slots on AWGN channel: r! 6N M 3 SNR - Error probability Q - Data rate = log M N - As long as M N ⅓, the error probability 0 as N - But, the data rate! =! log N! still 0 as N 3N 9

Channel Coding Theorem For every memoryless channel, there is a definite number C that is computable such that: - If the data rate R < C, then there exists a coding scheme that can deliver rate R data over the channel with error probability 0 as the coding block length N - Conversely, if the data rate R > C, then no matter what coding scheme is used, the error probability 1 as N We shall focus on the additive white Gaussian noise (AWGN) channel - Give a heuristic argument to derive the AWGN channel capacity 10

AWGN Channel Power constraint: NX x[n] apple NP n=1 x[n] z[n] N (0, ) y[n] =x[n]+z[n] We consider real-valued Gaussian channel As mentioned earlier, repetition coding yield zero rate if the error probability is required to vanish as N Because all codewords are spread on a single dimension in an N-dimensional space How to do better? 11

Sphere Packing Interpretation y = x + z R N By the law of large numbers, as N, most y will lie p N(P + ) inside the N-dimensional sphere of radius p N(P + ) p N Also by the LLN, as N, y will lie near the surface of the N-dimensional sphere centered at x with radius p N Vanishing error probability non-overlapping spheres How many non-overlapping spheres can be packed into the large sphere? 1

Why Repetition Coding is Bad R N p N(P + ) y = x + z It only uses one dimension out of N! 13

Capacity Upper Bound R N p N(P + ) p N y = x + z Maximum # of non-overlapping spheres = Maximum # of codewords that can be reliably delivered p N(P + NR ) N apple p N N =) R apple 1 N log p N(P + ) N p N N! = 1 log 1+ P This is hence an upper bound of the capacity C. How to achieve it? 14

Achieving Capacity (1/3) (random) Encoding: randomly generate NR codewords {x1, x,...} lying inside the x-sphere of radius Decoding: Performance analysis: WLOG let x 1 is sent - By the LLN, := P P + y! MMSE! y! Nearest Neighbor! bx y x 1 = w +( 1)x 1 N +( 1) NP = N P - As long as x1 lies inside the uncertainty sphere centered at αy r with radius!!!, decoding will be correct - Pairwise error probability (see next slide) = N P P + P + p NP P + N/ 15

Achieving Capacity (/3) x-sphere r p NP N P P + x 1 y When does an error occur? Ans: when another codeword x falls inside the uncertainty sphere of αy What is that probability (pairwise error probability)? Ans: the ratio of the volume of the two spheres Pr {x 1! x } = p NP /(P + p N NP ) N x = P + N/ Union bound: Total error probability apple NR P + N/ 16

Achieving Capacity (3/3) Total error probability (by union bound) Pr {E} apple NR P + N/ = N R+ 1 log 1 1+ P!! As long as the following holds, Pr {E}! 0 as N!1 R< 1 log 1+ P Hence, indeed the capacity is C = 1 1+ log P bits per symbol time 17

Resources of AWGN Channel 18

Continuous- Time AWGN Channel System parameters: - Power constraint: P watts; Bandwidth: W Hz - Spectral density of the white Gaussian noise: N0/ Equivalent discrete-time baseband channel (complex) - 1 complex symbol = real symbols Capacity: Power constraint: NX x[n] apple NP n=1 x[n] z[n] CN(0,N 0 W ) y[n] =x[n]+z[n] C AWGN (P, W) = 1 log 1+ P/ bits per symbol time N 0 W/ SNR := P/N = W log 1+ P 0 W SNR per complex symbol bits/s = log (1 + SNR) bits/s/hz N 0 W 19

Complex AWGN Channel Capacity C AWGN (P, W) =W log 1+ P N 0 W bits/s = log (1 + SNR) bits/s/hz Spectral Efficiency The capacity formula provides a high-level way of thinking about how the performance fundamentally depends on the basic resources available in the channel No need to go into details of specific coding and modulation schemes Basic resources: power P and bandwidth W 0

Power SNR = P N 0 W log (1 + SNR) 7 6 5 4 3 1 Fix W: High SNR: - Logarithmic growth with power Low SNR: - Linear growth with power 0 0 0 40 60 80 100 SNR C = log(1 + SNR) log SNR C = log(1 + SNR) SNR log e 1

Bandwidth Fix P C(W )=W log 1+ P N 0 W W P N 0 W log e = P N 0 log e P N 0 log e 1.6 1.4 1. Power limited region 1 C(W ) (Mbps) 0.8 0.6 Capacity Limit for W 0.4 Bandwidth limited region 0. 0 0 5 10 15 0 Bandwidth W (MHz) 5 30

Bandwidth- limited vs. Power- limited C AWGN (P, W) =W log SNR = When SNR 1: (Power-limited regime) C AWGN (P, W) W P N 0 W 1+ P N 0 W - Linear in power; Insensitive to bandwidth When SNR 1: (Bandwidth-limited regime) - Logarithmic in power; Approximately linear in bandwidth P N 0 W C AWGN (P, W) W log bits/s log e = P N 0 log e P N 0 W 3

Capacity of Some LTI Gaussian Channels 4

SIMO Channel x h y y = hx + w C L # h MRC, h # ey = h x + ew, ew CN 0, Power constraint: P w CN 0, I L MRC is a lossless operation: we can generate y from ey : y = ey (h/ h ) Hence the SIMO channel capacity is equal to the capacity of the equivalent AWGN channel, which is C SIMO = log 1+ h P Power gain due to Rx beamforming 5

MISO Channel x h h = h 1 y h y = h x + w C, # Tx Beamforming x = xh/ h # y = x h + w x, h C L Power constraint: NX x apple NP n=1 Goal: maximize the received power h* x - The answer is h P! (check. Hint: Cauchy-Schwarz inequality) Achieved by Tx beamforming - Send a scalar symbol x on the direction of h - Power constraint on x : still P Capacity: C MISO = log 1+ h P 6

Frequency- Selective Channel y[m] = LX 1 h l x[m l]+w[m] l=0 Key idea 1: use OFDM to convert the channel with ISI into a bunch of parallel AWGN channels - But there is loss/overhead due to cyclic prefix Key idea : CP overhead 0 as N c First focus on finding the capacity of parallel AWGN channels of any finite Nc Then take N c to find the capacity of the frequencyselective channel 7

Recap: OFDM x [1] = d[n L + 1] y[1] x [L 1] = d[n 1] y[l 1] d 0 d[0] Cyclic prefix x [L] = d[0] Channel y[l] Remove prefix y[l] ỹ 0 IDFT DFT d N 1 d[n 1] x [N + L 1] = d[n 1] y[n + L 1] y [N + L 1] ỹ N 1 y := y[l : N c + L 1], w := w[l : N c + L 1], h := h 0 h 1 h L 1 0 0 T ey n = e h n e dn + ew n, n =0, 1,...,N c 1 Nc parallel AWGN channels ey n := DFT (y) n, e dn := DFT (d) n, ew n := DFT (w) n, e hn := p N c DFT (h) n 8

Parallel AWGN Channels e h0 ew 0 [m] Parallel Channels ed 0 [m] ey 0 [m] ey n = e h n e dn + ew n, n [0 : 1 : N c 1] ed 1 [m] e h1 ew 1 [m] ey 1 [m] Equivalent Vector Channel ey = H e d e + ew ew CN 0, eh = diag eh0,..., e h Nc 1 I... ed Nc 1[m] e hnc 1 ew Nc 1[m] ey Nc m =1,,...,M (M channel uses) 1[m] Power Constraint MX d[n] e apple MN c P n=1 Due to Parseval theorem of DFT 9

Independent Uses of Parallel Channels One way to code over such parallel channels (a special case of a vector channel): treat each channel separately - It turns out that coding across parallel channels does not help! Power allocation: - Each of the Nc channels get a portion of the total power NX - c 1 Channel n gets power Pn, which must satisfy P n apple N c P For a given power allocation {P n}, the following rate can be achieved: R = NX c 1 n=0 log 1+ e h n P n! n=0 30

Optimal Power Allocation Power allocation problem: max P 0,...,P N c 1 subject to NX c 1 n=0 NX c 1 n=0 log 1+ e h n P n It can be solved explicitly by Lagrangian methods!, P n = N c P, P n 0, n =0,...,N c 1 Final solution: let (x)+ := max(x, 0) P n = e h n! +, satisfies NX c 1 n=0 e h n! + = N c P 31

Water]illing e h n P* 1 = 0 * P * P 3 Note: e h n = H b nw N c Subcarrier n Baseband frequency response at f = nw/nc 3

Frequency- Selective Channel Capacity Final step: making N c - Replace all!!!!! by Hb(f), summation over [0 : Nc 1] becomes integration from 0 to W Power allocation problem becomes Optimal solution becomes P (f) = max P (f) Z W 0 subject to e hn = H b nw N c log H(f) 1+ H(f) P (f) Z W 0 +, df, P (f) =P, P(f) 0, f [0,W] satisfies Z W 0 H(f) + df = P 33

Water]illing over the Frequecy Spectrum H(f) 4 3.5 3.5 1.5 1 P *( f ) 0.5 0 0.4W 0.W 0 0.W 0.4W Frequency ( f ) 34

Capacity of Fading Channels 35

Flat vs. Frequency Selective Fading Frequency selectivity induces ISI converted to parallel flat fading channels Frequency selective underspread channel can be We focus on flat fading channels (single tap): E h[m] =1, 8 m Channel Gain Power constraint: NX x[n] apple NP h[m] w[m] Noise n=1 Channel Input x[m] y[m] Channel Output y[m] =h[m]x[m]+w[m] 36

Slow vs. Fast Fading Slow fading (quasi-static) - h[m] = h for all m - h is random, x[m] h[m] w[m] y[m] - h is unknown to Tx (if known to Tx same as AWGN channel!) - h is known to the Rx (if not, it is the non-coherent setting) Fast fading - h[m] = hl for all m within the l-th coherence time (Tc) period - hl : i.i.d. over l, that is, i.i.d. over different coherence time periods - hl is random, could be known or unknown to Tx - hl is known to the Rx (if not, it is the non-coherent setting) 37

Slow Fading Channel h[m] = h for all m and h is random - Conditional on h, channel capacity = log(1+ h SNR) - Tx does not know the channel h Suppose Tx send at rate R bits/s/hz: - If R < log(1+ h SNR), Shannon: Pe 0 as N - If R > log(1+ h SNR), Shannon: Pe 1 as N Total Error probability: P (N) e =Pr R<log 1+ h SNR Pr E R<log 1+ h SNR +Pr R>log 1+ h SNR Pr E R>log 1+ h SNR! Pr R<log 1+ h SNR 0 +Pr R>log 1+ h SNR 1 = Pr R>log 1+ h SNR as N!1 38

Shannon Capacity = 0 Error probability Pr{R > log(1+ h SNR)} as N Vanishing error probability: impossible due to deep fade! According to Shannon s definition, capacity = 0 Outage probability p out(r) := Pr{R > C(h ; SNR)} - Determines the reliability level of sending at a particular rate R - C(h ; SNR) = log(1+ h SNR) for the point-to-point channel 39

Outage Probability: Computation p out (R) :=Pr R>log 1+ h SNR =Pr h < ( R 1)SNR 1 pdf of log(1+ h SNR) 0.45 0.4 0.35 0.3 0.5 0. 0.15 0.1 0.05 =1 e (R 1) SNR Area = p out (R) for h CN(0, 1): Rayleigh fading 0 0 R 1 3 4 5 log(1+ h SNR) 40

Outage Capacity Shannon capacity of a slow fading channel is 0 - Because we insist that Pe 0 as N! More realistic capacity measure: - Set a reliability level ϵ - Find the maximum rate R * such that Pout(R) ϵ ϵ-outage capacity: C := max {R p out (R) apple } It s just the inverse function of outage probability! (why?) C = log 1+F 1 (1 ) SNR F (x) :=Pr h >x : complementary CDF of h 41

Outage Capacity: Computation p out (C )= () Pr C > log 1+ h SNR = () Pr C 1 SNR 1 > h = () 1 F C 1 SNR 1 = () C 1=F 1 (1 ) SNR () C = log 1+F 1 (1 ) SNR For Rayleigh fading: C log (1 + SNR) when 1 h CN(0, 1) : y = F (x) =e x [0, 1], F 1 (y) = ln y 0 F 1 (1 ) = ln(1 ), for 1 4

Fade Margin AWGN vs. ϵ-outage capacity: C AWGN = log (1 + SNR) C = log 1+F 1 (1 ) SNR Typically less than 1 because ϵ 10% Extra Tx power improve deep fade Under a reliability level ϵ, to achieve the same capacity as the AWGN channel, in slow fading channel the Tx has to use an extra 10log10(1/F 1 (1-ϵ)) db of power Fade margin: the extra amount of Tx power to improve the system when channel is in deep fade 43

Impact of Fading High SNR: C log F 1 (1 )SNR C AWGN log - Additive loss - Smaller when SNR 1 F 1 (1 ) Low SNR: C F 1 (1 )SNR log e F 1 (1 )C AWGN - Multiplicative loss - For Rayleigh, F 1 (1 ) = ln(1 ), for 1 - Significant loss when SNR is small Impact of fading is also significant at low SNR 44

AWGN Capactiy vs Outage Capacity C C AWGN 1 0.8 0.6 = 0.1 0.4 0. = 0.01 0 10 5 0 5 10 15 0 5 30 SNR (db) 35 40 45

Diversity Order Recall: - Error probability Outage probability pout(r) as N Recall from Lecture : - For uncoded transmission and some coding scheme, we see that Error probability ~ SNR 1 for some fixed N - Is this true for the optimal coding scheme and arbitrarily large N? Outage probability at high SNR: p out (R) =1 e (R 1) SNR R 1 SNR when SNR 1 - Even for optimal coding scheme and large N, error prob. ~ SNR 1 Optimal diversity order is 1 46

Optimal Diversity Order and pout(r) Hence we are able to define the optimal diversity order for point-to-point slow fading channels: d := lim SNR!1 log p out (R) log SNR where p out (R) :=Pr{C (h; SNR) <R} Note: in taking the limit, we assume that R is a constant This view will be modified in later lectures when we discuss the diversity-multiplexing tradeoff 47

Receive Diversity x h y y = hx + w C L P Power constraint: P SNR := w CN 0, I L E h = L C (h; SNR) = log 1+ h SNR Outage probability Outage capacity p out (R) :=Pr log 1+ h SNR <R =Pr h < R 1 SNR Lecture Slide #5 (R C log L! 1) L SNR L = p out (C ) 1+(L!) 1 L ( ) 1 L SNR for Rayleigh faded h s at high SNR 48

SIMO Outage Capacity C C SIMO 1 = 1% C SIMO = log 1+ h SNR = log (1 + LSNR) 0.8 L = 5 L = 4 L = 3 0.6 L = 0.4 0. L = 1 0 10 5 0 5 10 15 0 5 30 35 40 SNR (db) 49

Transmit Diversity x h h = h 1 y h y = h x + w C, # Tx Beamforming x = xh/ h # y = x h + w x, h C L Power constraint: NX x apple NP n=1 SNR := P E h = L Tx beamforming is impossible since Tx does not know h - For SIMO, Tx does not need to know h to achieve C(h ; SNR) How to find the optimal outage probability? For i.i.d. Rayleigh fading, it can be shown (cf. Appendix B.8 and Exercise 5.15, 5.16) that the optimal outage probability p out (R) =Pr log 1+ h SNR L <R 50

Impact of CSIT: Loss in Power Gain Comparison of outage probability p Tx out(r) =Pr log 1+ h SNR L <R p Rx out(r) =Pr log 1+ h SNR <R - The same diversity order L - SIMO has L-fold power gain over MISO Lack of channel state information at the transmitter (CSIT) Loss in power gain 51

Repetition Coding u Time 1 Time h 1 h 1 h u h Equivalent Channel: apple apple y1 h1 = u + y h apple w1 w Projection h! ey = y = h u + ew h Supports rates up to 1 log 1+ h SNR Outage probability: p Repetition out (R) =Pr blocks, but just 1 scalar channel 1 log 1+ h SNR <R 5

Alamouti Scheme u 1 Time 1 Time h 1 u h 1 u h u 1 h u 1,u C Equivalent Channel: X = apple u1 u u u 1 space-time codeword eh 1 e h e h1? e h apple y1 y = apple h1 h h h 1 apple u1 u + apple w1 w = u 1 apple h1 h + u apple h h 1 + apple w1 w Projection onto the two column vectors respectively, we can get two clean channels for u1 and u! 53

Performance of Alamouti Scheme apple y1 y = u 1 apple h1 h + u apple h h 1 + apple w1 w ey = u h1 e? h e 1h1 e + u h e + ew Projection onto two orthogonal directions ey 1 := e h 1 e h 1 ey = u 1 e h 1 + ew 1 = u 1 h + ew 1 ey := e h e h ey = u e h + ew = u h + ew Power allocation: Lack of CSIT No idea which channel is better Uniform power allocation (P/ each) Each channel supports rates up to Outage probability: achieves optimal p Alamouti out (R) =Pr log log 1+ h SNR Two parallel channels, each for one symbol! 1+ h SNR <R 54

Repetition Coding vs. Alamouti Repetition Alamouti C(h;SNR) 1 log 1+ h SNR log 1+ h SNR pout(r) when SNR 1 Cϵ for ϵ 1 ( R 1) SNR 1 log 1+ p SNR ( R 1) SNR log 1+ r SNR Diversity Order = 55

Time and Frequency Diversity Recall from Lecture : - Time diversity is obtained by coding + interleaving across multiple (L) coherence time - Frequency diversity is obtained by coding + hopping across multiple (L) coherence bandwidth Hence, time and frequency diversity techniques are equivalent to coding over L parallel channels: y l [m] =h l x l [m]+w l [m], Channel l has power constraint P l, P1+P+ +PL LP, where P is the power constraint of the original channel No CSIT cannot do water-filling l =1,,...,L 56

Outage Probability Instead, first use uniform power allocation P l = P, l Given h and SNR, L parallel channels can support up to LX l=1 Subtlety: due to lack of CSIT, coding across parallel channel is necessary log 1+ h l SNR bits/s/hz - Because Tx does not know for each of the L channels, how high the rate should be! ( L ) X Outage probability: p out (R) =Pr log 1+ h l SNR <LR For i.i.d. Rayleigh fading, it can be shown (cf. Exercise 5.17) l=1 that uniform power allocation is optimal! 57

Fast Fading Channel without CSIT Block fading model: - h[m] = hl for all m within the l-th coherence time (Tc) period - hl : i.i.d. over l, that is, i.i.d. over different coherence time periods If 1 T c N (comm. time) - L parallel channels; L - For finite L, capacity = 0 because C (h; SNR) := 1 LX log 1+ h l SNR L - is random l=1 But as L, C(h;SNR) x[m] h[m] l = 0 h[m] w[m] l = 1 l = l = 3 E log 1+ h SNR y[m] m 58

Ergodic Capacity What if T c 1 does not hold? - In particular, y[m] = h[m]x[m] + w[m], {h[m] m [1:N]}: i.i.d. It turns out that the capacity of such channel is C = lim N!1 1 N NX m=1 log 1+ h[m] SNR By LLN, we get C = E log 1+ h SNR In fact, for any fading process is stationary and ergodic, the capacity of the fading channel is given by the above - Stationary and ergodic the long-term average (over time) converges to the expectation (under the stationary distribution) - Note: for slow fading channel, the fading process is not ergodic 59

Ergodic Capacity vs. AWGN Capacity C AWGN = log (1 + SNR), C CSIR = E log 1+ h SNR In the fast fading channel, if channel state information is available only at Rx, then CCSIR CAWGN - Due to Jensen s inequality and the concavity of log(): E log 1+ h SNR apple log 1+E h SNR = log (1 + SNR) Even if Tx can code over multiple coherence time periods and average out the fluctuation of the fading channel, the capacity gets hit by fading With CSIT, the situation is changed 60

Transmitter State Information So far we assume that Tx does not know the realization of the fading coefficients (channel state information) Such assumption might be too conservative How to obtain CSI at Tx (CSIT)? - TDD system: channel reciprocity - FDD system: feedback from Rx How to use CSIT? - Slow fading channel: Channel Inversion Control Tx power according to the current channel condition so that a fixed data rate can be supported May not be feasible because many systems are also peak power constrained - Fast fading channel: Water Filling 61

Fast Fading Channel with CSIT (1) Idea: with CSIT, Tx should allocate more power when the channel is in good state! L parallel AWGN channels - Power allocation problem 1 max P 1,...,P L L LX log l=1 - Solution: 1+ h l P l P l =, subject to 1 L h l LX P l = P, P l 0, l =1,...,L - The value of ν depends on all L channels {h1,, hl} - Not feasible because it requires the Tx to know the future channel beforehand! l=1 +, satisfies 1 L LX l=1 h l + = P 6

Fast Fading Channel with CSIT () Final step: taking L - Due to LLN, optimal solution becomes: P (h) = h +, satisfies E " h + # = P - Now the value of ν only depends on the distribution of h Ergodic capacity of fast fading channel with CSIT: C = E apple log 1+ h P (h) 63

Power Allocation at High SNR Optimal Uniform: Near-Optimal h[m] h[m] m m 64

Power Allocation at Low SNR Optimal Best Only: Near-Optimal h[m] h[m] m m 65

Performance Comparison 7 6 CAWGN > CFull CSI CCSIR C (bits /s / Hz) 5 4 3 1 AWGN Full CSI CSIR CFull CSI > CAWGN CCSIR C AWGN = log 0 0 15 10 1+ P apple, C CSIR = E log 5 0 5 10 15 SNR (db) 1+ h P, E h =1 0 C Full CSI = E apple log 1+ h P (h) P (h) = h +, satisfies E " h + # = P 66

Performance at Low and High SNR High SNR regime: - The optimal power allocation is approximately uniform apple - Hence, C Full CSI C CSIR E log h P = log SNR + E log( h ) C AWGN 0.83 Low SNR regime: - The optimal power allocation is approximately put all the power in the best channel - Hence, C Full CSI Pr h = G max log 1+G max P/Pr h = G max G max SNR log e G max C AWGN! - Beside, C CSIR E h SNR log e = E h SNR log e = SNR log e C AWGN 67

Summary A slow fading channel is a source of unreliability - Poor outage capacity - Diversity is needed A fast fading channel with only CSIR: - Ergodic capacity close to AWGN capacity - Decoding delay is long compared to coherence time A fast fading channel with full CSI: - Ergodic capacity can be greater than AWGN capacity - Turn fading into a friend rather than an enemy Opportunistic communication - Send more when the channel is good - Even more powerful in multiuser situations 68