Advanced Topics in Digital Communications Spezielle Methoden der digitalen Datenübertragung Dr.-Ing. Carsten Bockelmann Institute for Telecommunications and High-Frequency Techniques Department of Communications Engineering Room: N360, Phone: 041/18-6386 bockelmann@ant.uni-bremen.de Lecture Thursday, 10:00 1:00 in N150 Exercise Wednesday, 14:00 16:00 in N40 Dates for exercises will be announced during lectures. Tutor Tobias Monsees Room: N410 Phone 18-6407 tmonsees@ant.uni-bremen.de www.ant.uni-bremen.de/courses/atdc/
Outline Part 1: Linear Algebra Eigenvalues and eigenvectors, pseudo inverse Decompositions (QR, unitary matrices, singular value, Cholesky ) Part : Basics and Preliminaries Motivating systems with Multiple Inputs and Multiple Outputs (multiple access techniques) General classification and description of MIMO systems (SIMO, MISO, MIMO) Mobile Radio Channel Repetition of IT basics, channel capacity for SISO AWGN channel Extension to SISO fading channels Generalization for the MIMO case Part 4: Multiple Antenna Systems SIMO: diversity gain, beamforming at receiver MISO: space-time coding, beamforming at transmitter MIMO: BLAST with detection strategies Influence of channel (correlation) Part 5: Relaying Systems Basic relaying structures Relaying protocols and exemplary configurations Outline
Outline Part 6: In Network Processing Part 7: Compressive Sensing Motivating Sampling below Nyquist Reconstruction principles and algorithms Applications Outline 3
Information Theory for MIMO Systems Repetition: Information Theory for SISO Entropy, joint entropy, mutual information Channel capacity and channel capacity theorem Continuously Distributed Signals Capacity for AWGN Channel Extension: Channel Capacity for SISO Fading Channels Ergodic capacity, outage probability, outage capacity Generalization of Information Theory for MIMO Systems Multivariate distributions, entropies for MIMO systems Mutual information without CSI at transmitter Mutual information with perfect CSI at transmitter water filling Performance evaluation 4
Repetition: Information Theory for SISO Systems
Information and Entropy for Discrete Signals Discrete alphabet X with elements X ν : Pr{x=X ν }=Pr{X ν } Amount of information per digit: I(X ν )= log Pr{X ν } = log Pr{X ν } Entropy is average information of whole set H(x) =E{ log Pr{x}} = X Pr{X ν } log Pr{x} ν Joint Entropy: H(x, y) = E{log Pr{x, y}} = X ν Mutual information: X Pr{X ν,y μ } log Pr{X ν,y μ } I(x; y) =H(x) H(x y) =H(y) H(y x) =H(x)+H(y) H(x, y) = X X Pr{X ν Y μ } Pr{Y μ X ν } Pr{X ν } log P ν μ l Pr{Y μ X l } Pr{X l } μ Mutual entropy of source and sink H(x) represents uncertainty about x, H(x y) represents uncertainty about x after y is received H(x) - H(x y) = amount of information provided about x by y 6
Channel capacity Channel Capacity Maximum of mutual information over all possible input statistics Pr{x=X ν } C = sup I(x; y) Pr{x} Channel coding theorem bits per transmission bits/s/hz At least one code of rate R c C exists for which an error-free transmission can be ensured this assumes perfect decoding and the code length may be arbitrary long For R c > C it can be shown that error-free transmission is impossible! 7
Repetition for Continuously Distributed Signals Differential entropy of continuously Z distributed signal: H(x) =E{ log p(x)} = p(x) log p(x)dx Entropy of Gaussian distributed signal with power Real Gaussian distribution: 1 p(x) = p exp x πσ x σ x σ x = E s T s H(x) = 1 log (πeσ x ) Complex Gaussian distribution: p(x) = 1 exp x πσx σx H(x) = log (πeσ x) Complex inputs can contain twice as much information as real inputs However, the power is equally distributed between real and imaginary part (σ x σ x) 8
Channel Capacity of SISO Systems AWGN channel with Gaussian distributed input and output n [k] d [i] x [k] y [k] ˆd [i] FEC encoder FEC decoder H(x y) (x) I(x;y) H(y) H(y x) 9
Repetition of Capacity for AWGN Channel Mutual information for AWGN: y[k] = x[k] + n[k] I(x; y) =H(y) H(y x) =H(y) H(n) x and n are Gaussian distributed and statistically independent, i.e. y is Gaussian distributed with σy = σx + σn Real valued signals: H(y) = 1 log (πe(σ x + σ n 0 )) Mutual information with σ n 0 = σ n = N 0/ T s and σ x = E s T s σ x + σ n 0 I(x; y) = 1 log (πe(σx + σn )) 1 0 log (πeσn )= 1 0 log =log 1+ σ x σn 0 C 1 dim = 1 log 1+ E s N 0 σ n 0 10
Repetition for Continuously Distributed Signals Complex valued signals: H(y) =log (πe(σ x + σ n)) Mutual information with σ n = N 0 T s I(x; y) =log (πe(σx + σn)) log (πeσn) σ =log x + σ n =log σn 1+ σ x σn C dim =log 1+ E s N 0 and σ x = E s T s Complex inputs can transmit twice as much information as real inputs However, the signal to noise ratio is halved Since log grows less than linearly complex signals superior to real signals 11
Capacity of AWGN Channel C 4 3 BPSK QPSK 8-PSK 16-PSK 16-QAM Gaussian C 4 3 BPSK QPSK 8-PSK 16-PSK 16-QAM 7 db 1 1 0-10 0 10 0 E / N in db s 0 0-5 0 5 10 15 E / N in db b 0 1
Always Gaussian distributed input assumed Extension: Channel Capacity for SISO Fading Channels
Channel Capacity of SISO Fading Channels Flat Rayleigh fading channel with Gaussian input h [k] n [k] d [i] FEC x [k] y [k] ˆd [i] encoder FEC decoder Signal power at receiver now influenced by channel coefficient h[k] Extension of results from AWGN channel for perfect Channel State Information (CSI) at receiver C dim (h) =log 1+ h Es N 0 Channel capacity is random variable due to dependence on h[k] Rayleigh fading: h[k] is chi-squared distributed with degrees of freedom p h (ξ) = 1 σh exp ξ σh 14
Extension for SISO Fading Channels Ergodic capacity obtained by averaging with respect to h[k] : Z c = E h {C(h)} = log 1+ξ E 0 1 0 N 0 σh e ξ σ h dξ 1 1 =log (e) exp σh E expint s/n 0 σh E s/n 0 Outage Probability: probability that a certain capacity cannot be reached Rayleigh fading channel ½ ¾ P out =Pr{C(h) <C out } =Pr h < C out 1 1 C out =1 exp E s /N 0 σh E s/n 0 Outage Capacity: capacity that cannot be reached to a certain percentage Rayleigh fading channel C out =log 1 σh Es ln(1 P out ) N 0 with expint = Z x e t /t dt 15
Capacity of Rayleigh Fading Channel C x denotes the capacity that cannot be ensured for x% of all channels (outage probability of x%) 10 8 AWGN For high SNR, large capacities can be achieved with low outage probability. P out grows rapidly with decreasing SNR P out of ergodic capacity is close to 50% 1 0.8 0 db 5 db C 6 4 0 C erg C 1 C 5 C 10 C 0 C 50 0 5 10 15 0 5 30 E / N in db b 0 P out 0.6 0.4 0. 10 db 15 db 0 db 5 db 30 db P out (C erg ) 0 0 4 6 8 10 R 16
Always Gaussian distributed input assumed Generalization of Information Theory for MIMO Systems
Multivariate Distributions Distribution of vectors equals joint distribution of vector elements Described by multivariate distributions Multivariate Gaussian density for real valued process x (n elements) 1 p(x) =p(x 1 x n )= p with det(πφx ) exp xt Φ 1 x x Φ x =E x x T ª x is diagonal for independent variables with diagonal elements 1 p(x) = s exp x 1 nq σ x n ny 1 πσx x 1 σx = p exp x i n i=1 πσ xi σx i i i=1 Multivariate Gaussian density for complex valued process x (n elements) p(x) = 1 det(πφ x ) exp x H Φ 1 x x with xi Φ x =E x x Hª = ny p(x i ) i=1 18
Entropies for MIMO Systems Definition of differential entropy for complex vectors equivalent to scalar signals H(x) = E{log p(x)} =log (det(πeφ x )) Proof ¾ 1 H(x) = E{log p(x)} = E ½log det(πφ x ) exp( xh Φ 1 x x) =log (det(πφ x )) + 1 ln E x H Φ 1 x x ª =log (det(πφ x )) + log e tr Φ 1 x E{xx H } =log (det(πφ x )) + n log e Linear equation system with N T inputs and N R outputs: y = H x + n Since x and n are Gaussian, y is also Gaussian with covariance matrix Φ y =E{yy H } =E (Hx + n)(hx + n) Hª o =E nhxx H H H + Hxn H + nx H H H + nn H = HΦ x H H + Φ n 19
Entropies for MIMO Systems Perfect Channel State Information at receiver (CSIR, CSI@R) means perfect knowledge of H: I(x; y H) =H(y H) H(n) Conditional differential entropy of y H(y H) = E{log [p(y H)]} =log[det(πeφ y )] = log (πe) N R det(hφ x H H + Φ n ) Noise is a stationary process and samples in n are statistically independent n is diagonal with identical diagonal elements σ n : Φ n = σ n I NR Mutual information between x and y for known H I(x; y H) =log (πe) N R det(hφ x H H + Φ n ) log (πe) N R det(φ n ) =log det =log det(hφx H H + Φ n ) det(φ n ) I NR + 1 σn HΦ x H H 0
Mutual Information without CSI at Transmitter Two cases have to be distinguished: No CSI at transmitter available Some kind of CSI available at transmitter No CSI at transmitter available transmission of independent data streams over N T inputs is optimum covariance matrix of x has diagonal structure: Φx x I NT I(x; y H) =log det I NR + σ x HH H σn Singular value decomposition (SVD) of system matrix with rank r: U: unitary N R x N R matrix : diagonal N R x N T matrix with r nonzero singular values i of H on diagonal (eigenvalues i of HH H are the squared singular values of H) V: unitary N T x N T matrix Σ = H = UΣV H Σ 1 0..... 0 Σ r 0 1
Mutual Information without CSI at Transmitter Mutual information obtained by summing mutual information of r equivalent (virtual) parallel SISO systems I(x; y H) =log det =log det =log " ry i=1 I NR + σ x σ n I NR + σ x σ n 1+Σ i σ x σ n UΣV H VΣ H U H ΣΣ H # = =log det =log det ½ diag 1+Σ σx 1 The higher the rank r of H, the higher the number of parallel virtual channels SIMO: H is column vector and has rank 1 MISO: H is row vector and has rank 1 MIMO: H has full rank r = min(n R, N T ) MIMO: H has reduced rank r < min(n R, N T ) fewer eigenvalues of H are nonzero Capacity is multiplied by factor r compared to SISO system rx i=1 log 1+λ i ES N 0 U I NR + σ x σn ΣΣ H U H σ n,, 1+Σ r σx ¾ σn
Mutual Information with perfect CSI at Transmitter If transmitter has access to perfect CSI (CSIT, CSI@T), x has to be adapted to CSI in order to achieve capacity I(x; y H) =log det I NR + 1 HΦ σn x H H =log det I NR + 1 UΣV H V σn x Σ xvx H VΣ H U H =log det I NR + 1 ΣV H V σn x Σ xvx H VΣ H The maximum mutual information is obtained by choosing V x = V, i.e. the eigenvectors of x are identical to right singular vectors of H (we exploit the eigenmodes of the channel described by H) I(x; y H) =log det I NR + 1 σn ΣΣ xσ H =log " ry i=1 1+Σ i σ x i σ n Open question: How to distribute power on virtual channels? Φ x = V x Σ xv H x # = rx i=1 log 1+λ i Es,i N 0 3
Mutual Information with perfect CSI at Transmitter Task: Optimize transmit power per sub channel in order to maximize the total throughput, i.e max E s,1,,e s,r rx i=1 log 1+λ i Es,i N 0 Solution by Lagrange method Lagrange function rx à F = log 1+λ i Es,i + ξ P total N 0 i=1 subject to E s,i 0and rx E s,i P total Setting derivation of F w.r.t. E s,i equal to zero leads to maximum of F i=1! rx E s,i F =0 for i =1,,r E E s,i + N 0 = θ =const. for i =1,,r s,i λ i E s,i = θ N ( + 0 with (a) + a a 0 = λ i 0 else i=1 x 1 x r Σ 1 Σ r ñ 1 ñ r ỹ 1 ỹ r 4
Mutual Information with perfect CSI at Transmitter Water filling approach: θ Vessel with bumpy ground is filled with limited amount of water (water amount represents total transmit power) Height of ground represents the noise floor/attenuation of virtual channels High water columns over low ground levels, i.e. spend much power on good channels and low or even no power on bad E s,1 E s, E s,4 N 0 λ 1 N 0 λ E s,5 N 0 λ 3 N 0 λ 4 N 0 λ 5 channels Layer Water filling solution obtained by Lagrange method Karush-Kuhn-Tucker conditions have to be fulfilled Spending much energy on good channels and few energy on bad channels leads to capacity! This implies high spectral efficiency on good channels 5
Comparison of Ergodic Capacities C erg in bit / channel use 35 30 5 0 15 10 5 1x1 SISO 1x4 SIMO 4x1 MISO w/o CSIT 4x1 MISO with CSIT 4x4 MIMO w/o CSIT 4x4 MIMO with CSIT 0-10 0 10 0 30 E / N in db s 0 SIMO Diversity gain Antenna gain MISO Diversity gain Antenna gain only with CSIT MIMO Multiplexing gain Diversity gain Receive antenna gain CSIT only useful at small SNR 6
Comparison of 1%-Outage Capacities C 1 in bit / channel use 30 5 0 15 10 5 1x1 SISO 1x4 SIMO 4x1 MISO w/o CSIT 4x1 MISO with CSIT 4x4 MIMO w/o CSIT 4x4 MIMO with CSIT Relatively small difference to ergodic capacity for SIMO, MISO, and especially for MIMO 1%-outage capacity for SISO much smaller than ergodic capacity MISO also useful without CSIT 0-10 0 10 0 30 E / N in db s 0 Diversity reduces probability of bad channel realizations with very low capacity 7
10 0 10-1 Comparison of Outage Probabilities for High SNR Signal to noise ratio fixed at E s / N 0 = 0 db MIMO achieves highest rate for any given outage probability P out 10-10 -3 10-4 1x1 SISO 1x4 SIMO 4x1 MISO w/o CSIT 4x1 MISO with CSIT 4x4 MIMO w/o CSIT 4x4 MIMO with CSIT 0 10 0 30 R in bit / channel use SIMO and MISO with CSIT are better than SISO for all outage probabilities MISO without CSIT is better than SISO for reasonable outage probabilities, but worse for very high ones 8
10 0 Comparison of Outage Probabilities for Low SNR Signal to noise ratio fixed at E s / N 0 = -10 db P out 10-1 10-10 -3 10-4 1x1 SISO 1x4 SIMO 4x1 MISO w/o CSIT 4x1 MISO with CSIT 4x4 MIMO w/o CSIT 4x4 MIMO with CSIT 0 0.5 1 1.5 R in bit / channel use MIMO with CSIT: Transmit power is usually focussed on strongest subchannel Antenna gain becomes more important than diversity and multiplexing gain 9
Matlab Demo for ULA Matlab-Demo for Uniform Linear Array (ULA) Line of sight (no fading) Channel is not singular due to phase difference Special cases of antenna orientation 6 y in m 1 0.5 0 antenna setup -0.5 0 0.5 x in m singular value profile Broadside Endfire 4 Investigation of Singular values Capacity 0 0 0.5 1 1.5.5 3 3.5 4 4.5 5 0 15 10 capacity 5 0 0 50 100 150 00 50 300 350 30
Paper: Selected Literature C. E. Shannon: A mathematical theory of communication, Bell System Technical Journal, 1948 E. Telatar: Capacity of multi-antenna Gaussian channels. European Transactions on Telecommunications, 1999 Online books: D. Tse, P. Viswanath: Fundamentals of Wireless Communication, Cambridge, 005 Printed books: V. Kühn: Wireless Communications over MIMO Channels, Wiley, 006 A. Goldsmith: Wireless Communications, Cambridge, 005 31