Painlevé Transcendents and the Information Theory of MIMO Systems

Transcription

1 Painlevé Transcendents and the Information Theory of MIMO Systems Yang Chen Department of Mathematics Imperial College, London [Joint with Matthew R. McKay, Department of Elec. and Comp. Engineering, HKUST] 1

2 Presentation Outline MIMO Outage Probability and Problem Statement New Tools from RMT and Physics: Hankel Determinants and Orthogonal Polynomials Exact Painlevé Representation for Capacity Generating Function MIMO Outage Probability: A (Refined) Large-System Analysis Numerical Studies and Conclusions 2

3 MIMO Outage Probability & Problem Statement Ergodic Capacity: Average mutual information between Tx and Rx Studied extensively for over more than a decade Non-asymptotic [Telatar, Shin, Lozano, Chiani, Smith, Kiessling, Kang, Alfano,...] Asymptotic [Telatar, Chuah, Verdu, Tse, Debbah, Rapajic, Moustakas, Tulino,...] Outage Capacity: Require entire distribution of mutual information Also studied extensively, but less understood than the ergodic capacity Non-asymptotic (complicated!): [Smith, Wang, McKay, Salo, Lozano, Shin,...] Asymptotic Large number of antennas: Gaussian distribution [Hochwald, Moustakas, Hachem,...] Large SNR: Diversity-multiplexing-tradeoff [Zheng Tse] 3

4 MIMO Outage Probability & Problem Statement Existing techniques: Classical RMT (finite distributional analysis) Steiltjes Transform Replica Method Free Probability Large Deviations Issues: Finite analysis, whilst exact, is often too complicated to give insight Asymptotic analysis is often too asymptotic to fully describe the distribution behavior 4

5 A Simple Example: MIMO Capacity Distribution (2 2) P = 5 db Simulation Gaussian Approx. (Coulomb) PDF P = 15 db P = 30 db Strong deviation from Gaussian as P grows! New approaches are needed to give a more refined asymptotic analysis (i.e., asymptotic distributions with corrections) Normalized Mutual Information (nats/s/hz/antenna) 5

6 MIMO Outage Probability & Problem Statement Outage probability corresponding to an outage capacityc out : P out (C out ) = Pr ( log det (I nr + Pnt HH ) < C out ) Assumptions: Tx signal CN (0, Pnt I nt ), Noise signal CN (0,I nr ) Channel H is IID Rayleigh fading, known at Rx but not Tx Convenient to deal with the moment generating function (MGF): M(λ) = E H [ I nr + 1 t HH λ ] = E {x i } n i=1 are the eigenvalues ofhh (complex Wishart) t = n t /P n = min{n t,n r }, m = max{n t,n r } [ n i=1 ( 1+ 1 ) ] λ t x i 6

7 MIMO Outage Probability & Problem Statement Using the joint PDF of the eigenvalues{x i }, the MGF is evaluated as: Here M(λ) t nλ R n + n (x i x j ) 2 i<j = t nλ D n [w] D n [w] = 0 k=1 x i+j w(x,t,λ)dx (x k +t) λ x m n k e x k dx k n 1 i,j=0 is a Hankel determinant generated from the weight function: w(x,t,λ) = (x+t) λ x m n e x 7

8 MIMO Outage Probability & Problem Statement Key Problem: It is very difficult to extract meaningful insight from the Hankel determinant representation of the MGF Impact of the number of antennas? Impact of the SNR? What about if BOTH the SNR and the number of antennas become large?... Technical Challenge: Require a more convenient representation for the Hankel determinant! 8

9 MIMO Outage Probability & Problem Statement Methods for Computing the Hankel Determinant (used in physics): 1. Operator Theory [Tracy Widom,...] 2. Integrable Systems [Kanzeiper Ozipov,...] 3. Reimann-Hilbert [Its,...] 4. Orthogonal Polynomials and Ladder Operators [Magnus 95, Chen Manning 94, Dai Zhang 2010,...] These methods are common in physics, but not in communication theory Our focus: Orthogonal polynomial approach since: 1. it is a relatively simple systematic theory (involving basic algebra) 2. it directly gives the simplest solution 9

10 Presentation Outline MIMO Outage Probability and Problem Statement New Tools from RMT and Physics: Hankel Determinants and Orthogonal Polynomials Exact Painlevé Representation for Capacity Generating Function MIMO Outage Probability: A (Refined) Large-System Analysis Numerical Studies and Conclusions 10

11 Hankel Determinants and Orthogonal Polynomials Question: Why the connection to orthogonal polynomials? Recall the integral representation of the Hankel determinant: D n [w] = R n + Very important observation (trick): n (x i x j ) 2 i<j k=1 (x k +t) λ x m n k e x k dx k (x i x j ) = x i 1 n j = Pi 1 (x i,j=1 j ) n i,j=1 i<j P i 1 (x j ) can be ANY arbitrary monic polynomial of degreei 1 E.g.,P i 1 (x) = x i 1 +a i 2 x i 2 +a i 3 x i 3 + +a 0 Hankel determinant can then be expressed as: D n [w] = P i (x)p j (x)w(x)dx R + n 1 i,j=0 11

12 Hankel Determinant and Orthogonal Polynomials If we can compute the polynomialsp k (x),k = 0,1,2,... which are orthogonal with respect to w(x), i.e., R + P i (x)p j (x)w(x)dx = h i δ i,j, i,j = 0,1,2,... then the Hankel determinant collapses to D n [w] = n 1 k=0 h k The problem of computing the MGF of the MIMO capacity is thus transformed into one of characterizing a specific class of orthogonal polynomials This is non-trivial, since the (deformed Laguerre) weight function is non standard 12

13 Hankel Determinants and Orthogonal Polynomials Characterizing the desired class of orthogonal polynomials (high level overview) Exploit basic relations from the general theory of orthogonal polynomials Recurrence relations Orthogonality conditions Ladder operators and their supplementary conditions Substitute the specific weight function into these formulas and apply a systematic algebraic procedure (coefficient matching, derivatives, integrals, etc) 13

14 Presentation Outline MIMO Outage Probability and Problem Statement New Tools from RMT and Physics: Hankel Determinants and Orthogonal Polynomials Exact Painlevé Representation for Capacity Generating Function MIMO Outage Probability: A (Refined) Large-System Analysis Numerical Studies and Conclusions 14

15 Exact Painlevé Representation for Capacity MGF Theorem: The log-derivative of the Hankel determinantd n [w] satisfies: t d dt logd n[w] = H n (t), whereh n (t) satisfies the Painlevé V continuous Jimbo-Miwa-Okamotoσ form: (th n) 2 = [ th n H n +H n(n+m+λ)+nλ ] 2 4(tH n H n +n(m+λ)) [ (H n) 2 +λh n ] With this, the MGF of the MIMO capacity is computed as M(λ) = e t Hn(x) nλ x This key result is a remarkably simple EXACT characterization of the MGF in terms of a classical Painlevé differential equation dx 15

16 Exact Painlevé Representation for Capacity MGF IMPORTANT QUESTIONS: How can we use this representation? What insight does it give us into the capacity distribution? Scaling? 16

17 Presentation Outline MIMO Outage Probability and Problem Statement New Tools from RMT and Physics: Hankel Determinants and Orthogonal Polynomials Exact Painlevé Representation for Capacity Generating Function MIMO Outage Probability: A (Refined) Large-System Analysis Numerical Studies and Conclusions 17

18 MIMO Outage Probability: A (Refined) Asymptotic Analysis DefineG n (t) := H n (t) nλ Focusing onn t = n r case, the Theorem becomes ( M(λ) = exp withg n the solution to: Objectives: t G n (x) x ) dx (tg n) 2 = ( G n(t+2n+λ) G n ) 2 4(tG n G n +n 2 )(G 2 n +λg n) (E0) Capture the effect of the number of antennas Capture the effect of SNR Examine the Gaussianity of the outage probability Methods: Power series expansion method: valid for small SNR General method: valid for arbitrary SNR 18

19 MIMO Outage Probability: A (Refined) Asymptotic Analysis Power series expansion solutions 1. Assume thatg n (t) has a formal power series expansion in1/t (equivalently,p/n): b k (λ) G n (t) = t k k=1 2. Substitute power series into the Painlevé, (E0), and solve forb k s. 3. RearrangeG n (t) into the following form: G n (t) = λg 1 (t)+λ 2 g 2 (t)+λ 3 g 3 (t) where theg i (t) s are each a power series in1/t 4. IntegrateG n (t) according to the theorem: logm(λ) = P and match coefficients withlogm(λ) = l=1 κ l λl l! 0 G n (n/y) y dy, to obtain thel-th cumulant,κ l 19

20 MIMO Outage Probability: A (Refined) Asymptotic Analysis Power series expansion solutions κ 1 = n + 1 n ( P P P3 7 ) 2 P4 + ( 1 3 P3 5 2 P ) ( ) 1 5 P5 + +O, n 3 κ 2 = P 2 4P P P ( 5 n 2 2 P4 32P ) 3 P6 + κ 3 = 1 n +O ( ) 1 n 4 ( 2P 3 18P P 5 + )+ 1 n 3 ( 18P 5 350P 6 + )+O( 1 n 5, ). Note the structure of the cumulants: κ l = O(n 2 l ), etc. 20

21 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: Mean Basic idea: Substitute the generic series G n (t) = λg 1 (t)+λ 2 g 2 (t)+λ 3 g 3 (t) into (E0) and obtain a set of equations forg i (t),i = 1,2,... Use the structure ofg i (t) s obtained via the power series solutions, and solve the general expressions for each cumulant The equation involvingg 1 (t): (g 1 ) 2 4n 2 g 1 2(t+2n)g 1 g 1 +(t 2 +4nt)(g 1) 2 t 2 (g 1) 2 = 0 (E1) Note that the solution to this leads to the EXACT expression for the mean capacity Substituting the generic seriesg 1 (nt) = n 1 k=0 Y n 2k k (T), wheret = 1/P, and comparing the coefficients ofn 2k 1,Y k (T) can be solved in closed-form systematically. 21

22 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: Mean A remarkable phenomenon: Once they 0 (T) is computed and substituted back into the Painlevé, all of the equations for the higher order termsy k (T),T > 0, collapse to simple ALGEBRAIC equations. This, in turn, allows us to compute the higher correction terms recursively! Recursion equation fory k : Y k (T) = T 2 Y 0 Y k 1 + k 1 l=1 [ Y l Y k l 2(T + 2)Y l Y k l +T(T + 4)Y l Y k l T2 Y 2 ( Y 0 (T + 2) Y ) 0 l Y k 1 l ] Examples: Y 0 (T) = 4 + T T(4 + T) 4 + T + T(4 + T), Y 1 1(T) = T(4 + T) 5/2, Y 2(T) = 8T2 + 8T + 9 T 3/2,... (4 + T) 11/2 22

23 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: Mean Integratingg 1 (nt) gives a power series innfor the mean capacity: κ 1 = with P 0 g 1 (n/z) dz = nµ z n µ Correction,1 + 1 n 3µ Correction,2 + µ 0 = 2log µ Correction,1 = 1 12 ( P 2 [ 1+6P+6P 2 µ Correction,2 = ) ( 1 1+4P) 2 4P (4P+1) 3/2 1 ] [ 1+18P+126P P P 4 180P 6 (1+4P) 9/2 ] 1 Note that these closed form expressions are the EXACT corrections! 23

24 MIMO Outage Probability: A (Refined) Asymptotic Analysis Large SNR analysis: Mean For large SNR (i.e. smallt ), we find that Y k (T) = A k T k 1/2 + O ( 1 T k 3/2 ), k = 1,2,... Substituting this form into the recursion equation ofy k, and obtaining the recursion equation fora k : A k = A k 1 ( k 3 2 )( k ) k 2 l=1 + 2 k 1 l=1 A l A k l ( k l 1 2 )( l ( A l A k l 1 l 1 )( l + 1 )( k l 3 )( k l 1 ) ) From this, we get the large n large SNR expansion for the mean: ( ) 2k 1 A k P κ 1 nlogp = nlogp + 1 ( P k 1/2 n 16 n + 3 P 1024 n k=1 ) 3 + Unexpected result: Corrections form a simple power series in the double-scaling variable, P/n. Observation: Correction terms increase with P at a faster rate than the leading term! 24

25 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: Variance Similar to the case ofg 1 (t), we obtain the equation involvingg 2 (t): 2g 1(1+g 1)(g 1 tg 1) 2n 2 (1+2g 1)g 2 +{ g 1 +(2n+t)g 1}{ g 2 +g 1 +(2n+t)g 2} t 2 g 1 g 2 = 0 (E2) Recalling the structureg 2 (nt) = solved in closed-form systematically Z 0 (T) = T 2 4+T T k=0 1 n 2k Z k (T), substituting it into (E2),Z k (T) can be Z 1 (T) = T3/2 3T1/2 (T +4) 9/2 (T +4) 9/2 + T (T +4) T 1/2 (T +4) 9/2 4 T(T +4) 4 4 (T +4) 4. 25

26 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: Variance Integratingg 2 (nt) gives a power series innfor the capacity variance: with κ 2 = P 0 g 2 (n/z) dz = σ z n 2σ Correction,1 + 1 n 4σ Correction,2 + [ ] (4P +1) 1/4 +(4P +1) 1/4 σ 0 = 2log, 2 σ Correction,1 = 1 [ 1 8P3 (1 3P) 12 (4P +1) 3 12P3 +30P 2 ] +10P +1 (4P +1) 5/2. These closed-form expressions give the EXACT corrections. 26

27 MIMO Outage Probability: A (Refined) Asymptotic Analysis Large SNR analysis: Variance For large SNR (i.e. smallt ), we realize that Z k (T) B k T k +O Closed-form recursion obtained forb k s ( 1 T k 1/2 ), k = 0,1,2,... From this, we get the large n large SNR expansion for the variance: ( ) ( ) P 2k B k P κ 2 log 2 2 k n k=1 ( ) ( ) P = log ( P + 9 P 2 32 n 1024 n ) 4 + Again: Corrections form a simple power series in the double-scaling variable, P/n. Again: correction terms increase with P at a faster rate than the leading term! 27

28 MIMO Outage Probability: A (Refined) Asymptotic Analysis General solutions: the third cumulant Following the previous procedure, higher cumulants, which characterize deviations from Gaussian, can be computed: P g 3 (n/z) κ 3 = 3! dz = 1 z n κ 3,Correction1 + 1 n 3κ 3,Correction κ 3,Correction1 = 1 3P 1 + 4P 1 + 4P + P (1 + 4P) 3/ P 1/2 + 3 ( ) P + O P 3/2 κ 3,Correction2 = 96P P P P P P + 1 6(4P + 1) 9/2 72P 5 + [ ] + 1 6(4P + 1) P3/ P P1/2 25 ( ) O P 1/2 Interesting Conclusions: AsP,κ 3,Correction1 1/4. WHY CONSTANT? ANY IDEAS? Also,κ 3,Correction2 O(P 3/2 ). Important Message: As P grows, one should not simply ignore the correction terms!! 28

29 MIMO Outage Probability: A (Refined) Asymptotic Analysis Extracting the leading order terms of ALL cumulants Make the replacementλ nλ in (E0). The leading terms of every cumulant are now taking the order ofo(n 2 ). Taken, givingg n (t) n 2 G(T). G(T) is independent of n and satisfies: 4 ( TG G+1 )( G 2 +λg ) = [ (T +2+λ)G G ] 2 (L0) SubstituteG(T) = λg 1 (T)+λ 2 g 2 (T)+... into (L0) and solve forg i (T),i = 1,2,... Integrate to obtain the leading order terms of theith cumulant,κ i 29

30 MIMO Outage Probability: A (Refined) Asymptotic Analysis The leading order terms of all cumulants Recursion equation forg k : where A k = g k = k 1 k i g i i=1 j=1 A k k 1 i=2 Examples: 3rd and 4th cumulants κ 3 = 1 n [(T 2 + 4T)g i g k+1 i (2T + 4)g i g k+1 i + g i g k+1 i ] 2g 1 (2T + 4)g 1 ( 4Tg j ) k 1 g k+1 i j 4g j g k+1 i j + P 4P +1+ 4P +1 3P 1 (4P +1) 3/2 i=1 [(2T 4)g i g k i 2g i g k i] κ 4 = 1 n 2 12P 2 1+4P +10P 1+4P + 1+4P 8P 3 30P 2 12P 1 (4P +1) 3., k 2 i=1 g i g k i 1 With this result, we can compute the leading order (inn) term for any desired number of cumulants. 30

31 Presentation Outline MIMO Outage Probability and Problem Statement New Tools from RMT and Physics: Hankel Determinants and Orthogonal Polynomials Exact Painlevé Representation for Capacity Generating Function MIMO Outage Probability: A (Refined) Large-System Analysis Numerical Studies and Conclusions 31

32 Numerical Study: Correction Terms Corrections to Mutual Information Mean Variance κ 3 κ 3, Correction A = σ 2 Correction = µ Correction = Number of Antennas, n (a)p = 0 db Corrections to Mutual Information Corrections to Mutual Information Mean Variance κ 3 κ 3, Correction A = σ 2 Correction = µ Correction = Number of Antennas, n (b)p = 5 db Mean Variance κ 3 σ 2 Correction = κ 3, Correction A = µ Correction = Observations: For all three cumulants, first correction term (limiting value) increases with P, as expected Lower order correction terms become much more significant as SNR increases Higher order cumulants (e.g.,κ 3 ) are much more sensitive to SNR than the mean Number of Antennas, n (c)p = 10 db 32

33 Numerical Study: 3 3MIMO Capacity PDF PDF PDF SNR= 5dB Monte Carlo Gaussian Approx. Gram Chalier. (1 Corr) Gram Chalier. (2 Corr) x (nats/channel use/antenna) SNR=5dB PDF PDF SNR=15dB x (nats/channel use/antenna) SNR=30dB Observation: Strong deviation from Gaussian as P grows, even with higher cumulant corrections! Key Message: One must consider the large n large SNR double-scaling to properly characterize the Gaussian deviations when P is sufficiently larger than n This problem is still open! x (nats/channel use/antenna) x (nats/channel use/antenna) Gram-Chalier (Corrected-Gaussian): f(t) = 1 2πσ e z2 2 [ 1 + κ 3 3!σ 3 H 3 (z) + κ ] 4 4!σ 4 H 4 (z) z := t µ σ, H 3(z) = z 3 3z, H 4 (z) = z 4 6z

34 Numerical Study: MIMO Capacity PDF (SNR = 30dB) N=3 Monte Carlo Gaussian Approx. Gram Chalier. (1 Corr) Gram Chalier. (2 Corr) N=5 PDF PDF x (nats/channel use/antenna) x (nats/channel use/antenna) N= PDF x (nats/channel use/antenna) 34

35 Numerical Study: 3 3MIMO Capacity CDF 10 0 SNR= 5dB 10 0 SNR=5dB Observations: Outage Probability Outage Probability Monte Carlo Gaussian Approx. Gram Chalier. (1 Corr) Gram Chalier. (2 Corr) Large Dev R (nats/channel use) SNR=15dB R (nats/channel use) Outage Probability Outage Probability R (nats/channel use) SNR=30dB R (nats/channel use) As SNR increases, Gaussian is loosening around bulk, but getting more accurate around (low) tails. Gaussian corrections significant increase accuracy at low SNR Large deviations results are also quite good, particularly in tails Query: Why are the tails more accurate for all curves? Outage Probability SNR=45dB Require systematic analysis of double-scaling, with SNR sufficiently large compared withn R (nats/channel use) 35

36 Main Results: Conclusions New exact representation for the MGF of MIMO capacity in terms of a Painlevé V differential equation Systematic procedure for computing the asymptotic cumulants (not just mean and variance), and the correction terms for finite n Key Insights: For large number of antennas, the outage probability is approximately Gaussian, provided the SNR is not too large There is a double-scaling behavior for large n large SNR. We have drawn key insight into this. Final Word: Underlying theory (orthogonal polynomial approach) is rigorous and systematic, and can be applied in many other contexts in communications and signal processing. E.g., Multi-user MIMO MIMO relaying Spectrum sensing...?? Ref: Y. Chen and M. R. McKay, Coulomb Fluid,, submitted to IEEE Trans. IT (currently under revision) 36