National University of Singapore. Ralf R. Müller

Transcription

1 Ê ÔÐ Å Ø Ó Ò Ï Ö Ð ÓÑÑÙÒ Ø ÓÒ Ì National University of Singapore March 006 Ralf R. Müller Professor for Wireless Networks Norwegian University of Science & Technology

2 Ê ÔÐ Å Ø Ó Ò ÅÙÐØ Ø Ø Ø ÓÒ Ì Ï Ö Ð ÓÑÑÙÒ Ø ÓÒ Ò National University of Singapore March 006 Ralf R. Müller Professor for Wireless Networks Norwegian University of Science & Technology

3 The Replica Method 1 Part I: The Replica Method

4 The Replica Method Thermodynamics vs. Wireless Communications Since Boltzmann, physicists have studied the behavior of systems with interactions of many particles. Particular correspondences can be drawn between spin glasses (amorph magnetic materials, e.g. the magnetic surface of a hard disk drive) and communication systems. The binary nature of the bits corresponds to the quantum-mechanical constraints of electron spins to ± 1. Spin glass theory is both very rich and very complicated. Various methods have been proposed by physicists to analyze them: The replica method The cavity method... The TAP approach Gauge Theory This course will be restricted in scope to the replica method.

5 The Replica Method 3 Spin Glasses

6 The Replica Method 3 Spin Glasses

7 The Replica Method 3 Spin Glasses

8 The Replica Method 3 Spin Glasses Energy function (Hamiltonian): i r ij x i x j i j<i h i x i

9 The Replica Method 3 Spin Glasses h external magnetic field Energy function (Hamiltonian): i r ij x i x j i j<i h i x i

10 The Replica Method 4 Optimal Detection of Vector Channel y = Sx + n

11 The Replica Method 4 Optimal Detection of Vector Channel Best estimate for transmitted data: ˆx = argmin x {±1} K y Sx y = Sx + n argmin x {±1} K 1 x Rx h x + y y with R = S S h = S y

12 The Replica Method 4 Optimal Detection of Vector Channel Best estimate for transmitted data: ˆx = argmin x {±1} K y Sx y = Sx + n = argmin x {±1} K 1 x Rx h x + y y with R = S S h = S y

13 The Replica Method 4 Optimal Detection of Vector Channel Best estimate for transmitted data: y = Sx + n ˆx = argmin x {±1} K y Sx = argmin 1 x {±1} K x Rx h x + y y = argmin r ij x i x j x {±1} K i j<i i with h i x i 1 R = S S h = S y r ii x i i

14 The Replica Method 4 Optimal Detection of Vector Channel Best estimate for transmitted data: y = Sx + n ˆx = argmin x {±1} K y Sx = argmin 1 x {±1} K x Rx h x + y y = argmin r ij x i x j x {±1} K i j<i i with h i x i 1 R = S S h = S y r ii x i i Minimization of the energy function of a spin glass!

15 The Replica Method 5 A Phase Transition in Random CDMA optimal detection K = N 1 bit error probability without interference signal to noise ratio [db]

16 The Replica Method 6 Large Systems C K K O ( K ) interactions C C K O(K) microscopic objects O(1) macroscopic variables Macroscopic variables are self-averaging.

17 The Replica Method 7 Boltzmann Distribution The Thermodynamic Equilibrium maximizes the entropy for given constant energy H(X) = i E(X) = i Pr(x i ) log Pr(x i ) x i Pr(x i ) yielding the Boltzmann distribution Pr(x i ) = e 1 T x i e. T 1 x i i

18 The Replica Method 8 Free Energy Since the energy is constant, we can minimize the free energy F(X) = E(X) TH(X) instead of maximizing entropy. This is often less complicated. With the Boltzmann distribution, the free energy is given by [ ] F(X) = T log. It depends only on the partition function. i e 1 T x i The free energy is self-averaging.

19 The Replica Method 9 Energy vs. Entropy The following two tasks are dual: Minimize the energy for fixed entropy Maximize the entropy for fixed energy Consider free energy F(X) = E(X) TH(X) and read the temperature (or its inverse) as Lagrange multiplier. For the dual problem have 1 T F(X) = H(X) 1 T E(X)

20 The Replica Method 10 The Meaning of the Energy Function In physics, the energy function varies with the force causing the potential. Theoretically speaking, the choice of the energy function is arbitrary as long as it is uniformly bounded from below. Nature maximizes entropy for a given energy. In communications engineering, the energy function is the metric used by the decoder. The decoder does the dual job of nature, to minimize the metric for a given output entropy. Since the decoder dictates the thermodynamics of our toy universe, the same holds true if the decoder uses a suboptimal (wrong) or insufficient metric, perhaps due to lack of knowledge about the channel state. The free choice of the energy function allows to analyze mismatched receivers.

21 The Replica Method 11 LMMSE Detector with Mismatched Powers Theorem: Let (U 1,...,U K ) be an arbitrary sequence of non-negative numbers such that, as K, the empirical joint cdf of the pairs {(U k, P k ) : k = 1,...,K} converges weakly to a given non-random cdf F(u, p). Moreover, let the P k s are uniformly bounded above and the U k s are uniformly bounded below by a positive number for all K. Then, the output SINR of the mismatched LMMSE detector assuming powers {U k } instead of the true powers {P k } in the standard random spreading model converges as K = βn almost surely to ηp k 1 + β 1 + β u df(u, p) (1 + uη) where η = p (1 + uη) df(u, p) ( 1 + β u 1 + uη ) 1 df(u, p) is the multiuser efficiency of an LMMSE detector of a virtual channel having powers given by {U k } instead of {P k }.

22 The Replica Method 1 Average Free Energy When analyzing a random system, we evaluate the average free energy as Shannon analyzed the average performance of all codes.

23 The Replica Method 1 Average Free Energy When analyzing a random system, we evaluate the average free energy as Shannon analyzed the average performance of all codes.

24 The Replica Method 13 Free Energy for a Random Parameter Consider a self-averaging random parameter, e.g. a spreading matrix. F(X y j ) = E F(X Y ) Y [ = T E log Y i e 1 T x i ] The energy function depends on the random parameter y j. The expectation of a logarithm is a hard problem.

25 The Replica Method 13 Free Energy for a Random Parameter Consider a self-averaging random parameter, e.g. a spreading matrix. F(X y j ) = E F(X Y ) Y [ = T E log Y i e 1 T x i ] The energy function depends on the random parameter y j. The expectation of a logarithm is a hard problem.

26 The Replica Method 14 Replica Continuity E log(y ) = lim Y n 0 n E Y Y n Evalute n th moments for integer n and assume analytic continuity for the limit. More general, we have E log Y f(x, Y )dx = lim n 0 n E Y f(x, Y )dx n Ê Ê

27 The Replica Method 14 Replica Continuity E log(y ) = lim Y n 0 n E Y Y n Evalute n th moments for integer n and assume analytic continuity for the limit. More general, we have E log Y f(x, Y )dx = lim n 0 n E Y f(x, Y )dx n Ê Ê

28 The Replica Method 14 Replica Continuity E log Y f(x, Y )dx = lim n 0 n E Y f(x, Y )dx n Ê Ê With we finally get E log Y Ê g(x)dx n = n a=1 f(x, Y )dx = lim n 0 n E Y Ê g(x a )dx a n a=1 f(x a, Y )dx a Ê Ê

29 The Replica Method 14 Replica Continuity E log Y f(x, Y )dx = lim n 0 n E Y f(x, Y )dx n Ê Ê With we finally get E log Y Ê g(x)dx n = n a=1 f(x, Y )dx = lim n 0 n E Y Ê g(x a )dx a n a=1 f(x a, Y )dx a Ê Ê

30 The Replica Method 14 Replica Continuity E log Y f(x, Y )dx = lim n 0 n E Y f(x, Y )dx n Ê Ê With we finally get E log Y Ê g(x)dx n = n a=1 f(x, Y )dx = lim n 0 n E Y Ê g(x a )dx a n a=1 f(x a, Y )dx a Ê Ê

31 The Replica Method 15 Replica Symmetry Throughout the calaculations, we solve integrals of the form I = 1 K log Kf(x 1,x ) dx 1 dx maxf(x 1,x ) x 1,x for K by saddle point integration. Ê e If the maximization is too tedious, we assume replica symmetry: maxf(x 1,x ) = maxf(x, x) x 1,x x Replica symmetry is a strong assumption and not always valid.

32 The Replica Method 15 Replica Symmetry Throughout the calaculations, we solve integrals of the form I = 1 K log Kf(x 1,x ) dx 1 dx maxf(x 1,x ) x 1,x for K by saddle point integration. Ê e If the maximization is too tedious, we assume replica symmetry: maxf(x 1,x ) = maxf(x, x) x 1,x x Replica symmetry is a strong assumption and not always valid.

33 The Replica Method 15 Replica Symmetry Throughout the calculations, we solve integrals of the form I = 1 K log Kf(x 1,x ) dx 1 dx maxf(x 1,x ) x 1,x for K by saddle point integration. Ê e If the maximization is too tedious, we assume replica symmetry: maxf(x 1,x ) = maxf(x, x) x 1,x x Replica symmetry is a strong assumption and not always valid.

34 The Replica Method 16 A Counterexample to Replica Symmetry f(x 1,x ) = sin(x 1 x )e x 1 x 0. f(x 1,x ) x 0 0 x However, f(x 1,x ) is replica symmetric.

35 The Replica Method 17 Phase Transitions If the final equations allow for multiple solutions, the correct solution is identified by minimizing the free energy.

36 The Replica Method 18 Phase Transitions and Neural Networks 10 0 bit error probability a) b) load β

37 The Replica Method 19 Individually Optimum Maximum Likelihood Detector Let A = {+1; 1}, the chips of any user be i.i.d. random variables with finite variance and vanishing odd moments, the powers of all users identical, and N,K, but β = K/N fixed. Then, the multiuser efficiency is a solution to the fixed point equation 1 η IO = 1 + β σ n 1 ηio πσ n Ê tanh ( ηio σ n ) x exp ( η IO(x 1) ) dx. In case the fixed point equation has multiple solutions, the correct one is that solution for which the term η IO + η [ ( )] IO log η IO ηio ηio log cosh x exp ( η IO(x 1) ) dx σn β πσn σn σn σ n is smallest. Ê

38 The Replica Method 0 Example for Replica Calculations Consider the analysis of an asymptotically large CDMA systems with arbitrary joint distribution of the variances of the random chips. It includes the practically imporant case of multi-carrier CDMA transmission with users of arbitrary powers in frequencyselective fading as a special case. The vector-valued, real additive white Gaussian noise channel is characterized by the conditional pdf p y x,h (y, x, H) = e 1 σ 0 (y Hx) T (y Hx). (1) (πσ0 )N Moreover, let the detector be characterized by the assumed conditional probability distribution p y x,h (y,x,h) = e 1 σ (y Hx)T (y Hx) and the assumed prior distribution p x (x). (πσ ) N

39 The Replica Method 1 Let the entries of H be independent zero-mean with vanishing odd order moments and variances wck /N for row c and column k. Applying Bayes law, we find 1 p x y,h (x, y,h) = e σ (y Hx)T (y Hx)+log p x (x). e 1 σ (y Hx)T (y Hx) d P x (x) Since the Boltzmann distribution holds for any temperature T, we set w.l.o.g. T = 1 and find the appropriate energy function to be x = 1 σ (y Hx)T (y Hx) log p x (x). () This choice of the energy function ensures that the thermodynamic equilibrium models the detector defined by the assumed conditional and prior distributions.

40 The Replica Method With (1) and (), the definition of free energy, and replica continuity, we find for the free energy per user F(x) K = 1 ( K E log H N Ê = 1 K lim n 0 = 1 K lim n 0 n log E H N Ê e 1 σ 0 (y Hx) T (y Hx) (πσ0 )N n log ) 1 e 1 σ (y Hx)T (y Hx) e σ (y Hx) T (y Hx) 0 d P x (x) (πσ0 )N ( E H N a=0 Ê ) n e 1 σ (y Hx)T (y Hx) d P x (x) dydp x (x) n e 1 σ a(y Hx a ) T (y Hx a ) dy n dp a (x a ) } (πσ0 )N {{ a=0 } =Ξ n with σ a = σ, a 1, P 0 (x) = P x (x), and P a (x) = P x (x), a 1. dydp x (x) (3)

41 The Replica Method 3 The integral in (3) is given by n E exp 1 N H a=0 Ê Ξ n = c=1 σ a ( y c πσ0 K k=1 ) h ck x ak dy c n dp a (x a ), (4) with y c, x ak, and h ck denoting the c th component of y, the k th component of x a, and the (c,k) th entry of H, respectively. The integrand depends on x a only through v ac = 1 β K k=1 h ck x ak, a = 0,...,n. These quantities v ac can be regarded, in the limit K as jointly Gaussian random variables with zero mean and covariances a=0 Q ab [c] = E v ac v bc = 1 H K x (c) a x b (5) where the parametric inner products are defined by x a (c) x b = K k=1 x ak x bk w ck.

42 The Replica Method 4 In order to perform the integration in (4), the K(n + 1)-dimensional space spanned by the replicas and the vector x 0 is split into subshells { } S{Q[ ]} = (c) x 0,...,x n x a x b = KQ ab [c] where the inner product of two different vectors x a and x b is constant. 1 The splitting of the K(n+1)-dimensional space is depending on the chip time c. With this splitting of the space, we find N KI{Q[ ]} Ξ n = e G{Q[c]} dq ab [c], (6) a b with appropriate choices of the function I{Q[ ]} and G {Q[c]}. Ê N(n+1)(n+)/ e c=1 1 The notation f{q[ ]} expresses the dependency of the function f( ) on all Q ab [c], 0 a b n, 1 c N. The notation a b is used as shortcut for n a=0 n b=a.

43 The Replica Method 5 In (6), e KI{Q[ ]} = a=0 a b N δ x a N c=1 (c) x b βq ab[c] denotes the probability weight of the subshell and [ e G{Q[c]} 1 n = E exp β ( ) ] yc v πσ0 H σa ac {Q[c]} β n dp a (x a ) a=0 dy c + O(K 1 ). Ê This procedure is a change of integration variables in multiple dimensions where the integration of an exponential function over the replicas has been replaced by integration over the variables Q ab [ ]. In the following the blue and green terms in (6) are evaluated separately.

44 The Replica Method 6 First, we calculate the measure e KI{Q[ ]} Ê. As for some t, we have the Fourier expansion of the Dirac measure δ x (c) a x b N βq ab[c] = 1 exp Qab [c] x (c) a x b πj N βq ab[c] d Q ab [c] J with J = (t j ; t + j ), the measure e KI{Q[ ]} can be expressed as N Q ab [c] x a (c) x b e KI{Q[ ]} N = βq ab[c] d Q ab [c] n e πj dp a (x a ) with = c=1 a b J β N e J N(n+)(n+1)/ M k { Q[ ] } = c=1 a b exp 1 N a=0 ( Q ab [c]q ab [c] K { } ) N M k Q[ ] a b N c=1 k=1 Q ab [c]x ak x bk wck c=1 a b n dp a (x ak ). a=0 d Q ab [c] πj (7)

45 The Replica Method 7 In the limit of K one of the exponential terms in (6) will dominate over all others. Only the maximum value of the correlation Q ab [c] is relevant for calculation of the integral. We assume that the replicas within the dominant subshell are symmetric (replica symmetry). Thus, the maximum values of the correlations Q ab [c] are identical for all positive a b. The same applies to the the correlations Q a0 [c]. Hereby, we reduce the number of different correlation variables from (n + 1)(n + )/ to four per chip time and set Q 00 [c] = p 0c, Q 0a [c] = m c, a 0, Q aa [c] = p c, a 0, Q ab [c] = q c, 0 a b 0. We apply the same idea to the correlation variables in the Fourier domain and set Q 00 [c] = G 0c /, Qaa [c] = G c /, a 0, Q0a [c] = E c, a 0, and Q ab [c] = F c, 0 a b 0. At this point the crucial benefit of the replica method becomes obvious. Assuming replica continuity, we have managed to reduce the evaluation of a continuous function to sampling it at integer points. Assuming replica symmetry we have reduced the task of evaluating infinitely many integer points to calculating 8 different correlations (4 of them in the original and 4 of them in the Fourier domain).

46 The Replica Method 8 The assumption of replica symmetry leads to n(n 1) Q ab [c]q ab [c] = ne c m c + and a b M k {E, F,G, G 0 } = where = N 1 N wck c=1 e n+1 Ê Ê n+1 e Ẽ k = 1 N G k = 1 N F c q c + G 0cp 0c ( ) G 0c x 0k + n E c x 0k x ak + G c x ak + n F c x ak x bk a=1 b=a+1 G 0k x 0k + n Ẽ k x 0k x ak + G k x ak + n a=1 b=a+1 N c=1 N c=1 E c w ck, Fk = 1 N G c w ck, G0k = 1 N F k x ak x bk + n G cp c (8) n dp a (x ak ) a=0 n dp a (x ak ) (9) a=0 N F c wck (10) c=1 N G 0c wck. (11) (9) cannot be simplified further for a general prior distribution dp a (x ak ). c=1

47 The Replica Method 9 Second, we evaluate e G{Q[c]} in (6). We use the replica symmetry to construct the correlated Gaussian random variables v ac out of independent zero-mean, unit-variance Gaussian random variables u c,t c,z ac by v 0c = u c p 0c m c m c t c q c qc With that substitution, we get e G(m c,q c,p c,p 0c ) = = v ac = z ac pc q c t c qc, a > 0. 1 πσ0 Ê Ê Ê exp exp [ β σ0 β σ u c p 0c m c t cm c y c Du c q c qc β ( z c pc q c t c qc y c (1 + β σ (p c q c )) 1 n ) ] Dz c β n Dt c dy c 1 + β σ (p c q c ) + n β σ ( σ 0 β + p 0c m c + q c ) (1) with the Gaussian measure Dz = exp( z /) dz/ π.

48 The Replica Method 30 Since the integral in (6) is dominated by the maximum argument of the exponential function, the derivatives of 1 N G{Q[c]} β Q ab [c]q ab [c] (13) N c=1 a b with respect to m c, q c, p c and p 0c must vanish as N. Taking derivatives after plugging (8) and (1) into (13), solving for E c, F c,g c, and G 0c and letting n 0 yields for all c E c = 1 σ + β(p c q c ) (14) F c = σ 0 + β (p 0c m c + q c ) [σ + β(p c q c )] G c = F c E c (15) G 0c = 0. (16) In order to proceed with the calculations, a prior distribution needs to be specified.

49 The Replica Method 31 Gaussian Prior Distribution Assume the Gaussian prior p a (x ak ) = 1 π e x ak / a. The integration in (9) can be performed explicitly and we find ( M k (Ẽk, F k, G k, G ) 0k = 1 + Fk G ) 1 n k ( )( ) (17) 1 G 0k 1 + F k G k n F k nẽ k. In the large system limit, the integral in (7) is dominated by that value of the integration variable which maximizes the argument of the exponential function. Thus, partial derivations of log K ) N M k (Ẽk, F k, G k, G 0k β k=1 c=1 n(n 1) ne c m c + F c q c + G 0cp 0c with respect to E c,f c,g c,g 0c must vanish for all c as N. + n G cp c (18)

50 The Replica Method 3 An explicit calculation of these derivatives yields m c = 1 K w Ẽ k ck K 1 + Ẽ k=1 k (19) q c = 1 K w Ẽk + F k ck ( ) K 1 + Ẽk (0) p c = 1 K p 0c = 1 K k=1 K k=1 w ck Ẽ k + Ẽ k + F k + 1 ( 1 + Ẽk ) (1) K wck () k=1 in the limit n 0 with (15) and (16). Surprisingly, if we let the true prior to be binary and only the replicas to be Gaussian we also find (19) to (). Note from Chapter 1 that this setting corresponds to linear MMSE detection.

51 The Replica Method 33 Collecting our previous results to evaluate the free energy, we find 1 K n log Ξ n= 1 N [ G(m c, q c, p c, p 0c ) + βne c m c + K n c=1 K log M k (Ẽk, F k, G k, 0 ) = 1 K n 0 1 K k=1 [ N log c=1 βn(n 1) F c q c + βn ] G cp c ( 1 + β ) σ (p c q c ) + βe c m c + β(n 1)F c q c + βg c p c ] σ0 + + β(p 0c m c + q c ) σ + β(p c q c ) + nσ0 + nβ(p 0c m c + q c ) + 1 K log ( ) Ẽ 1 + K Ẽk k + F k 1 + k=1 Ẽk nẽ k n F k [ N ( log 1 + β ) σ (p c q c ) + 1 K = F(x) K. c=1 K k=1 log ( ) Ẽ 1 + Ẽk k + F k 1 + Ẽk + E c F c + βe c m c βf c q c + βg c p c ]

52 The Replica Method 34 This is the final result for the mismatched detector. The six macroscopic parameters E c, F c,g c,m c, q c, p c are implicitly given by the simultaneous solution of the system of equations (14) to (15) and (19) to (1) with the definitions (10) to (11) for all chip times c. This system of equations can only be solved numerically. Specializing our result to the matched detector by letting σ σ 0, we have F c E c, G c G 0c, q c m c, p c p 0c. This makes the free energy simplify to with F(x) K = 1 K = 1 K [ N log c=1 [ N c=1 ( 1 + β σ 0(p 0c m c ) σ 0E c log ( σ 0E c ) ] + 1 K E c = ) βe c m c ] K k=1 σ 0 + β K + 1 K log ( ) 1 + Ẽk 1 K k=1 w ck 1+Ẽk K log ( 1 + Ẽk) Ẽ k k=1. (3) This result is more compact and it requires only to solve (3) numerically which is conveniently done by fixed-point iteration.

53 The Replica Method 35 Girko s Law Let the N K random matrix H be composed of independent entries (H) ij with zero-mean and variances w ij /N such that all w ij are uniformly bounded from above. Assume that the empirical joint distribution of variances w : [0, 1] [0, β] Ê defined by w(x,y) = w ij for i,j satisfying i N x i + 1 j and N N y j + 1 N converges to a bounded joint limit distribution w(x,y) as K = βn. Then, for each a,b [0, 1],a < b, and I(s) > 0 1 N bn i= an ( HH H si ) 1 ii b a u(x, s)dx

54 The Replica Method 36 where convergence is in probability and u(x, s) satisfies the fixed point equation u(x,s) = s + β w(x,y) dy u(x, s)w(x, y) dx for every x [0, 1]. The solution always exists and is unique in the class of functions u(x, s) 0, analytic for I(s) > 0 and continuous on x [0, 1]. Moreover, almost surely, the empirical eigenvalue distribution of HH H converges weakly to a limiting distribution whose Stieltjes transform is given by G HH H(s) = 1 0 u(x,s) dx. 1

55 The Replica Method 37 Comparing (3) to Girko s law, Ẽ k is recognized as the signal-to-interference and noise ratio of user k. Using the similarity of free energy and the entropy of the channel output allows for the simple relationship I(x, y) K = F(x) K 1 (4) β between the (normalized) free energy and the (normalized) mutual information between channel input signal x and channel output signal y given the channel matrix H. Assuming that the channel is perfectly known to the receiver, but totally unknown to the transmitter, (4) gives the channel capacity per user.

56 The Replica Method 38 Consider a non-uniform binary prior Binary Prior Distribution p a (x ak ) = 1 + t k δ(x ak 1) + 1 t k δ(x ak + 1). (5) Plugging the prior distribution into (9), we find M k (Ẽk, F k, G k, G 0k ) = G 0k +n G k + n Ẽ k x 0k x ak + n F k x ak x bk e a=1 b=a+1 n+1 Ê = e 1 ( G 0k +n G k ) {x ak,a=1,...,n} { 1+t k exp n dp a (x ak ) a=0 [ n Ẽ k x ak + a=1 n b=a+1 F k x ak x bk ] [ n ]} + 1 t k exp Ẽkx ak + n n F k x ak x bk Pr(x ak ) a=1 b=a+1 a=1 { [ ( = e( 1 G 0k +n G k n F k ) n ) ] 1+t k F exp k n x ak + Ẽk x ak {x ak,a=1,...,n} a=1 a=1 [ ( n ) ]} + 1 t k F exp k n n x ak Ẽk x ak Pr(x ak ). (6) a=1 a=1 a=1

57 The Replica Method 39 We can use the following property of the Gaussian measure ( S exp F ) ( ) k = exp ± F k zs Dz to linearize the exponents M k (Ẽk, F k, G k, G 0k ) = e 1 ( G 0k +n G k n F k ) + 1 t k exp {x ak,a=1,...,n} [ ( z Fk + Ẽk S Ê ( 1+tk exp[ z ) n ] Fk + Ẽk x ak a=1 ) n ] x ak Dz n Pr(x ak ). a=1 a=1 Since f n = = x kn {x ka,a=1,...,n} = f n 1 cosh ( ) n exp[ z F k + Ẽk Pr(x kn )f n 1 exp a=1 [( ) ] z F k + Ẽk x kn [ λ k / + z ] Fk + Ẽk cosh (λ k /) = x ka ] n a=1 Pr(x ka ) cosh n [ λ k / + z Fk + Ẽk cosh n (λ k /) ]

58 The Replica Method 40 with t k = tanh(λk /), we find ( 1+tk M k (Ẽk, F k, G k, G ) cosh n z ) Fk + Ẽk + λ k 0k = cosh n ( λ k ) exp + 1 t k ( cosh n z ) Fk + Ẽk λ k Dz ). ( n Fk G 0k n G k In the large system limit, the integral in (7) is dominated by that value of the integration variable which maximizes the argument of the exponential function. Thus, partial derivations of log K M k (Ẽk, F k, G k, G ) N 0k β ne c m c + k=1 c=1 n(n 1) with respect to E c,f c,g c,g 0c must vanish for all c as N. F c q c + G 0cp 0c + n G cp c

59 The Replica Method 41 An explicit calculation of these derivatives gives m c = 1 K q c = 1 K K k=1 K k=1 p c = p 0c = 1 K w ck w ck K k=1 in the limit n 0. ( 1+t k tanh z ) ( Fk + Ẽk + λ k + 1 t k tanh z ) Fk + Ẽk λ k Dz (7) ( 1+t k tanh z ) ( Fk + Ẽk + λ k + 1 t k tanh z ) Fk + Ẽk λ k Dz (8) w ck In order to obtain (8), note from (6) that the first order derivative of M k exp(n F k /) with respect to F k is identical to half of the second order derivative of M k exp(n F k /) with respect to Ẽk. (9)

60 The Replica Method 4 Collecting our previous results to evaluate the free energy, we find 1 K n log Ξ n= 1 N [ G(m c, q c, p c, p 0c ) + βne c m c + K n c=1 K log M k (Ẽk, F k, G k, 0 ) n 0 1 K k=1 + 1 t k βn(n 1) F c q c + βn ] G cp c N [ ( log 1 + β ) σ (p c q c ) + E ] c + βe c m c βf c q c + βg c p c F c=1 c 1 K ( 1 + tk log cosh z F k + K Ẽk + λ ) k k=1 ( log cosh z F k + Ẽk λ ) k Dz + 1 log ( ) 1 t Fk + G k k = F(x) K. This is the final result for the free energy of the mismatched detector. The six macroscopic parameters E c,f c, G c, m c,q c, p c are implicitly given by the simultaneous solution of the system of equations (14) to (15) and (7) to (9) with the definitions (10) to (11) for all chip times c. This system of equations can only be solved numerically. In case of multiple solutions, the correct solution is that one which minimizes the free energy.

61 The Replica Method 43 Specializing our result to the matched detector by letting σ σ 0, we have F c E c, G c G 0c, q c m c. This makes the free energy simplify to F(x) K = 1 K + = 1 K + N c=1 1+t k [ log ) ] βe c m c 1 K log 1 t k K Ẽk k=1 ( z ) ( Ẽ k + Ẽk + λ k + 1 t k log cosh z ) Ẽ k + Ẽk λ k Dz ( 1 + β σ 0(p 0c m c ) log cosh N [ σ 0 E c log ( σ0 E 1 c)] K c=1 1+t k log cosh K log k=1 1 t k Ẽk ( z ) Ẽ k + Ẽk + λ k + 1 t k log cosh where the macroscopic parameters E c are given by 1 E c = σ 0 + β K + 1 t k = σ 0 + β K K k=1 tanh K k=1 w ck [ tk ( z Ẽ k + Ẽk λ k w ck ( 1 t k ) 1 tanh ( z Ẽ k + Ẽk λ k ) Dz ( tanh z Ẽ k + Ẽk + λ ) k ) ] Dz ( z Ẽ k + Ẽk ) 1 t k tanh ( z Ẽ k + Ẽk ) Dz.

62 The Replica Method 44 Similar to the case of Gaussian priors, Ẽ k can be shown to be a kind of signal-tointerference and noise ratio, in the sense that the bit error probability of user k is given by Pr(ˆx k x k ) = Ẽk Dz. An equivalent additive white Gaussian noise channel can be defined to model the multiuser interference for any prior. For any input alphabet to the channel mutual information is given by (4) with the free energy corresponding to that input alphabet.

63 The Replica Method 45 MC-CDMA in Multipath Fading Equivalent baseband vector channel in frequency domain: y = ( W S ) x + n N 1 N K N K K 1 N 1 frequency channel Hadamard spreading users noise chips matrix product matrix data vector The noise n has i.i.d. Gaussian entries with zero-mean and unit variance. The columns of S are the random spreading sequences of the users. The columns of W are the random frequency responses of the users.

64 The Replica Method 46 Minimum Probability of Error for MAP Detector Maximum a-posteriori detector: ˆx k = arg max x k Pr(x k y, W ) In the large system limit, there is an equivalent AWGN channel with SINR Ẽ k such that ) Pr(ˆx k x k W ) = Dz = Q ( Ẽ k Ẽk and ) Pr(ˆx k x k ) = E Pr(ˆx k x k W ) = E Q ( Ẽ k W W

65 The Replica Method 47 SINR of Equivalent AWGN Channel For N,K large, solve the fixed-point system of equations N Ẽ k = 1 N E c = c=1 σ n + β K E c w ck K (1 t k ) wck k=1 1 1 tanh (z ) Ẽ k + Ẽ k ( 1 t k tanh z ) Dz Ẽ k + Ẽ k In practice, the fading statistics obey some structure: Asymptotic frequency-invariance on the uplink (reverse link) Rank-1 statistics on the downlink (forward link)

66 The Replica Method 48 Asymptotic Frequency Invariance (Uplink) E c = E c The fading is ergodic across the user population for each frequency c. Ẽ k = σ n + β K P k K 1 tanh (z ) Ẽ n + Ẽ n (1 t n ) P ( n 1 t n tanh z ) Dz Ẽ n + Ẽ n n=1 with N P k = 1 N w ck c=1 The spectrum of the received signal is white (frequency-invariant). Full diversity is achieved.

67 The Replica Method 49 Rank 1 Statistics (Downlink) W = fu T w ck = f c u k All users experience the same fading channel except for a scalar factor u k. Ẽ k = u k N N c=1 σ n f c + β K K (1 t n ) u n n=1 1 ( 1 tanh z ) Ẽ n ( + Ẽn 1 t n tanh z ) Dz Ẽ n + Ẽ n Full diversity is achieved. The spectrum of the received signal is colored = degradation.

68 The Replica Method 50 Uplink vs. Downlink 10 1 L= L=4 Uniform priors Bit error probability L=6 L equal power paths K N = SNR [db] «Ö Ò

69 The Replica Method 50 Uplink vs. Downlink 10 1 L= L=4 Uniform priors Bit error probability L=6 L equal power paths K N = SNR [db] «Ö Ò

70 Multistage Detection 51 Part II: Multistage Detection

71 Multistage Detection 5 Matrix Inversion Let R be non singular. Let λ i denote the eigenvalues of R. Then, K ( ) R λ k I = 0 = I + K α k R k = 0 k=1 k=1 Cayley Hamilton Theorem with appropriate α k s. Solution to matrix inversion problem given the eigenvalues: R 1 = K α k R k 1 k=1

72 Multistage Detection 53 Linear Multistage Detection Linear MMSE filter: L MMSE = ( ) 1 R + σni Approximation by power series: Cayley Hamilton theorem yields: ( R + σ n I ) K 1 1 = i=0 D 1 w i R i w i R i for D < K. i=0 For random spreading the optimum weights converge almost surely, as K, N with β = K N, and can be given in closed form.

73 Multistage Detection 54 Semi-Universal Weights Filter shall be independent from the realization of the random matrix S, but may use its statistics. For most large random matrices, as K = βn, many finite dimensional functions of the eigenvalues, e.g. the filter coefficients, freeze. The asymptotic limits depend only on parts of the statistics of the random matrix. The weights can be calculated off-line with the help of random matrix and free probability theory.

74 Multistage Detection 55 Weight Design is given by Yule Walker equations: m 1 m. = m + σ m 1 m 3 + σ m... m D+ + σ m D+1 m 3 + σ m m 4 + σ m 3... m D+3 + σ m D w 0 w 1. m D+1 m D+ + σ m D+1 m D+3 + σ m D+... m D+ + σ m D+1 w D with the total moments m n = E {λ n } = Tr ( S H S ) n

75 Multistage Detection 56 Example for Asymptotic Weight Design Random matrix with i.i.d. entries: D = w 0 = σ w β w 1 = 1 w 0 = σ w β + 3β D = 3 w 1 = σ w 3 3β w = 1 m n = 1 n ( )( n n n i=1 i i 1 ) β i. D = 4 w 0 = σ w β + 6β + 4β 3 w 1 = σ w 6 9β 6β w = σ w β w 3 = 1 w 0 = σ w β + 9β + 8β 3 + 5β 4 w 1 = σ w 10 18β 18β 10β 3 D = 5 w = σ w β + 10β w 3 = σ w 4 5 5β w 4 = 1

76 Multistage Detection 57 Rate of Convergence Theorem 1 Let A = I and the chips of any user be i.i.d. zero mean random variables with finite fourth moment and the sequences of all users jointly independent. Then, the multi user efficiencies of all users converge almost surely, as N,K but β = K N fixed, to 1 η WLPIC,D+1 = β 1 + σn + η WLPIC,D with η 0 = 0 for optimally chosen weights. The approximation converges to the exact MMSE performance as a continued fraction. For optimal coefficients w i, the approximation error ǫ decreases exponentially with the number of stages D: There are even tighter bounds. ǫ < const. (1 + SNR) D

77 Multistage Detection 58 Individual Weight Design Allow for different weights for different users ( R + σ n I ) K 1 1 = i=0 D 1 w i R i W i R i for D < K and allw i diagonal. i=0 Weight design by the same Yule-Walker equations, but with the k-partial moments [ (S m (k) n = H S ) ] n. kk For users with different powers, individual weight design is better. Do the k-partial moments convergence asymptotically?

78 Multistage Detection 59 Convergence of Partial Moments Let the random matrix H fufill the same conditions as needed for the deformed quarter circle law. Let A be an K K diagonal matrix such that its singular value distribution converges almost surely, as K to a non-random limit distribution. Let R = A H H H HA. Then, ( R l) kk, the kth diagonal element of R l converges, conditioned on a kk, the k th diagonal element of A, almost surely, as K = βn to l R (l) kk = a kk β R (q 1) kk m (R) l q, l > 1 with m (R) q = Tr(R q ). The total moments of R are conveniently given by the recursion m (R) l = β l q=1 q=1 m (R) l q lim K 1 K K k=1 a kk R (q 1) kk.

79 Multistage Detection 60 Let Structure of Proof R = A H H H HA, T = HAA H H H Bound the difference between the quadratic form and the trace as Silverstein and Bai did. Use Lyapunov inequality to bound the forth moment of the entries of the random matrix H. Recursively proof convergence of the diagonal elements of (R) k and the traces of (T ) k flipping between the two.

80 Multistage Detection 61 Multipath Fading Channels Let the path differences be only a few chip intervals. Approximate the linear time shift by a cyclic shift modulo N. For large N this becomes more and more accurate. paths: All odd column of the N K matrix H are i.i.d. Each even column of H is a cyclically shifted version of the adjacent column to the left. E { [ ] bb H} 1 1 = I 1 1 and (A) kk are independent zero-mean and complex Gaussian. This setting is equivalent to the full i.i.d. setting in all asymptotic aspects if the users powers follow the same distribution. Equivalence holds for an arbitrary number of paths.

81 Multistage Detection 6 Asynchronous Users S n y n y n+1 y n+ y n+3 S n+1 S n+ A n A n+1 A n+ b n b n+ Convergence of k-partial moments proven. Recursive expressions to construct them known. Proof follows the same lines. Y = SA }{{} B + N H

82 Multistage Detection 63 Detector Structure for Asynchronous Users y(n+1) D D Matched Re- Matched Re- Matched Filtering Spreading Filtering Spreading Filtering H H (n) H(n) D H(n 1) H H(n M+1) D H(n M) H D M-1 D D M- D D W 0 W 1 W M 1st Stage M-th Stage ˆb n M H H Y H H HH H Y H H (HH H ) M Y No truncation effects.

83 Multistage Detection 64 Chip-Asynchronous Users Convergence of k-partial moments proven. Recursive expressions to construct them known. Proof follows the same lines, but makes use of properties of circulant matrices to cope with shifted bandlimited chip waveforms. For non-zero roll-off factor, chip-asynchronicity gives significant improvements at high SNR.

84 Multistage Detection 65 CDMA with Dual Antenna Arrays The system is described by the virtual NR K spreading matrix h 11 s 1 h 1 s... h 1K s K S = h 1 s 1 h s... h K s K h R1 s 1 h R s... h RK s K Note that with the Kronecker product : s k = h k s k Note also that the entries of S are not jointly independent even if those ones of S and H are.

85 Multistage Detection 66 A Resource Pooling Result Theorem Let the chips of any user be i.i.d. zero mean random variables with finite 6th moment, the sequences of all users jointly independent, and the antenna array channel h rk follow the i.i.d. complex Gaussian model. Then, the multi user efficiency of the linear MMSE detector converges for all users almost surely, as N, K but β = K N and R fixed, to the deterministic unique positive solution of the fixed point equation 1 = 1 + β x η MMSE R σn + η MMSE x dp à (x), if the powers of the users converge weakly to the limit distribution PÃ(x) with R Ãk = A k h rk. r=1

86 Multistage Detection 67 Resource Pooling for Correlated MIMO Channels Theorem 3 Let the chips of any user be i.i.d. zero mean random variables with finite 6th moment, the sequences of all users jointly independent, and the empirical distributions of the channel gains h rk across the users converge, jointly for all receive antennas r to an R-dimensional joint limit distribution P H (x). Then, with linear MMSE detection, the SINR of user k converges, as N,K but β = K and R fixed, conditioned N on the channel gains of user k to h H k Ah k σ where A is the deterministic unique positive definite solution of the matrix-valued fixed point equation A 1 xx H = I + β σn + x H Ax dp H(x), Asymptotic performance is characterized by an R R matrix.

87 Multistage Detection 68 MS Detection for Correlated Resource Pooling Let S = [ s 1, s,..., s K ] with s k = h k s k where S is i.i.d. the entries of H may be arbitrarily dependent as long as the rows have a joint limit distribution and are finite in number. Then, as the dimensions of S grow the k-partial moments conditioned on h k converge, recursive expressions for them are known, the proof follows along the same lines as before.