THE optimal detection of a coded signal in a complicated

Size: px
Start display at page:

Download "THE optimal detection of a coded signal in a complicated"

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER Achevable Rates of MIMO Systems Wth Lnear Precodng and Iteratve LMMSE Detecton Xaojun Yuan, Member, IEEE, L Png, Fellow, IEEE, Chongbn Xu, and Aleksandar Kavcc, Senor Member, IEEE Abstract We establsh area theorems for teratve detecton and decodng or smply, teratve detecton over coded lnear systems, ncludng multple-nput multple-output channels, ntersymbol nterference channels, and orthogonal frequency-dvson multplexng systems. We propose a lnear precodng technque that asymptotcally ensures the Gaussanness of the messages passed n teratve detecton, as the transmsson block length tends to nfnty. Area theorems are establshed to characterze the behavor of the teratve recever. We show that, for unconstraned sgnalng, the proposed sngle-code scheme wth lnear precodng and teratve lnear mnmum mean-square error LMMSE detecton s potentally nformaton lossless, under varous assumptons on the avalablty of the channel state nformaton at the transmtter. We further show that, for constraned sgnalng, our proposed sngle-code scheme consderably outperforms the conventonal multcode parallel transmsson scheme based on sngular value decomposton and water-fllng power allocaton. Numercal results are provded to verfy our analyss. Index Terms Lnear precodng, teratve LMMSE detecton, area theorem, superposton coded modulaton. I. INTRODUCTION A. Area Propertes THE optmal detecton of a coded sgnal n a complcated wreless envronment may ncur excessve computatonal complexty. Iteratve detecton provdes a low-cost soluton by decomposng the overall recever nto two or more local processors and conductng message-passng between the local processors to refne the detecton output see [] [] and the references theren. The analyss of an teratve detecton process s an ntrgung problem. The densty-evoluton technque [2] shows that carefully desgned low-densty partycheck LDPC codes can acheve near-capacty performance n addtve whte Gaussan nose AWGN channels wth teratve Manuscrpt receved March 8, 22; revsed August 6, 24; accepted August 6, 24. Date of publcaton August 9, 24; date of current verson October 6, 24. Ths work was supported by the Research Grant Councl of Hong Kong under Grant 4872 and Grant CtyU 83. Ths paper was presented at the 2 Internatonal Symposum on Informaton Theory. X. Yuan s wth the School of Informaton Scence and Technology, ShanghaTech Unversty, Shangha 23, Chna e-mal: yuanxj@ shanghatech.edu.cn. L. Png and C. Xu are wth the Department of Electrcal Engneerng, Cty Unversty of Hong Kong, Hong Kong e-mal: eelpng@ctyu.edu.hk; xchongbn2@ctyu.edu.hk. A. Kavcc s wth the Department of Electrcal Engneerng, Unversty of Hawa at Manoa, Honolulu, HI USA e-mal: kavcc@hawa.edu. Communcated by D. Guo, Assocate Edtor for Shannon Theory. Color versons of one or more of the fgures n ths paper are avalable onlne at Dgtal Object Identfer.9/TIT message-passng decodng algorthms. It was further shown n [2] that the achevable rate of an teratve scheme for an erasure channel can be measured by the area under the so-called extrnsc nformaton transfer EXIT curves [3] and the channel capacty s approachable when the two local processors have matched EXIT curves. Ths area property s extended n [4] to scalar AWGN channels or smply, AWGN channels usng the measure of mnmum mean-square error MMSE, n whch a suffcent condton s establshed to approach the capacty of a bnary-nput AWGN channel wth teratve detecton. It s commonly accepted that, wth random nterleavng, the extrnsc nformaton.e., the messages from an a posteror probablty APP decoder for a bnary forward-error-control code can be modeled as a sequence of observatons from an effectve scalar AWGN channel [3]. Thus, the authors n [4] made two basc assumptons: the messages passed between the local processors are modeled as the observatons from an effectve scalar AWGN channel referred to as the AWGN assumpton and the localprocessorsareoptmaln the sense of APP detecton/decodng. Assumpton ensures that each local processor can be characterzed by a snglevarable transfer functon nvolvng sgnal-to-nose rato SNR and MMSE. The mutual nformaton and MMSE relatonshp establshed n [5] s then appled to derve the area property, and capacty-approachng performance s proven when the transfer curves of the local processors are matched. B. MIMO Channels The work n [4] s lmted to scalar channels. A straghtforward extenson to multple-nput multple-output MIMO channels s to convert a MIMO channel nto a set of ndependent scalar sub-channels usng lnear processng based on sngular value decomposton SVD, to allocate power among these sub-channels usng the water-fllng WF prncple [44] or ts varaton mercury water-fllng [6], and then to apply varable-rate codng to these sub-channels wth one codebook assgned to one sub-channel. Hence the results n [4] can be appled to each sub-channel ndvdually. Ths mult-code SVD-WF approach wth varable-rate codng s conceptually smple, but may encounter the followng dffcultes n practce. Frst, the rate and power par obtaned by SVD-WF for each sub-channel vares and so a large number of encoders wth varous rates are requred. The related complexty can be IEEE. Personal use s permtted, but republcaton/redstrbuton requres IEEE permsson. See for more nformaton.

2 774 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 reduced by rate quantzaton, but performance may deterorate notceably when quantzaton nterval ncreases. Second, ths SVD-WF approach requres accurate channel state nformaton at the transmtter CSIT. In practce, CSIT error can be caused by the Doppler effect or lmted backhaul capacty. In general, perfect orthogonalty between the sub-channels cannot be establshed n the presence of CSIT error. Then the optmal transcever desgn based on the SVD-WF prncple becomes a complcated ssue. The above dffcultes motvate us to study alternatve transmsson technques for MIMO systems wthout orthogonal channel decomposton. Lnear precodng LP [2] [26] can be used for ths purpose. Iteratve detecton can be appled to MIMO systems wth LP. However, the extenson of the area property n [4] to MIMO systems wthout orthogonal channel decomposton s not straghtforward. Ths s because a multvarate functon s requred n such a system to characterze the recever s behavor, but then assumpton mentoned earler may not hold. Also, optmal decodng for a MIMO system wthout orthogonal channel decomposton can be too costly, especally n the recent development of massve MIMO systems. Lnear MMSE LMMSE detecton [8] has the advantage of low complexty, but t s not optmal for non- Gaussan sgnalng, mplyng that assumpton may not hold ether. C. Contrbutons of Ths Paper In ths paper, we consder a jont LP and teratve LMMSE detecton scheme. We show that the proposed lnear precodng technque ensures that the AWGN assumpton asymptotcally holds for the output of the LMMSE detector, provded that the transmsson block length s suffcently large. Ths allows us to use a sngle par of nput and output parameters to characterze the behavor of the LMMSE detector. Also, we adopt superposton coded modulaton SCM [9], [2] to approxmate Gaussan sgnalng, for whch the local LMMSE detecton s near-optmal. These treatments ensure the valdty of the two assumptons n Secton I.A for MIMO setups. Based on ths, we establsh area theorems for the proposed LP and teratve LMMSE detecton LP-LMMSE scheme n MIMO channels, under the assumptons of perfect and partal CSIT, respectvely. We show that, for unconstraned sgnalng, the proposed LP-LMMSE scheme wth a sngle code s potentally nformaton lossless based on the curve-matchng prncple, even though the suboptmal teratve LMMSE detecton technque s employed. The above area theorems provde gudelnes n the practcal desgn of the LP-LMMSE scheme. Specfcally, we show that optmal power allocaton for non-gaussan nputs can be obtaned by solvng a convex optmzaton problem. Also, effcent practcal FEC codes, such as LDPC codes, can be desgned based on the curve-matchng prncple. Furthermore, we show that, for non-gaussan sgnalng, the proposed snglecode LP-LMMSE scheme acheves much hgher rates than the mult-code SVD-WF scheme when the encodng and decodng complextes of the two schemes are kept at a comparable level. From nformaton theory, the capacty of a MIMO channel can be acheved usng a sngle random code wth SVD and WF [58]. However, the realzaton of such a snglecode scheme wth affordable encodng and decodng complexty remans a challengng problem. As such, one of the major contrbutons of ths paper s to show a practcal sngle-code scheme wth LP and teratve LMMSE detecton that acheves the MIMO capacty. D. Comparsons Wth Exstng LP Technques It s nterestng to compare the proposed lnear precodng technque wth other alternatves see [2] [28] for full CSIT and [29] [34] for partal CSIT. From an nformaton-theoretc vewpont, the ultmate crteron for precoder desgn s to acheve the channel capacty see [23], [33], [34] and the references theren. It s well known that channel codng s requred to acheve the capacty. However, most exstng works on precoder desgn focus on un-coded MIMO systems equpped wth a sgnal detector at the recever performng far from optmal. Along ths lne, a varety of desgn crtera have been studed, such as par-wse error probablty mnmzaton [22], max-mn sgnal-to-nterference-plus-nose rato SINR [23], average SINR maxmzaton [23], SINR equalzaton [24], [25], and MMSE [26], [27], etc. The works for un-coded systems can be appled to coded systems by concatenatng the detector wth a decoder. Unfortunately, the resdual nterference at the output of the detector n general results n a consderable performance loss [36], [55], [56]. Iteratve detecton and decodng can effcently suppress the resdual nterference left by the detector. Then, a major challenge s to jontly desgn FEC codng and lnear precodng at the transmtter, takng nto account the effect of teratve detecton at the recever. Ths paper provdes a smple soluton to ths problem. Our analyss shows that, wth curve-matchng codes, the proposed LP-LMMSE scheme s capacty-achevng under varous assumptons on CSIT. E. Comparsons Wth the SVD-WF Approach As aforementoned, the conventonal SVD-WF approach s conceptually smple, provded that deal Gaussan sgnalng s used and perfect CSIT s avalable. For non-gaussan nputs, the mercury water-fllng MWF technque [6] can be used for power allocaton among dfferent egen-modes after SVD. However, t s shown n [4] and [42] that the optmal lnear precoder for non-gaussan nputs s n general non-dagonal even for ndependent parallel channels. To the best of our knowledge, t s computatonally ntensve to determne the optmal precoder n ths case. Furthermore, the SVD-WF approach requres perfect CSIT that s dffcult to acqure n practce, e.g., due to the Doppler effect or lmted backhaul capacty. CSIT uncertanty n general mpars the orthogonalty between the parallel sub-channels. Thus, effcent nterference cancellaton technques, such as teratve detecton and decodng, are requred at the recever [4], [5], [7], [].

3 YUAN et al.: ACHIEVABLE RATES OF MIMO SYSTEMS 775 The above dscussons mply that lnear precodng and teratve detecton are necessary n the practcal desgn of an effcent MIMO transcever. In ths regard, the LP-LMMSE scheme developed n ths paper provdes a low-cost soluton to the problem. In partcular, only one encoder s requred n the proposed LP-LMMSE scheme. Ths s much smpler than the SVD-WF scheme that requres multple encoders wth varous rates. Further, the proposed LP-LMMSE scheme can acheve hgher nformaton rates than SVD-WF and ts varaton SVD-MWF for practcal systems wth non-gaussan nputs, as demonstrated by the numercal results provded later n Secton VI. F. Outlne of the Remander of the Paper Secton II descrbes our system model and teratve LMMSE detecton under consderaton. Secton III descrbes lnear precodng for MIMO systems wth perfect CSIT, and characterzes the correspondng teratve recever usng SINR-varance evoluton. In Secton IV, we establsh the area theorems and consder the precoder desgn. Secton V extends our results to MIMO channels wth CSIT uncertanty. Secton VI provdes numercal results, and Secton VII offers some concludng remarks. II. SYSTEM MODEL AND ITERATIVE LMMSE DETECTION A. Generc Lnear System A generc complex-valued lnear system s modeled as y Ax + η where y s a J -by- receved sgnal vector, A s a J -by-j transfer matrx, x [x, x 2,,x J ] T s a J-by- transmt sgnal vector, and η CN,σ 2 I a J -by- addtve nose vector. Notatons and I, respectvely, represent an all-zero vector and an dentty matrx wth a proper sze. Note that both J and J are ntegers, wth the meanngs revealed later. Throughout ths paper, we assume full channel state nformaton at the recever,.e., the recever perfectly knows the transfer matrx A. We wll dscuss the stuatonsof both perfect and partal CSIT. B. Messages The transmtter structure for the system n s shown n the upper part of Fg.. The forward-error-correcton FEC encoder generates a frame of JK coded symbols, denoted by x [x, x 2,...,x JK ]T where K s the number of the uses of system n a codeword frame. These coded symbols are randomly nterleaved by the nterleaver and then parttoned nto K segments wth equal length J. Each segment serves as an nput x to the system. We emphasze that x and x are not of the same length, for that x represents the symbols n an overall codeword frame consstng of K segments, whle x only represents the symbols n any one of these segments. The teratve recever, as llustrated n the lower half of Fg., conssts of two local processors, namely, the detector Fg.. The transcever structure of the proposed scheme over the generc lnear system n. s the nterleaver and s the correspondng de-nterleaver. and the decoder, nter-connected by the nterleaver and the de-nterleaver. Partcularly, the detector consdered n ths paper follows the LMMSE prncple, hence the name LMMSE estmator. The LMMSE estmator estmates x based on the channel observaton y and the messages from the decoder denoted by α. The outputs of the estmator are extrnsc messages denoted by β. The message set α s defned as follows. Denote by x the th entry of x. Each x s constraned on a dscrete sgnalng constellaton S {s, s 2,...,s S },where S represents the sze of S. We always assume that each x s randomly and unformly taken over the constellaton ponts n S. Ths assumpton does not lose any generalty snce sgnal shapng e.g., to approach Gaussan sgnalng can be realzed by properly desgnng the constellaton ponts of S. For each use of system, let α be a vector of J messages as α [α,α 2,...,α J ] T, and each message α s a set of S lkelhood values for x,.e. α {α, α 2,...,α S }, where α k represents the lkelhood of x s k S pror to the processng of the detector, and S α k. k Smlar notaton apples to the message vector β [β, β 2,...,β J ] T. The decoder decodes x based on the nput message set β, and outputs the extrnsc message set α. Decodng s performed on the overall frame, and so both α and β contan JK messages. Let x be the th entry of x,andβ k be the lkelhood of x s k S pror to decodng. Then, we can express β [β,β 2,...,β JK ]T where β {β, β 2,...,β S } for,...,jk. Messages n α allow smlar expressons. After decodng, α are nterleaved and parttoned to form the nput of the LMMSE estmator, whch completes one round of teraton.

4 776 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 The LMMSE estmator and the decoder are executed teratvely untl convergence. C. Basc Assumptons Here we dscuss some basc assumptons used throughout ths paper. For notatonal convenence, we ntroduce an auxlary random varable, denoted by a, to represent the nformaton carred by each α. The condtonal probablty of a gven x s defned as pa x s k α k. Smlarly, an auxlary random varable b s defned for each β. The condtonal probablty of b gven x s gven by pb x s k β k. Denote a [a,...,a J ] T and b [b,...,b JK ]T. We make the followng assumptons on the nputs of the local processors. Assumpton : For the detector, each x s ndependently drawn from S wth equal probablty,.e., px s k / S, forany and k; the messages n a are condtonally ndependent and dentcally dstrbuted..d. gven x,.e. J pa x pa x, and for any dummy varable t and k, 2,..., S, pa t x s k pa j t x j s k. Assumpton 2: For the decoder, the messages n b are condtonally..d. gven x,.e. pb x JK pb x, and for any dummy varable t and k, 2,..., S, pb t x s k pb j t x j s k. Wth Assumpton, the jont probablty space of x, a, andy seen by the detector can be expressed as J px, a, y px pa x py x, 2a where py x s determned by. Smlarly, wth Assumpton 2, the jont probablty space of x and b seen by the decoder s px, b px JK pb x 2b where x s assumed to be unformly taken over the codebook,.e., px s a unform dstrbuton over the codebook. Remark : Assumptons and 2 decouple the probablty space seen by the detector and decoder, whch smplfes the analyss of the teratve process. Smlar assumptons have The auxlary random varables are ntroduced for notatonal convenence. Usually, an teratve recever drectly evaluates β wthout explctly calculatng {b }. However, we wll see later that {b } are ndeed calculated n the teratve LMMSE detecton, based on whch β s determned. been wdely used n teratve decodng and turbo equalzaton algorthms [2], [3], [7], [37]. We emphasze that, n fact, Assumptons and 2 are requred only for those nvolved symbols n the computaton tree of the teratve algorthm at a depth of the maxmum number of teratons. Assumpton only nvolves J symbols, and thus can be made vald by random nterleavng as K tends to nfnty. However, strctly speakng, Assumpton 2 cannot be justfed n ths way, for that ths assumpton nvolves JK symbols and the correspondng messages are generally correlated due to the LMMSE detecton operaton, no matter what nterleaver s used. A remedy to ths ssue s to requre Assumpton 2 hold only for the neghborhood of the computaton tree, nstead of for all the JK symbols n the codeword. Ths remedy wll make the dervaton of the results n ths paper more rgorous, but wll complcate the notaton and the related dscussons. For ease of presentaton, we do not adopt ths remedy and keep Assumpton 2 as t s. D. LMMSE Estmaton The detector delvers the extrnsc messages defned as β k p x s k a, y, for, 2,...,J and k,..., S where the jont probablty space of x, a,andy s gven n 2a, and a represents the vector obtaned by deletng the th entry of a. Here, extrnsc means that the contrbuton of the aprormessage a s excluded n calculatng each β. For system, a drect evaluaton of β k may be excessvely complcated. The LMMSE estmaton s a low cost alternatve. It s suboptmal n general, but t s optmal f x are generated usng Gaussan sgnalng. Denote the mean and covarance of x seen by the detector by x [ x,..., x J ] T and vi, respectvely, wth S x E[x a ] α ks k 3a and k [ v E x x 2], for any ndex, 3b where the expectaton s taken over the jont dstrbuton of a and x. From Assumpton, v s nvarant wth respect to the ndex. In practce, v s approxmated by the sample varance as J S v J α k s k x 2, k whch converges to the true varance when J. The LMMSE estmator of x gven y s [38] ˆx x + va H R y A x where R s the covarance matrx of y gven as R vaa H + σ 2 I. 4a 4b Recall that the extrnsc message β should be ndependent of a. To meet ths requrement, we calculate the extrnsc mean and varance for x denoted by b and u, respectvely

5 YUAN et al.: ACHIEVABLE RATES OF MIMO SYSTEMS 777 by excludng the contrbuton of a accordng to the Gaussan message combnng rule cf., 54 and 55 n [39] as u M, v 5a and b ˆx u M, x v, 5b where M, represents the, th entry of the MMSE matrx M vi v 2 A H R A. 5c Fnally, the output messages of the detector can be calculated as pb x s k β k S k pb, for any and k. 5d x s k In the above, β k can be readly calculated by assumng that b s an observaton of x over an effectve AWGN channel wth nose power u. The justfcaton of ths assumpton can be found n Lemma n Secton III.C. We note that b n 5b s just the auxlary varable of β k defned n Secton II.C; see Footnote. E. APP Decodng The decoder decodes x based on the messages β followng the APP decodng prncple. The extrnsc output of the decoder for each x s defned as α k p x s k b 6 for, 2,..., JK and k,..., S, where the jont probablty space of x and b s gven n 2b. Assumpton 3: The local decoder performs APP decodng. In practce, APP decodng s usually computatonally expensve. Low-complexty message-passng algorthms can be used to acheve near-optmal performance, provded that the code structure allows a sparse graphc descrpton [37]. Messagepassng decodng s well-studed n the lterature, and thus the detals are omtted here. Compared wth APP decodng, the performance loss of message-passng decodng s usually margnal. Ths loss s not of concern n ths paper. Therefore, we ntroduce Assumpton 3 to smplfy our analyss. III. LINEAR PRECODING WITH PERFECT CSIT In ths secton, we assume perfect CSIT. The case of mperfect CSIT wll be dscussed n Secton V. We propose a lnear precodng technque to equalze the output SINRs of the detector. We then establsh an SINR-varance transfer chart technque to analyze the performance of the proposed LP-LMMSE scheme. A. MIMO Channels The teratve LMMSE detecton prncple descrbed n the prevous secton can be appled to any coded lnear systems, ncludng MIMO channels, nter-symbol nterference ISI channels, and orthogonal frequency-dvson multplexng OFDM systems, etc. In the followng, as an example, we establsh a connecton between a MIMO channel and the system n. A Gaussan MIMO channel wth N transmt and M receve antennas can be modeled as ỹ l H x l + η l 7 where l represents the l-th channel use, ỹ l s an M-by- receved sgnal vector, H s the M-by-N channel transfer matrx known at both the transmtter and the recever, x l s an N-by- transmt sgnal vector, and η l CN, σ 2 Isan M-by- addtve nose vector. A transmsson block.e., one use of system nvolves J/N uses of the channel 7, where J s the block length defned n Secton II.A. We assume that J and J are properly chosen so that J /M J/N s an nteger representng the number of channel uses n system. Denote ỹ [ỹ T,...,ỹT J/N ]T x [ x T,..., xt J/N ]T η [ η T,..., ηt J/N ]T. Combnng J/N channel uses, we wrte an extended system as ỹ H x + η where the extended channel s gven by H I J/N H 8a 8b wth beng the Kronecker product and I n beng an n-by-n dentty matrx. The sgnal power s constraned as E [ x H x ] /J P. 8c The SVD of H s gven by H UΛV H 9 where Λ s an JM/N-by-J dagonal matrx wth non-negatve dagonal elements, and U and V are untary matrces. We assume that U and V are chosen such that the dagonal entres of Λ are asymptotcally uncorrelated as J tends to nfnty. 2 Ths property s useful n establshng Lemma n Secton III.C. B. Lnear Precodng We focus on the followng lnear precodng operaton: x VW /2 Fx a where x and x are gven n 8a and, respectvely, V s defned n 9, W s a dagonal matrx wth non-negatve dagonal elements for power allocaton, and F s the normalzed DFT matrx wth the, kth entry gven by wth j. F, k J /2 exp j2π k /J b 2 Although H s determnstc, the orderng of ts sngular values can be arbtrarly chosen. Here we choose such a random orderng that the dagonal of Λ s an asymptotcally uncorrelated sequence as J tends to nfnty.

6 778 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 From Assumpton, the entres of x are uncorrelated. Wthout loss of generalty, we further assume that the entres of x have normalzed power. Denote by W, the, -th dagonal entry of W. Then the power constrant n 8c becomes J J W, P. The optmzaton of W wll be detaled n Secton IV. The use of F n a s to ensure that the SINRs are equal for all symbols after LMMSE detecton; see Secton III.C. Incdentally, the choce of the DFT matrx for F also allows the fast Fourer transform FFT algorthm n the mplementaton of LMMSE detecton []. We note that smlar DFT-based precoders have been used n MIMO systems for other purposes, e.g., for harnessng dversty n [46]. At the recever sde, the receved vector ỹ s post-processed by the matrx U H : Combnng 8 2 and lettng y U H ỹ. 2 D ΛW /2, we obtan an equvalent channel as y DFx + U H η Ax + η 3a 3b where A DF and η U H η CN,σ 2 I snce U s untary. Clearly, 3b has the same form as. The teratve detecton and decodng procedure outlned n Secton II can be drectly appled to 3b. C. Characterzaton of the Estmator We now am to characterze the behavor of the estmator. It was shown n [23] that, wth the precoder n a, the SINR becomes unform for non-teratve LMMSE estmaton. We next show that a smlar property holds n our proposed teratve system. We further show that, for the extended system n 8a, the resdual nterference n the output of the estmator s asymptotcally Gaussan, followng the central lmt theorem. To start wth, we establsh the fact that M,, the th dagonal element of M n 5c, s not a functon of. Tosee ths, we substtute 4b and A DF nto 5c: Then M vi v 2 F H D H vdd H + σ 2 I DF. M, v v 2 f H DH vdd H + σ 2 I Df J v J v 2 Dk, k 2 vdk, k 2 + σ 2 J k J v + k 4a 4b Dk, k2 σ 2 4c where f s the th column of F, and 4b utlzes b. Clearly, M, s the same for all. We now ntroduce a defnton: φv M, v. Note that φv defned above s not a functon of the ndex,as M, s nvarant wth respect to. Recall that {Dk, k Λk, kwk, k /2, k, 2,...,J} andthat{λk, k} are the sngular values of H n 8b and so {Λk, k} contans J/N copes of the sngular values of H denoted by λ,...,λ N. We assume that a same amount of power s allocated to any two egen-modes wth the same channel gans,.e., f Λk, k Λ j, j, thenwk, k W j, j. 3 Then, {Dk, k} haveatmostn dfferent values, denoted by d,...,d N. Together wth 4c and the defnton of φv above, we obtan N φv N v + d2 n σ 2 v. 5 n Remark 2: We descrbe some useful propertes of φv. Frst, as the sgnal power.e., the power of each entry of x s normalzed to, the varance v n 5 vares from to. Second, φv s monotoncally decreasng n v, mplyng that for the LMMSE estmator the hgher the nput varance, the lower the output SINR. Thrd, as v,φv tends to the average SNR gven by φ N dn 2 N σ 2. n Remark 3: We emphasze that F s chosen to meet the followng crtera: a F s untary; b Fx allows fast computaton wth complexty OJlogJ; c the dagonal elements of FTF H s nvarant for any real dagonal matrx T and so are the dagonal elements of M. The choce of F to meet these crtera s not unque. Besdes the DFT matrx, we may alternatvely choose F to be, e.g., the Hadamard matrx. We next show that the outputs of the LMMSE estmator can be modeled as the observatons from an AWGN channel provded that J s suffcently large, and the related SINR s gven by ρ φv. Defne v n M, ρ f H DH v DD H +σ 2 I Ax \ x \ +η 6 where x \ or x \ represents the vector obtaned by settng the th entry of x or x to zero. It s clear that n s ndependent of x. Moreover, we have the followng result wth the proof gven n Appendx A. Lemma : For any ndex, b n 5d can be expressed as b x + n, 7 where n s ndependent of x and ts dstrbuton converges to CN, /φv as J. In the above, n represents the resdual nterference plus nose at the output of the LMMSE estmator n teraton. Hence, ρ φv n 5 represents the related SINR. It s well-known that the resdue error of the LMMSE estmaton s approxmately Gaussan [38], [52]. Lemma reveals that 3 Ths treatment doesn t ncur any nformaton loss, as seen from Theorem.

7 YUAN et al.: ACHIEVABLE RATES OF MIMO SYSTEMS 779 ths approxmaton becomes exact for the precoder n a over the extended channel n 8a for a suffcently large J. We henceforth always assume that J s suffcently large, so that each b can be modeled as an observaton of x from an effectve AWGN channel wth SNR φv. D. Characterzaton of the Decoder We now consder the characterzaton of the decoder s behavor. The decoder performs APP decodng upon recevng the messages b modeled as ndependent observatons of x over an AWGN channel wth SNR ρ cf., Lemma. Defne the extrnsc varance of each x as v MMSEx b E [ x E[x b ] 2], 8a where the expectaton s taken over the jont probablty space of x and b. We denote by v the average of v over the ndex. Clearly, the average varance v s a functon of ρ, denoted as v ψρ. 8b Note that 8b s referred to as the SINR-varance transfer functon of the decoder. Remark 4: For practcal FEC codes, the output varances {v } generally vary wth the ndex. Then, the queston s whether the average varance v s a good performance measure of the decoder output, or equvalently, whether the behavor of the estmator solely depends on v. To answer ths queston, we recall the estmator s sgnal model n 3b: y DFx + η. Letv be the aprorvarance of x. Note that {v } are obtaned by nterleavng {v }. From Assumpton, the messages from the decoder are uncorrelated. Thus, the apror covarance matrx of x s gven by dag {v, v 2,...,v J }. Then, the a pror covarance of the vector Fx s gven by Fdag{v, v 2,...,v J } F H. Clearly, Fdag{v, v 2,...,v J } F H s a crculant matrx, and ts dagonal s a constant gven by the average varance v. Further, due to random nterleavng, {v, v 2,...,v J } s a random sequence. Then, t can be shown that the off-dagonals of Fdag{v, v 2,...,v J } F H tend to zero as J tends to nfnty, or equvalently, the aprorcovarance of Fx s approxmately a scaled dentty matrx gven by vi. Ths mples that, provded that J s suffcently large, the performance of the LMMSE detector only depends on the average output varance v of the decoder. Ths justfes the use of v to characterze the output of the decoder. We next establsh a relaton between the rate of the FEC code per symbol of x and the transfer functon ψρ.our result s based on the SCM [9], [2]. The detal can be found n the lemma below, wth the proof gven n Appendx B. Lemma 2: Assume ψρ satsfes the followng regularty condtons: ψ andψρ, for ρ [, ; 9a monotoncally decreasng n ρ [, ; 9b contnuous and dfferentable everywhere n [, except for a countable set of values of ρ; v lm ρψρ. 9d ρ 9c Let Ɣ n be an n-layer SCM code wth the SINR-varance transfer functon ψ n ρ and the rate R n. Then, there exst {Ɣ n } such that: ψ n ρ ψρ, foranyρ andany nteger n; as n, R n R ρ + ψρ dρ. 2 We now gve an ntutve explanaton of Lemma 2. Let the MMSE of each x after decodng be mmseρ MMSEx b E [ x E[x b ] 2], 2a where the expectaton s taken over the jont probablty space of x and b. Compared wth 8, the MMSE n 2a s obtaned by ncludng the contrbuton of b n estmatng x. We next establsh a connecton between ψρ and mmseρ based on the followng two hypotheses: x s Gaussan, and the extrnsc message of x s also Gaussan. Wth these hypotheses, we conclude that the condtonal dstrbuton of x gven b s also Gaussan and gven by px b CNE[x b ], ψρ. 2b Then, mmseρ and ψρ are related as mmseρ MMSEx b MMSEx b, b MMSEx b, x px b MMSEx b, x CNE[x b ], ψρ ρ + ψρ. where the thrd equalty follows from the fact that b x x b forms a Markov chan from Assumpton 2 n Secton II.C, the fourth equalty from 2b, and the last one utlzes the fact that b can be modeled as an AWGN observaton n Lemma. From [4] see [4, Lemma ], we express the code rate as R mmseρdρ ρ + ψρ dρ 22 whch s exactly the rate lmt gven n Lemma 2. Recall that the above dervaton s based on two Gaussan hypotheses. These hypotheses are dffcult to meet exactly, as practcal systems mostly employ dscrete sgnalng. However, the concdence of 2 and 22 mples that SCM can approxmate Gaussan sgnalng as the number of SCM layers tends to nfnty, and therefore, the rate n 22 s ndeed approachable by properly constructng an SCM-based FEC code. E. SINR-Varance Transfer Chart From the prevous dscussons, the LMMSE estmator can be characterzed by ρ φv; and smlarly, the decoder can be characterzed by v ψρ. The teratve process of the estmator and decoder can be tracked by the recurson of ρ and v. Letq be the teraton number. We have ρ q φv q and v q ψρ q, q, 2,...

8 78 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 Fg. 2. An llustraton of the SINR-varance transfer chart. The recurson contnues and converges to a pont v satsfyng φv ψ v and φv >ψ v, for v v, ] where ψ s the nverse of ψ whch exsts snce ψ s contnuous and monotonc [43]. Note that v snce the sgnal power s normalzed; v mples that x can be perfectly recovered. The above recursve process s llustrated by the SINR-varance transfer chart n Fg. 2. We say that the estmator and the decoder are matched f φv ψ v, for v, ]. 23 Note that: φ > snce the estmator s output always contans the nformaton from the channel even f there s no nformaton from the decoder; and φ < snce the estmator s extrnsc output cannot resolve the uncertanty ntroduced by the channel nose even f the messages from the decoder are perfectly relable. Then, we equvalently express the curve-matchng condton 23 as ψρ φ φ, for ρ<φ; ψρ φ ρ, for φ ρ<φ; ψρ, for φ ρ<. 24a 24b 24c The above curve-matchng prncple plays an mportant role n establshng the area theorems, as seen n the next secton. IV. AREA THEOREMS AND PRECODER OPTIMIZATION In ths secton, we establsh area theorems for the proposed LP-LMMSE scheme. We further dscuss the optmzaton of the power matrx W. A. Area Property for Unconstraned Sgnalng The man theorem of ths paper s presented below, wth the proof gven n Appendx C. Theorem : As J, f the detector and the decoder are matched, then an achevable rate per antenna per channel use of the LP-LMMSE scheme s gven by R N N n log + d2 n σ Remark 5: The curve-matchng condton 23 s assumed n achevng the rate n 25; see the proof n Appendx C. Then, a fundamental queston s whether ths curve-matchng condton can be fulflled for any channel realzaton. The answer s affrmatve. To see ths, we recall from Lemma 2 that an SCM code can be constructed to match an arbtrary functon ψρ satsfyng the regularty condtons n 9 and to acheve the rate n 2. Then, wth 23, the queston can be rephrased as: for any channel realzaton, does the matched ψρ n 24 satsfy the regularty condtons n 9? Wth φv gven by 5 and the propertes of φv n Remark 2, t s straghtforward to verfy that the regularty condtons n 9 are ndeed satsfed for any channel realzaton. Note that a typcal realzaton of φ ρ s llustrated n Fg. 2. Remark 6: We remark that 25 s the nput-output mutual nformaton of the channel n 3b wth Gaussan nputs. Ths observaton mples that our proposed scheme s potentally nformaton lossless. In addton, the rate R n 25 s acheved usng the curve-matchng SCM code wth the number of SCM layers tends to nfnty. Later, we wll show by numercal results that, for the proposed LP-LMMSE scheme, practcal values of R can be closely approached usng a small number of SCM layers. B. Water-Fllng Precodng Now we consder the optmzaton of the power matrx W. Recall from 3a that D ΛW /2, or equvalently, d λ w /2,for,...,N. Then, 25 becomes R N log + λ2 n w n N σ n We am at maxmzng the above rate over W subject to the power constrant n. Clearly, the soluton concdes wth the renowned water-fllng power allocaton [44]. Corollary : Under the condtons of Theorem, f W follows water-fllng power allocaton, then the LP-LMMSE scheme acheves the water-fllng capacty of the channel n 8a. Corollary shows that lnear precodng, together wth proper error-control codng and teratve LMMSE detecton, can potentally acheve the water-fllng channel capacty. Ths achevablty s based on a sngle code wth a matched decodng transfer functon. C. Area Property for Constraned Sgnalng So far, we have shown that the proposed lnear precodng scheme s asymptotcally optmal n the sense of achevng the water-fllng capacty as the sgnalng approaches Gaussan. However, n practce, the sgnalng s usually constraned on a fnte-sze dscrete constellaton. We next establsh an area theorem for constraned sgnalng.

9 YUAN et al.: ACHIEVABLE RATES OF MIMO SYSTEMS 78 Defne the γ -functon as [ γρ E x E[x x + η] 2] 27 where x s unformly taken over S, andη s ndependently drawn from CN, /ρ. We have the followng result wth the proof gven n Appendx D. Theorem 2: Suppose that the detector s nputs {a } are modeled as ndependent observatons of {x } from an effectve AWGN channel. Then, as J, f the detector and the decoder are matched, an achevable rate of the LP-LMMSE scheme s R log S + γρ + φγ ρdρ 28 where the φ-functon s gven by 5, and γ s defned n 27. Remark 7: Theorem 2 holds under the AWGN assumpton on {a }. It was observed that ths assumpton s emprcally true for BPSK, QPSK, and SCM wth BPSK/QPSK layers. However, ths assumpton may be far from true f other modulaton technques, such as bt-nterleaved coded modulaton [47], are employed. In ths later case, the area theorem based on the measure of mean-square error MSE establshed n [48] can be used for performance evaluaton. We refer nterested readers to [48] for detals. Remark 8: Theorem 2 can be readly extended to the case of dscrete non-unform nput of system. The only dfference s that, n the defnton of the γ -functon n 27, the dstrbuton of x s replaced by a non-unform dstrbuton over S. Correspondngly, log S n 28 s replaced by the entropy of x calculated based on the non-unform dstrbuton. Remark 9: Theorem and Corollary reveal that our proposed lnear precodng and teratve LMMSE detecton scheme s capacty-achevng. It s noteworthy that there are other approaches to acheve the channel capacty. One smple approach s to use SVD to convert the MIMO channel nto a set of ndependent parallel channels, and then to conduct water-fllng power allocaton over the parallel channels. It s desrable to compare the performance and complexty of our proposed scheme wth the SVD-MWF scheme n practcal systems wth non-gaussan nputs. Theorem 2 can be used n ths comparson. Later, we wll numercally demonstrate that, n ths case, our proposed scheme consderably outperforms the SVD-MWF scheme. D. Precoder Optmzaton for Constraned Sgnalng Recall that, for Gaussan sgnalng, the optmal power matrx W s the water-fllng soluton. However, water-fllng s not necessarly optmal for dscrete sgnalng. Based on the area property n Theorem 2, we can optmze W to maxmze the achevable rate of the LP-LMMSE scheme. Ths problem can be formulated as: maxmze log S + γρ+ φγ ρdρ 29a subject to N N w n P. 29b n In the above, P s the maxmum sgnal power gven n 8c. The followng result s useful n solvng 29. Lemma 3: The rate R n 28 s a concave functon of {w,...,w N }, provded that γρ s a convex functon of ρ. The proof of Lemma 3 can be found n Appendx E. The γ -functon s convex for most commonly used sgnalng constellatons [43]. From Lemma 3, the problem n 29 can be solved usng standard convex programmng [49]. V. EXTENSIONS TO MIMO CHANNELS WITH CSIT UNCERTAINTY In ths secton, we extend the results n Sectons III and IV to the general case that channel state nformaton s not perfectly known at transmtter. A. Lnear Precodng Wth Imperfect CSIT We now consder MIMO systems wth partal CSIT. Here, partal CSIT means that the channel matrces are not exactly known, nstead, only the statstcs of the channel s known at the transmtter sde. The precodng technque n Secton III.B requres perfect CSIT, and so cannot be drectly appled here. The followng s a modfed soluton. Return to the extended system n 8a ỹ H x + η, 3a where the extended channel contans J/N channel realzatons: H H 2 H.... 3b H J/N We focus on the ergodc case, n whch {H } are ndependent realzatons of an M-by-N random matrx, wth abuse of notaton, denoted by H. Ths model also ncludes orthogonal frequency-dvson multplexng OFDM systems [4] wth ndependent fadng over dfferent sub-carrers. The channel nput x s related to x as x P Π Fx 3a wth P dag{p, P,...,P} F dag{f, F,...,F}. 3b 3c where P s a J-by-J block-dagonal matrx wth P of sze N-by-N, Π s a J-by-J permutaton matrx, and F s a J-by-J block- dagonal matrx wth each block F beng the normalzed DFT matrx of sze L-by-L. Combnng 3a and 3 and lettng y ỹ and η η, we obtan an equvalent channel as wth y Ax + η A H P Π F. The power constrant now becomes 32a 32b N tr{pp H } P. 33

10 782 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 Wth abuse of notaton, defne [ { φv N E tr v I + σ 2 HQHH }] v. 34 where Q PP H, and the expectaton s taken over the dstrbuton of H. We am to show that ρ /u φv, for,...,j. 35 Wth 32b, we can rewrte 5c as Fg. 3. An llustraton of the proposed precoder for mperfect CSIT wth J 6 and N 4. The precodng matrx P dag{p,...,p} allows the precoder to explot the beneft provded by the avalable CSIT. 4 The optmzaton of P s brefly dscussed n Secton V.C. The precodng matrx F ensures that every coded symbol n x s suffcently dspersve over tme and space. Thus, t s requred that the DFT sze L s suffcently large. A convenent choce of L s L J/N, and then there are N length-l DFT matrces n F. Now we consder the desgn of the permutaton matrx Π. Let q [q T,...,qT N ]T Fx and x [x T,...,xT N ]T,where each DFT block q Fx s of sze L-by-. Also denote c [c T,...,cT J/N ]T Πq, where each c s an N-by- vector. Thetwocrterafor Π are lsted below. For each, the entres of q are transmtted at dfferent channel uses,.e., no two entres of q are connected to asamec j for any ndex j,...,j/n; and Treat PH k as the kth realzaton of the equvalent channel PH. Then, any N consecutve entres of q are transmtted at dfferent transmt-antennas of the equvalent channel PH. As a result, for each ndex j, the set of {the jth entry of c k k,...,j/n} contans J/N 2 entres of q,,...,n. 5 The choce of Π satsfyng the above crtera s not unque. A smple choce s as follows: for each ndex k, thekth entry of q s connected to the k mod Nth nput of the equvalent channel PH k ; then, for j 2,...,J/N, the connecton pattern of each q j s just a one-entry cyclc shft of the prevous one. An example for J 6 and N 4 s llustrated n Fg. 3. The above choce of F and Π ensures that the behavor of the teratve recever can be characterzed by a sngle-varable recurson, as wll be detaled n the next subsecton. B. SINR-Varance Transfer Chart Iteratve LMMSE detecton descrbed n Secton II s appled to the system n 32. We next show that the outputs of the detector can stll be characterzed by a sngle SINR value. 4 We emphasze that P s desgned to be adaptve to the avalable channel state nformaton. Here, P remans constant for dfferent channel uses, as the channel statstcs does not change. However, f the channel statstcs vares, P should vary accordngly. 5 Here we assume that J s properly chosen such that J /N 2 s an nteger. M F H Π H vi v 2 P H H H v H P P H H H + σ 2 I H P Π F F H B F where B Π H vi v 2 P H H H v H P P H H H + σ 2 I H P Π. 36 We can express M and B n a block-wse form wth each block of sze J/N-by-J/N. Let M and B be the th dagonal block of M and B, respectvely. Then, we obtan M dag FH B F dag... F H B N F dag, where dag returns a dagonal matrx specfed by the dagonal of the matrx n the parenthess. Recall that both H and P are block dagonal, and so s P H H H v H P P H H H +σ 2 I H P. Thus, from the crteron of Π n Secton V.A, every B s dagonal. Therefore, as J M k, k N J tr{b } 37a J tr{b} 37b {vi J tr v 2 P H H H v H P P H H H } + σ 2 I H P 37c { J tr v I + } σ H P P H H H 2 37d { J/N tr J v I + } σ 2 H PP H H H 37e { [tr N E v I + }] σ 2 HQHH 37f where 37a follows from the fact that, for each F H N N B F dag B n, n I N J J tr{ B }I, n 37b from the crteron of Π, 37c from 36, and 37e by substtutng H dag{h,...,h J/N }, P dag{p,...,p}, and Q PP H. Substtutng 37 nto 5a, we arrve at 35. Now, smlar to Lemma, we have the followng result. Lemma 4: The detector s output b can be expressed as b x + n,wheren s ndependent of x and converges to CN, /φv as J.

11 YUAN et al.: ACHIEVABLE RATES OF MIMO SYSTEMS 783 The above lemma s lterally the same as Lemma, except that φv here s gven by 34. The proof mostly follows that of Lemma. We omt the detals for brevty. Wth Lemma 4, we can stll characterze the behavor of the teratve recever usng the SINR-varance transfer functons, namely, ρ φv and v ψρ, as descrbed n Secton III.E. C. Area Theorem and Precoder Optmzaton Now we are ready to present the followng result. Theorem 3: As J, f the detector and the decoder are matched, an achevable nformaton rate per transmt antenna per channel use of the LP-LMMSE scheme s gven by R [log N E det I + ] σ 2 HQHH. 38 The proof of Theorem 3 s gven n Appendx F. Wth Theorem 3, we can formulate the followng rate-maxmzaton problem: [ maxmze E log det I + ] σ 2 HQHH subject to N tr{q} P. 39a 39b The above optmzaton problem s convex, and thus can be solved numercally usng standard convex programmng tools. Moreover, the explct solutons to 39 n a varety of CSIT settngs have been studed n the lterature. We refer nterested readers to [34], [5], and [5] for more detals. For constraned sgnalng, t s straghtforward to show that Theorem 2 holds lterally for the partal CSIT case, except that φv s replaced by 34. Then, we can formulate an optmzaton problem smlar to 29, whch s solvable usng standard convex programmng. Detals are omtted for smplcty. VI. NUMERICAL RESULTS In ths secton, we provde numercal examples to demonstrate the achevable rates of the proposed scheme. Note that the channel SNR s defned as SNR PN/σ 2. We frst consder the case of full CSIT n a randomly generated 2 2 MIMO 3-tap ISI channel wth the tap coeffcents [ ] j j.648, 4a.3347 j j.242 [ ].582 j j.7565, 4b.4968 j j.595 and [ ].5262 j j c.672 j j.2695 The DFT s appled to convert the above MIMO ISI channel to a set of J by-2 parallel MIMO channels. Note that the effect of cyclc prefx s gnored here. Fg. 4 shows the achevable rates of the proposed LP-LMMSE scheme wth varous power allocaton strateges. Varous power allocaton polces are consdered, ncludng equal power EP allocaton, water-fllng WF, and optmzed power OP allocaton. For EP, W εi where Fg. 4. The achevable rates of the proposed LP-LMMSE scheme and the SVD-based parallel transmsson scheme wth QPSK modulaton n the MIMO-ISI channel n 4. The water-fllng capacty and Gaussan EP capacty are also ncluded for comparson. ε s a scalng factor to meet the power budget; for WF, W s determned by the water-fllng soluton; and for OP, W s obtaned by solvng 29. QPSK modulaton s employed. From Fg. 4, we see that the LP-LMMSE schemes wth OP and WF power allocaton have smlar performance, and both consderably outperform the flat precoder n the low SNR regon. Ths mples that the water-fllng precoder provdes an attractve low-complexty near-optmum opton for dscrete sgnalng. Fg. 4 also ncludes the Gaussan water-fllng capacty and Gaussan EP.e.,..d. capacty as references. We see that the LP-LMMSE schemes wth OP and WP approach the water-fllng capacty n the low SNR regon. Further, we compare the performance of the LP-LMMSE scheme wth that of the parallel transmsson scheme n whch the MIMO channels are converted nto parallel ndependent scalar channels usng SVD wth varous power allocaton strateges. We see that the LP-LMMSE scheme outperforms the parallel transmsson scheme wth MWF consderably n the medum SNR range. Ths ndcates the advantage of lnear precodng over parallel ndependent transmsson for MIMO channels. In Fg. 5, we employ both QPSK modulaton and standard 6-QAM see [57, SCM-] for sgnalng. Fg. 5 demonstrates that the achevable rate of the LP-LMMSE scheme s sgnfcantly ncreased by changng the sgnalng constellaton from QPSK to 6-QAM. We see that the gap between the achevable rate of LP-LMMSE wth 6-QAM and the waterfllng capacty s not sgnfcant for a rate up to 4 bts. For a hgher transmsson rate, a larger sgnalng constellaton s necessary. We now provde code desgn examples to verfy that the theoretcal lmts gven by our analyss are ndeed approachable. We bascally follow the desgn approach descrbed n [45]. Here, we only provde the desgn results. We consder standard 6-QAM sgnalng wth 2-layer SCM. Each SCM

12 784 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 6, NO., NOVEMBER 24 Fg. 5. The achevable rates of the proposed LP-LMMSE scheme n the MIMO-ISI channel n 4. Both QPSK and standard 6-QAM are consdered. The water-fllng capacty s also ncluded for reference. Fg. 7. The BER performance of the LP-LMMSE scheme wth 2-layer SCM over the MIMO ISI channel n 4. Water-fllng precodng and standard 6-QAM constellaton are used. Some smulaton settngs are M N 2, J 256, and K 256. Fg. 6. The SINR-varance transfer functons of the detector and the decoder for the LP-LMMSE scheme wth 2-layer SCM over the MIMO ISI channel n 4. Water-fllng precodng and standard 6-QAM constellaton are used. The transfer functon of the detector s gven by 5 at the channel SNR 6.7 db. layer s QPSK modulated wth Gray mappng. Note: for standard 6-QAM sgnalng, the power rato of the two SCM layers s 4:. Two rregular LDPC codes are desgned based on the curve-matchng prncple. For the SCM layer wth hgher power, the edge degree dstrbutons of the LDPC code are gven by {λ x.3495x +.242x x x x 7 +.x 2,ρ x x 6 }; for the other layer, the edge degree dstrbutons of the LDPC code are {λ 2 x.397x +.854x x +.877x +.394x 33,ρ 2 x x 6 }. The system throughput s 4 bts per channel use. The transfer curves of the detector and the decoder are llustrated n Fg. 6, and the smulated system performance s gven n Fg. 7. From Fg. 7, the desgn threshold s 6.7 db,. db away from the water-fllng capacty,.5 db away from the performance lmt of the LP-LMMSE scheme wth standard 6-QAM. The smulated system performance wth code length 5 at BER 4 s about 7. db, only.3 db away from the desgn threshold. Fg. 8. The achevable rates of the LP-LMMSE scheme n the flat-fadng 2-by-2 MIMO channel wth partal CSIT of θ.5. The water-fllng capacty s also ncluded for reference. It s nterestng to pay specal attenton to pont A n Fg. 6. Ths s the transton pont where, for the two layers of the LDPC codes, the one wth hgher power s nearly fully decoded.e., wth very relable outputs and the one wth lower power starts to be decoded. Now consder the case of partal CSIT. We assume the mean- feedback model [53],.e., each H s a random realzaton of H E[H]+ H where the entres of E[H] are ndependently drawn from CN,θwth θ [, ], and those of H from CN, θ. In smulaton, θ s set to.5. The channel mean E[H] remans constant for each frame, but vares ndependently from frame to frame. Each frame contans 5 ndependently generated H. The achevable rates of the LP-LMMSE scheme averaged over 2 frames are llustrated n Fg. 8. We see that smlar trends as n Fg. 5 have been observed.

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations Applcaton of Nonbnary LDPC Codes for Communcaton over Fadng Channels Usng Hgher Order Modulatons Rong-Hu Peng and Rong-Rong Chen Department of Electrcal and Computer Engneerng Unversty of Utah Ths work

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

VQ widely used in coding speech, image, and video

VQ widely used in coding speech, image, and video at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng

More information

Composite Hypotheses testing

Composite Hypotheses testing Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter

More information

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models

More information

ECE559VV Project Report

ECE559VV Project Report ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate

More information

Lecture 3: Shannon s Theorem

Lecture 3: Shannon s Theorem CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform

More information

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or Sgnal Compresson Sgnal Compresson Entropy Codng Entropy codng s also known as zero-error codng, data compresson or lossless compresson. Entropy codng s wdely used n vrtually all popular nternatonal multmeda

More information

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number

More information

More metrics on cartesian products

More metrics on cartesian products More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of

More information

NUMERICAL DIFFERENTIATION

NUMERICAL DIFFERENTIATION NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the

More information

Limited Dependent Variables

Limited Dependent Variables Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages

More information

Errors for Linear Systems

Errors for Linear Systems Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch

More information

x = x 1 + :::+ x K and the nput covarance matrces are of the form ± = E[x x y ]. 3.2 Dualty Next, we ntroduce the concept of dualty wth the followng t

x = x 1 + :::+ x K and the nput covarance matrces are of the form ± = E[x x y ]. 3.2 Dualty Next, we ntroduce the concept of dualty wth the followng t Sum Power Iteratve Water-fllng for Mult-Antenna Gaussan Broadcast Channels N. Jndal, S. Jafar, S. Vshwanath and A. Goldsmth Dept. of Electrcal Engg. Stanford Unversty, CA, 94305 emal: njndal,syed,srram,andrea@wsl.stanford.edu

More information

The Order Relation and Trace Inequalities for. Hermitian Operators

The Order Relation and Trace Inequalities for. Hermitian Operators Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence

More information

Difference Equations

Difference Equations Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1

More information

Error Probability for M Signals

Error Probability for M Signals Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem. Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from

More information

COS 521: Advanced Algorithms Game Theory and Linear Programming

COS 521: Advanced Algorithms Game Theory and Linear Programming COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton

More information

Module 9. Lecture 6. Duality in Assignment Problems

Module 9. Lecture 6. Duality in Assignment Problems Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept

More information

Structure and Drive Paul A. Jensen Copyright July 20, 2003

Structure and Drive Paul A. Jensen Copyright July 20, 2003 Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.

More information

Pulse Coded Modulation

Pulse Coded Modulation Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal

More information

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder.

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder. PASSBAND DIGITAL MODULATION TECHNIQUES Consder the followng passband dgtal communcaton system model. cos( ω + φ ) c t message source m sgnal encoder s modulator s () t communcaton xt () channel t r a n

More information

Lossy Compression. Compromise accuracy of reconstruction for increased compression.

Lossy Compression. Compromise accuracy of reconstruction for increased compression. Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost

More information

Notes on Frequency Estimation in Data Streams

Notes on Frequency Estimation in Data Streams Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to

More information

MMA and GCMMA two methods for nonlinear optimization

MMA and GCMMA two methods for nonlinear optimization MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons

More information

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1 Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons

More information

Convergence of random processes

Convergence of random processes DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large

More information

Resource Allocation with a Budget Constraint for Computing Independent Tasks in the Cloud

Resource Allocation with a Budget Constraint for Computing Independent Tasks in the Cloud Resource Allocaton wth a Budget Constrant for Computng Independent Tasks n the Cloud Wemng Sh and Bo Hong School of Electrcal and Computer Engneerng Georga Insttute of Technology, USA 2nd IEEE Internatonal

More information

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4) I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes

More information

Problem Set 9 Solutions

Problem Set 9 Solutions Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem

More information

Hidden Markov Models & The Multivariate Gaussian (10/26/04)

Hidden Markov Models & The Multivariate Gaussian (10/26/04) CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models

More information

Global Sensitivity. Tuesday 20 th February, 2018

Global Sensitivity. Tuesday 20 th February, 2018 Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values

More information

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

Maximizing the number of nonnegative subsets

Maximizing the number of nonnegative subsets Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum

More information

Digital Modems. Lecture 2

Digital Modems. Lecture 2 Dgtal Modems Lecture Revew We have shown that both Bayes and eyman/pearson crtera are based on the Lkelhood Rato Test (LRT) Λ ( r ) < > η Λ r s called observaton transformaton or suffcent statstc The crtera

More information

Hidden Markov Models

Hidden Markov Models Hdden Markov Models Namrata Vaswan, Iowa State Unversty Aprl 24, 204 Hdden Markov Model Defntons and Examples Defntons:. A hdden Markov model (HMM) refers to a set of hdden states X 0, X,..., X t,...,

More information

MIMO Systems and Channel Capacity

MIMO Systems and Channel Capacity MIMO Systems and Channel Capacty Consder a MIMO system wth m Tx and n Rx antennas. x y = Hx ξ Tx H Rx The power constrant: the total Tx power s x = P t. Component-wse representaton of the system model,

More information

Lecture 12: Discrete Laplacian

Lecture 12: Discrete Laplacian Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly

More information

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there

More information

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011 Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected

More information

On an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems. Vahid Tadayon 1

On an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems. Vahid Tadayon 1 On an Extenson of Stochastc Approxmaton EM Algorthm for Incomplete Data Problems Vahd Tadayon Abstract: The Stochastc Approxmaton EM (SAEM algorthm, a varant stochastc approxmaton of EM, s a versatle tool

More information

Feature Selection: Part 1

Feature Selection: Part 1 CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?

More information

Games of Threats. Elon Kohlberg Abraham Neyman. Working Paper

Games of Threats. Elon Kohlberg Abraham Neyman. Working Paper Games of Threats Elon Kohlberg Abraham Neyman Workng Paper 18-023 Games of Threats Elon Kohlberg Harvard Busness School Abraham Neyman The Hebrew Unversty of Jerusalem Workng Paper 18-023 Copyrght 2017

More information

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also

More information

Appendix B: Resampling Algorithms

Appendix B: Resampling Algorithms 407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles

More information

Comparison of Regression Lines

Comparison of Regression Lines STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence

More information

Conjugacy and the Exponential Family

Conjugacy and the Exponential Family CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the

More information

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U) Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of

More information

U.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016

U.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016 U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and

More information

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve

More information

Estimation: Part 2. Chapter GREG estimation

Estimation: Part 2. Chapter GREG estimation Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the

More information

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence

More information

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003 Tornado and Luby Transform Codes Ashsh Khst 6.454 Presentaton October 22, 2003 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel

More information

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS Avalable onlne at http://sck.org J. Math. Comput. Sc. 3 (3), No., 6-3 ISSN: 97-537 COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

More information

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010 Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton

More information

CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE

CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng

More information

Grover s Algorithm + Quantum Zeno Effect + Vaidman

Grover s Algorithm + Quantum Zeno Effect + Vaidman Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the

More information

Supplementary Notes for Chapter 9 Mixture Thermodynamics

Supplementary Notes for Chapter 9 Mixture Thermodynamics Supplementary Notes for Chapter 9 Mxture Thermodynamcs Key ponts Nne major topcs of Chapter 9 are revewed below: 1. Notaton and operatonal equatons for mxtures 2. PVTN EOSs for mxtures 3. General effects

More information

Inner Product. Euclidean Space. Orthonormal Basis. Orthogonal

Inner Product. Euclidean Space. Orthonormal Basis. Orthogonal Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,

More information

Lecture 21: Numerical methods for pricing American type derivatives

Lecture 21: Numerical methods for pricing American type derivatives Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)

More information

Power Allocation/Beamforming for DF MIMO Two-Way Relaying: Relay and Network Optimization

Power Allocation/Beamforming for DF MIMO Two-Way Relaying: Relay and Network Optimization Power Allocaton/Beamformng for DF MIMO Two-Way Relayng: Relay and Network Optmzaton Je Gao, Janshu Zhang, Sergy A. Vorobyov, Ha Jang, and Martn Haardt Dept. of Electrcal & Computer Engneerng, Unversty

More information

(1 ) (1 ) 0 (1 ) (1 ) 0

(1 ) (1 ) 0 (1 ) (1 ) 0 Appendx A Appendx A contans proofs for resubmsson "Contractng Informaton Securty n the Presence of Double oral Hazard" Proof of Lemma 1: Assume that, to the contrary, BS efforts are achevable under a blateral

More information

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s

More information

The Geometry of Logit and Probit

The Geometry of Logit and Probit The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.

More information

APPENDIX A Some Linear Algebra

APPENDIX A Some Linear Algebra APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,

More information

NP-Completeness : Proofs

NP-Completeness : Proofs NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem

More information

Kernel Methods and SVMs Extension

Kernel Methods and SVMs Extension Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general

More information

Time-Varying Systems and Computations Lecture 6

Time-Varying Systems and Computations Lecture 6 Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 12 10/21/2013. Martingale Concentration Inequalities and Applications

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 12 10/21/2013. Martingale Concentration Inequalities and Applications MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 1 10/1/013 Martngale Concentraton Inequaltes and Applcatons Content. 1. Exponental concentraton for martngales wth bounded ncrements.

More information

An (almost) unbiased estimator for the S-Gini index

An (almost) unbiased estimator for the S-Gini index An (almost unbased estmator for the S-Gn ndex Thomas Demuynck February 25, 2009 Abstract Ths note provdes an unbased estmator for the absolute S-Gn and an almost unbased estmator for the relatve S-Gn for

More information

Numerical Heat and Mass Transfer

Numerical Heat and Mass Transfer Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017 U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that

More information

Perfect Competition and the Nash Bargaining Solution

Perfect Competition and the Nash Bargaining Solution Perfect Competton and the Nash Barganng Soluton Renhard John Department of Economcs Unversty of Bonn Adenauerallee 24-42 53113 Bonn, Germany emal: rohn@un-bonn.de May 2005 Abstract For a lnear exchange

More information

Power law and dimension of the maximum value for belief distribution with the max Deng entropy

Power law and dimension of the maximum value for belief distribution with the max Deng entropy Power law and dmenson of the maxmum value for belef dstrbuton wth the max Deng entropy Bngy Kang a, a College of Informaton Engneerng, Northwest A&F Unversty, Yanglng, Shaanx, 712100, Chna. Abstract Deng

More information

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have

More information

Low Complexity Soft-Input Soft-Output Hamming Decoder

Low Complexity Soft-Input Soft-Output Hamming Decoder Low Complexty Soft-Input Soft-Output Hammng Der Benjamn Müller, Martn Holters, Udo Zölzer Helmut Schmdt Unversty Unversty of the Federal Armed Forces Department of Sgnal Processng and Communcatons Holstenhofweg

More information

Stat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors

Stat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors Stat60: Bayesan Modelng and Inference Lecture Date: February, 00 Reference Prors Lecturer: Mchael I. Jordan Scrbe: Steven Troxler and Wayne Lee In ths lecture, we assume that θ R; n hgher-dmensons, reference

More information

Week 5: Neural Networks

Week 5: Neural Networks Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple

More information

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could

More information

Asymptotic Analysis of Dual-Diagonal LMMSE Channel Estimation in OFDM Systems

Asymptotic Analysis of Dual-Diagonal LMMSE Channel Estimation in OFDM Systems Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the IEEE Globecom proceedngs. Asymptotc Analyss of ual-agonal LMMSE Channel Estmaton

More information

EEE 241: Linear Systems

EEE 241: Linear Systems EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they

More information

Inexact Newton Methods for Inverse Eigenvalue Problems

Inexact Newton Methods for Inverse Eigenvalue Problems Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.

More information

Supplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso

Supplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed

More information

1 Convex Optimization

1 Convex Optimization Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,

More information

Linear dispersion code with an orthogonal row structure for simplifying sphere decoding

Linear dispersion code with an orthogonal row structure for simplifying sphere decoding tle Lnear dsperson code wth an orthogonal row structure for smplfyng sphere decodng Author(s) Da XG; Cheung SW; Yuk I Ctaton he 0th IEEE Internatonal Symposum On Personal Indoor and Moble Rado Communcatons

More information

A new construction of 3-separable matrices via an improved decoding of Macula s construction

A new construction of 3-separable matrices via an improved decoding of Macula s construction Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula

More information

Appendix for Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Appendix for Causal Interaction in Factorial Experiments: Application to Conjoint Analysis A Appendx for Causal Interacton n Factoral Experments: Applcaton to Conjont Analyss Mathematcal Appendx: Proofs of Theorems A. Lemmas Below, we descrbe all the lemmas, whch are used to prove the man theorems

More information

The Concept of Beamforming

The Concept of Beamforming ELG513 Smart Antennas S.Loyka he Concept of Beamformng Generc representaton of the array output sgnal, 1 where w y N 1 * = 1 = w x = w x (4.1) complex weghts, control the array pattern; y and x - narrowband

More information

5 The Rational Canonical Form

5 The Rational Canonical Form 5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces

More information

Chapter 13: Multiple Regression

Chapter 13: Multiple Regression Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to

More information

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,

More information

Lecture 12: Classification

Lecture 12: Classification Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna

More information

Chapter 11: Simple Linear Regression and Correlation

Chapter 11: Simple Linear Regression and Correlation Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests

More information

Lecture 5 Decoding Binary BCH Codes

Lecture 5 Decoding Binary BCH Codes Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture

More information