Linear Algebra Issues in Wireless Communications

Size: px
Start display at page:

Download "Linear Algebra Issues in Wireless Communications"

Transcription

1 Rome-Moscow school of Matrix Methods ad Applied Liear Algebra August 0 September 18, 016 Liear Algebra Issues i Wireless Commuicatios Russia Research Ceter [vladimir.lyashev@huawei.com]

2 About me ead of Iterferece Cacellatio Lab. at uawei Techologies, Russia Research Ceter, Wireless Research Divisio. Research Fellow at Souther Federal Uiversity, the Istitute of Sigal Processig ad Cotrol Systems Iteret of Iformatio Iteret of Thigs Iteret of Cotrol Page

3 Aim & Short Outlie Brief itroductio i the state-of-the-art problems of matrix calculatios i wireless commuicatio should allow participats to uderstad importace of mathematics ad mai problems here, which become part of our life. The itroductio is preseted by 3 parts: Itroductio i wireless commuicatio ad its relatio to liear algebra problems System model of wireless commuicatio Capacity of the system ad liear space extesio Defiitio of system matrix ad its properties discussio Outloo of matrix methods that are playig importat role i telecom. Matrix methods ad requiremets to them from the idustry Not well-posed problems i the sese of adamard (ill-posed problems) QR, LU, iterative methods ad requiremets to them from wireless systems Importat role of SVD i ultra-high rate commuicatios Belief propagatio approach i detectio problems Implemetatio issues Robustess & accuracy The simplest oe does t mea the fastest oe // free discussio Page 3

4 A istory [ 1948 ] Groudbreaig article by Claude E. Shao A mathematical theory of commuicatio itroducig the defiitio of capacity i commuicatio systems: The chael capacity is a measuremet of the maximum amout of iformatio that ca be trasmitted over a chael ad received with a egligible probability of error at the receiver. C B f log 1 SNR Iformatio is the resolutio of ucertaity. - Claude Elwood Shao Father of Iformatio Theory Page 4

5 Page 5 PDF: additive oise x y X Y P X Y P X P X Y Y P Y P Y ) ( )log ( ) ( ) ( ) ( )log ( ) ( X1 = +1; X = -1 Y1 = +1; Y = -1 P(X1) = 0.5; P(X) = 0.5 P(Y1 X1) = ; P(Y1 X) = ) P(Y X1) = ; P(Y X) = ) Error probability: 0.5%

6 Page 6 Sigal-to-Noise Ratio ; ) )( ( 0 0 x y P x x xx x x yy P x y 1 log 1 log log 1 x x y x y P B N P B N P B N P P W 0 pass bad 1 0 B W B B N B f

7 Detectio problem Page 7 X 1,,,3 Search costrait: Search strategy: X -? aswer ca be YES/NO bisectio method or iterval halvig or biary search or dichotomy method. As me Q#1 Q# Q#3 Q#4 Q#5

8 Resume 5 aswers YES/NO 5 bits of iformatio I P logm log1 artley's law: quatity of iformatio M, which is ecessary for detectio the specific value, is the base--logarithm of the umber of distict messages M that could be set. Page 8

9 Ope Problems i Wireless Commuicatio MULTI-USERS SYSTEMS (CDMA, SCMA, MU-MIMO, etc.) MULTI-ANTENNA SYSTEMS (massive-mimo, cell splittig/atea selectio, precodig/beamformig) Page 9

10 MIMO System Evolutio MORE spectrum efficiecy MORE atea elemets Page 10

11 Multiple Iput Multiple Output Commuicatios y x Page 11

12 Capacity of the system ad liear space extesio Page 1 P TX power of sigle trasmitter; N TX umber of trasmissio ateas; Q covariace matrix of trasmitted sigal. I curret product it is required matrix operatios for N RB (6 110) matrices with size 4x4 18x18 per 1ms or less I future N RB ca be exteded to 500, ad time scale ca be reduced 10 times!!!

13 What is chael matrix? Page 13

14 State-of-the-Art Problem To solve the matrix equatio as detectio problem Maximize Sigal-to-Noise Ratio for better performace User groupig for better throughput Page 14

15 Matrix applicatios i MIMO Eigevalue problems for beamformig ad iterferece cacellatio Reduced-ra sigal processig for regularizatio of iverse problems for atea combiig algorithms ad distributed MIMO/schedulig systems Low complexity algorithms for adaptive spatial filterig (dirty paper codig, STBC) System idetificatio (pilot cotamiatio problem i massive MIMO, type of traffic) Tesor Algebra for joit sigal processig (chael estimatio, multiuser detectio) Matrix volume maximizatio for MU-MIMO ad o-orthogoal access Polyomial series ad approximatio theory i itermodulatio compoets suppressio Page 15

16 Matrix Algebra for MIMO Problems Matrix Algebra is coveiet istrumet for problem formulatios i iformatio ad commuicatios systems ad to apply the cocepts ad algorithms of umerical liear algebra for optimal desig ad operatio of these systems Importat math. areas for MIMO Systems QR factorizatio ad least squares problems Coditioig ad stability issues i umerical liear algebra Numerical methods for solvig systems of liear equatios Eigevalue problems ad the computatio of the sigular value decompositio Aroldi algorithm Laczos iteratios CUR matrix decompositio Gradiets based approaches for o-liear problems Page 16

17 Vector Precodig The set of equatios that describe the MMSE solutio for vector precodig are: y Set x x ps p arg mi pˆ that R Thus, y U ΛV uu 1 y pˆ s Usig SVD, the matrix ca be diagoalized by orthogoal matrices U ad V x by trasmittig x xv U Λ Istead of x ad pre-multiply the receive sigal by vector U, the trasformed received sigal vector becomes: U Λ V x U Λx ~ y U y U ~ I TDD systems, the chael is the same o trasmitter ad receiver, but it is chagig i time ad robust precodig of the chael is challegeable problem i wireless comm. xv V Page 17

18 SVD is very importat operatio!!! Page 18

19 Eigevector calculatio algorithm Page 19

20 Tihoov Regularizatio i Iverse Problem Each least squares problem has to be regularized. I the liear case, we wat to solve miimizatio problem after regularizatio the solutio is x Page 0 Ax b Ax b ε Γx 1 A A Γ Γ A 1 A A I A b mi b ow to defie Tihoov matrix Г? Ax b ε x 5

21 Resume Typical size of the matrix is less elemets. For such matrix we eed fast algorithms for eigevector decompositio; matrix iversio (curret baselie is classical Cholesy decompositio algorithm) The questios are: 1.Ca we defie some less complexity algorithm for matrix iversio ad eigevector calculatio tha provided baselie algorithms?.do some fast algorithms (approaches) exist i moder liear algebra to compute such specific matrices? Page 1

22 omewor 1. Geerate Wishart-matrix A with followig parameters of distributio: Sigma = df = 8; Sigma = 0.1*oes(4) + 0.9*eye(4); A = wishrd(sigma,df)/df df = 8. Set up 000 samples of equatio: A ε x b ε, where x [ 1, 1]; (0, ), p is subject to A xˆ b p s, s. Page

23 omewor 3. Solve oisy equatio sample by sample ad defie probability of right solutio averaged over all samples. 1 p 1,1,1, // default vector T 1 4 p 1 (!) (1) 4. Repeat item #3 for p u - eigevector of matrix A, correspoded to the largest sigular value. 5. ow stochastic iformatio about additive oise ε ca be utilized for miimizatio of error probability?.. Page 3

24 omewor I. Geerate 000 samples of metrix A ad eep i memory for all umerical experimets. II. Set mappig vector p (two ways). III. Map oe-bit symbol (-1/+1) from 1x1 to 4x1: x = ps. IV. Compute b = Ax. V. Add gaussia oise (sigma is variatio parameter for aalysis): b = b +. VI. Solve oise equatio: Ax = b, that defie x as estimatio value. VII. Fid a way to defie s by ow p ad estimated x. VIII. Chec: how may s = s?... P error = 1 - <right s > / <umber of samples> IX. P error ca be defied as fuctio of deviatio of oise (sigma). Page 4

OFDM Precoder for Minimizing BER Upper Bound of MLD under Imperfect CSI

OFDM Precoder for Minimizing BER Upper Bound of MLD under Imperfect CSI MIMO-OFDM OFDM Precoder for Miimizig BER Upper Boud of MLD uder Imperfect CSI MCRG Joit Semiar Jue the th 008 Previously preseted at ICC 008 Beijig o May the st 008 Boosar Pitakdumrogkija Kazuhiko Fukawa

More information

TMA4205 Numerical Linear Algebra. The Poisson problem in R 2 : diagonalization methods

TMA4205 Numerical Linear Algebra. The Poisson problem in R 2 : diagonalization methods TMA4205 Numerical Liear Algebra The Poisso problem i R 2 : diagoalizatio methods September 3, 2007 c Eiar M Røquist Departmet of Mathematical Scieces NTNU, N-749 Trodheim, Norway All rights reserved A

More information

Session 5. (1) Principal component analysis and Karhunen-Loève transformation

Session 5. (1) Principal component analysis and Karhunen-Loève transformation 200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Signal Processing in Mechatronics

Signal Processing in Mechatronics Sigal Processig i Mechatroics Zhu K.P. AIS, UM. Lecture, Brief itroductio to Sigals ad Systems, Review of Liear Algebra ad Sigal Processig Related Mathematics . Brief Itroductio to Sigals What is sigal

More information

THE KALMAN FILTER RAUL ROJAS

THE KALMAN FILTER RAUL ROJAS THE KALMAN FILTER RAUL ROJAS Abstract. This paper provides a getle itroductio to the Kalma filter, a umerical method that ca be used for sesor fusio or for calculatio of trajectories. First, we cosider

More information

ELEC1200: A System View of Communications: from Signals to Packets Lecture 3

ELEC1200: A System View of Communications: from Signals to Packets Lecture 3 ELEC2: A System View of Commuicatios: from Sigals to Packets Lecture 3 Commuicatio chaels Discrete time Chael Modelig the chael Liear Time Ivariat Systems Step Respose Respose to sigle bit Respose to geeral

More information

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial

More information

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH : Theoretical Foudatios of Wireless Commuicatios 1 Thursday, May 19, 2016 12:30-15:30, Coferece Room SIP 1 Textbook: D. Tse ad P. Viswaath, Fudametals of Wireless Commuicatio 1 / 1 Overview Lecture 6:

More information

Dept. of Biomed. Eng. BME801: Inverse Problems in Bioengineering Kyung Hee Univ.

Dept. of Biomed. Eng. BME801: Inverse Problems in Bioengineering Kyung Hee Univ. Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Adaptive Filters - Statistical digital sigal processig: i may problems of iterest, the sigals exhibit some iheret variability plus additive

More information

The DOA Estimation of Multiple Signals based on Weighting MUSIC Algorithm

The DOA Estimation of Multiple Signals based on Weighting MUSIC Algorithm , pp.10-106 http://dx.doi.org/10.1457/astl.016.137.19 The DOA Estimatio of ultiple Sigals based o Weightig USIC Algorithm Chagga Shu a, Yumi Liu State Key Laboratory of IPOC, Beijig Uiversity of Posts

More information

PC5215 Numerical Recipes with Applications - Review Problems

PC5215 Numerical Recipes with Applications - Review Problems PC55 Numerical Recipes with Applicatios - Review Problems Give the IEEE 754 sigle precisio bit patter (biary or he format) of the followig umbers: 0 0 05 00 0 00 Note that it has 8 bits for the epoet,

More information

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition 6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.

More information

State Space Representation

State Space Representation Optimal Cotrol, Guidace ad Estimatio Lecture 2 Overview of SS Approach ad Matrix heory Prof. Radhakat Padhi Dept. of Aerospace Egieerig Idia Istitute of Sciece - Bagalore State Space Represetatio Prof.

More information

Semi-Blind Channel Estimation for LTE DownLink

Semi-Blind Channel Estimation for LTE DownLink TECHNICAL UNIVERSITY OF DENMARK Semi-Blid Chael Estimatio for LTE DowLik by Nicolò Michelusi A thesis submitted i partial fulfillmet for the degree of Master of Sciece MSc i Telecommuicatio Egieerig Supervisors:

More information

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220 ECE 564/645 - Digital Commuicatio Systems (Sprig 014) Fial Exam Friday, May d, 8:00-10:00am, Marsto 0 Overview The exam cosists of four (or five) problems for 100 (or 10) poits. The poits for each part

More information

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8 Digital Commuicatio Chael codig, liear block codes, Hammig ad cyclic codes Lecture - 8 Ir. Muhamad Asial, MSc., PhD Ceter for Iformatio ad Commuicatio Egieerig Research (CICER) Electrical Egieerig Departmet

More information

The Method of Least Squares. To understand least squares fitting of data.

The Method of Least Squares. To understand least squares fitting of data. The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve

More information

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that

More information

A new scalable decoder for linear space-time block codes with intersymbol interference

A new scalable decoder for linear space-time block codes with intersymbol interference A ew scalale der for liear space-time lock s with itersymol iterferece Marc Kuh, Armi Wittee Uiversity of Saarlad, Istitute of Digital Commuicatios, D 6604 Saarruecke, Germay Email: marc.kuh@lnt.ui-saarlad.de,

More information

Extended Weighted Linear Prediction Using the Autocorrelation Snapshot

Extended Weighted Linear Prediction Using the Autocorrelation Snapshot Exteded Weighted Liear Predictio Usig the Autocorrelatio Sapshot - A Robust Speech Aalysis Method ad its Applicatio to Recogitio of Vocal Emotios Joui Pohjalaie ad Paavo Alku Departmet of Sigal Processig

More information

Chapter 12 EM algorithms The Expectation-Maximization (EM) algorithm is a maximum likelihood method for models that have hidden variables eg. Gaussian

Chapter 12 EM algorithms The Expectation-Maximization (EM) algorithm is a maximum likelihood method for models that have hidden variables eg. Gaussian Chapter 2 EM algorithms The Expectatio-Maximizatio (EM) algorithm is a maximum likelihood method for models that have hidde variables eg. Gaussia Mixture Models (GMMs), Liear Dyamic Systems (LDSs) ad Hidde

More information

Semi-Blind Channel Estimation for LTE DownLink

Semi-Blind Channel Estimation for LTE DownLink TECHNICAL UNIVERSITY OF DENMARK Semi-Blid Chael Estimatio for LTE DowLik by Nicolò Michelusi A thesis submitted i partial fulfillmet for the degree of Master of Sciece (MSc) i Telecommuicatio Egieerig

More information

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio

More information

Information Theory and Coding

Information Theory and Coding Sol. Iformatio Theory ad Codig. The capacity of a bad-limited additive white Gaussia (AWGN) chael is give by C = Wlog 2 ( + σ 2 W ) bits per secod(bps), where W is the chael badwidth, is the average power

More information

Polynomial Multiplication and Fast Fourier Transform

Polynomial Multiplication and Fast Fourier Transform Polyomial Multiplicatio ad Fast Fourier Trasform Com S 477/577 Notes Ya-Bi Jia Sep 19, 2017 I this lecture we will describe the famous algorithm of fast Fourier trasform FFT, which has revolutioized digital

More information

Matrix inversion algorithm: applications in high speed MIMO LTE receiver

Matrix inversion algorithm: applications in high speed MIMO LTE receiver merica Joural of Networks ad Commuicatios 0; () : -6 Published olie December 30, 0 (http://www.sciecepublishiggroup.com/j/ajc) doi: 0.648/j.ajc.000. Matrix iversio algorithm: applicatios i high speed MIMO

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Iterative method for computing a Schur form of symplectic matrix

Iterative method for computing a Schur form of symplectic matrix Aals of the Uiversity of Craiova, Mathematics ad Computer Sciece Series Volume 421, 2015, Pages 158 166 ISSN: 1223-6934 Iterative method for computig a Schur form of symplectic matrix A Mesbahi, AH Betbib,

More information

Internal Information Representation and Processing

Internal Information Representation and Processing Iteral Iformatio Represetatio ad Processig CSCE 16 - Fudametals of Computer Sciece Dr. Awad Khalil Computer Sciece & Egieerig Departmet The America Uiversity i Cairo Decimal Number System We are used to

More information

Pipelined and Parallel Recursive and Adaptive Filters

Pipelined and Parallel Recursive and Adaptive Filters VLSI Digital Sigal Processig Systems Pipelied ad Parallel Recursive ad Adaptive Filters La-Da Va 范倫達, Ph. D. Departmet of Computer Sciece Natioal Chiao ug Uiversity aiwa, R.O.C. Fall, 05 ldva@cs.ctu.edu.tw

More information

Entropies & Information Theory

Entropies & Information Theory Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory

More information

Lainiotis filter implementation. via Chandrasekhar type algorithm

Lainiotis filter implementation. via Chandrasekhar type algorithm Joural of Computatios & Modellig, vol.1, o.1, 2011, 115-130 ISSN: 1792-7625 prit, 1792-8850 olie Iteratioal Scietific Press, 2011 Laiiotis filter implemetatio via Chadrasehar type algorithm Nicholas Assimais

More information

Filter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and

Filter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and Filter bas Separately, the lowpass ad highpass filters are ot ivertible T removes the highest frequecy / ad removes the lowest frequecy Together these filters separate the sigal ito low-frequecy ad high-frequecy

More information

Reliability and Queueing

Reliability and Queueing Copyright 999 Uiversity of Califoria Reliability ad Queueig by David G. Messerschmitt Supplemetary sectio for Uderstadig Networked Applicatios: A First Course, Morga Kaufma, 999. Copyright otice: Permissio

More information

Addition: Property Name Property Description Examples. a+b = b+a. a+(b+c) = (a+b)+c

Addition: Property Name Property Description Examples. a+b = b+a. a+(b+c) = (a+b)+c Notes for March 31 Fields: A field is a set of umbers with two (biary) operatios (usually called additio [+] ad multiplicatio [ ]) such that the followig properties hold: Additio: Name Descriptio Commutativity

More information

Information Hiding Problems: Hiding Capacity and Key Design

Information Hiding Problems: Hiding Capacity and Key Design Iformatio Hidig Problems: Hidig Capacity ad Key Desig Joseph. O Sulliva Electroic Systems ad Sigals Research Laboratory Departmet of Electrical Egieerig Washigto Uiversity i St. Louis Iformatio Hidig Problems

More information

A new iterative algorithm for reconstructing a signal from its dyadic wavelet transform modulus maxima

A new iterative algorithm for reconstructing a signal from its dyadic wavelet transform modulus maxima ol 46 No 6 SCIENCE IN CHINA (Series F) December 3 A ew iterative algorithm for recostructig a sigal from its dyadic wavelet trasform modulus maxima ZHANG Zhuosheg ( u ), LIU Guizhog ( q) & LIU Feg ( )

More information

Chandrasekhar Type Algorithms. for the Riccati Equation of Lainiotis Filter

Chandrasekhar Type Algorithms. for the Riccati Equation of Lainiotis Filter Cotemporary Egieerig Scieces, Vol. 3, 00, o. 4, 9-00 Chadrasekhar ype Algorithms for the Riccati Equatio of Laiiotis Filter Nicholas Assimakis Departmet of Electroics echological Educatioal Istitute of

More information

The picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled

The picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled 1 Lecture : Area Area ad distace traveled Approximatig area by rectagles Summatio The area uder a parabola 1.1 Area ad distace Suppose we have the followig iformatio about the velocity of a particle, how

More information

Quantum Signal Processing and Non-Orthogonal State Detection

Quantum Signal Processing and Non-Orthogonal State Detection Sesors & Trasducers Vol. 60 Issue December 03 pp. 54-546 Sesors & Trasducers 03 by IFSA http://www.sesorsportal.com Quatum Sigal Processig ad o-orthogoal State Detectio Webi YU Baoyu ZHEG Shegmei ZHAO

More information

ON THE CAPACITY OF THE MIMO CHANNEL - A TUTORIAL INTRODUCTION - Bengt Holter

ON THE CAPACITY OF THE MIMO CHANNEL - A TUTORIAL INTRODUCTION - Bengt Holter ON HE CAPACIY OF HE MIMO CHANNEL - A UORIAL INRODUCION - Begt Holter Norwegia Uiversity of Sciece ad echology Departmet of elecommuicatios O.S.Bragstads plass B, N-7491 rodheim, Norway bholter@tele.tu.o

More information

Applications in Linear Algebra and Uses of Technology

Applications in Linear Algebra and Uses of Technology 1 TI-89: Let A 1 4 5 6 7 8 10 Applicatios i Liear Algebra ad Uses of Techology,adB 4 1 1 4 type i: [1,,;4,5,6;7,8,10] press: STO type i: A type i: [4,-1;-1,4] press: STO (1) Row Echelo Form: MATH/matrix

More information

Machine Learning Regression I Hamid R. Rabiee [Slides are based on Bishop Book] Spring

Machine Learning Regression I Hamid R. Rabiee [Slides are based on Bishop Book] Spring Machie Learig Regressio I Hamid R. Rabiee [Slides are based o Bishop Book] Sprig 015 http://ce.sharif.edu/courses/93-94//ce717-1 Liear Regressio Liear regressio: ivolves a respose variable ad a sigle predictor

More information

Information-based Feature Selection

Information-based Feature Selection Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with

More information

*X203/701* X203/701. APPLIED MATHEMATICS ADVANCED HIGHER Numerical Analysis. Read carefully

*X203/701* X203/701. APPLIED MATHEMATICS ADVANCED HIGHER Numerical Analysis. Read carefully X0/70 NATIONAL QUALIFICATIONS 006 MONDAY, MAY.00 PM.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.

More information

Spectral Observer with Reduced Information Demand

Spectral Observer with Reduced Information Demand Spectral Observer with Reduced Iformatio Demad Gy. Orosz, L. Sujbert, G. Péceli Departmet of Measuremet ad Iformatio Systems, Budapest Uiversity of Techology ad Ecoomics Magyar tudósok krt. 2., -52 Budapest,

More information

Where do eigenvalues/eigenvectors/eigenfunctions come from, and why are they important anyway?

Where do eigenvalues/eigenvectors/eigenfunctions come from, and why are they important anyway? Where do eigevalues/eigevectors/eigeuctios come rom, ad why are they importat ayway? I. Bacgroud (rom Ordiary Dieretial Equatios} Cosider the simplest example o a harmoic oscillator (thi o a vibratig strig)

More information

1. Linearization of a nonlinear system given in the form of a system of ordinary differential equations

1. Linearization of a nonlinear system given in the form of a system of ordinary differential equations . Liearizatio of a oliear system give i the form of a system of ordiary differetial equatios We ow show how to determie a liear model which approximates the behavior of a time-ivariat oliear system i a

More information

Direction of Arrival Estimation Method in Underdetermined Condition Zhang Youzhi a, Li Weibo b, Wang Hanli c

Direction of Arrival Estimation Method in Underdetermined Condition Zhang Youzhi a, Li Weibo b, Wang Hanli c 4th Iteratioal Coferece o Advaced Materials ad Iformatio Techology Processig (AMITP 06) Directio of Arrival Estimatio Method i Uderdetermied Coditio Zhag Youzhi a, Li eibo b, ag Hali c Naval Aeroautical

More information

Outline. Linear regression. Regularization functions. Polynomial curve fitting. Stochastic gradient descent for regression. MLE for regression

Outline. Linear regression. Regularization functions. Polynomial curve fitting. Stochastic gradient descent for regression. MLE for regression REGRESSION 1 Outlie Liear regressio Regularizatio fuctios Polyomial curve fittig Stochastic gradiet descet for regressio MLE for regressio Step-wise forward regressio Regressio methods Statistical techiques

More information

Introduction to Optimization Techniques. How to Solve Equations

Introduction to Optimization Techniques. How to Solve Equations Itroductio to Optimizatio Techiques How to Solve Equatios Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually

More information

Chapter 4 : Laplace Transform

Chapter 4 : Laplace Transform 4. Itroductio Laplace trasform is a alterative to solve the differetial equatio by the complex frequecy domai ( s = σ + jω), istead of the usual time domai. The DE ca be easily trasformed ito a algebraic

More information

Overview of Gaussian MIMO (Vector) BC

Overview of Gaussian MIMO (Vector) BC Overview of Gaussia MIMO (Vector) BC Gwamo Ku Adaptive Sigal Processig ad Iformatio Theory Research Group Nov. 30, 2012 Outlie / Capacity Regio of Gaussia MIMO BC System Structure Kow Capacity Regios -

More information

Factor Analysis. Lecture 10: Factor Analysis and Principal Component Analysis. Sam Roweis

Factor Analysis. Lecture 10: Factor Analysis and Principal Component Analysis. Sam Roweis Lecture 10: Factor Aalysis ad Pricipal Compoet Aalysis Sam Roweis February 9, 2004 Whe we assume that the subspace is liear ad that the uderlyig latet variable has a Gaussia distributio we get a model

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

Last time: Moments of the Poisson distribution from its generating function. Example: Using telescope to measure intensity of an object

Last time: Moments of the Poisson distribution from its generating function. Example: Using telescope to measure intensity of an object 6.3 Stochastic Estimatio ad Cotrol, Fall 004 Lecture 7 Last time: Momets of the Poisso distributio from its geeratig fuctio. Gs () e dg µ e ds dg µ ( s) µ ( s) µ ( s) µ e ds dg X µ ds X s dg dg + ds ds

More information

Distributional Similarity Models (cont.)

Distributional Similarity Models (cont.) Distributioal Similarity Models (cot.) Regia Barzilay EECS Departmet MIT October 19, 2004 Sematic Similarity Vector Space Model Similarity Measures cosie Euclidea distace... Clusterig k-meas hierarchical

More information

For a 3 3 diagonal matrix we find. Thus e 1 is a eigenvector corresponding to eigenvalue λ = a 11. Thus matrix A has eigenvalues 2 and 3.

For a 3 3 diagonal matrix we find. Thus e 1 is a eigenvector corresponding to eigenvalue λ = a 11. Thus matrix A has eigenvalues 2 and 3. Closed Leotief Model Chapter 6 Eigevalues I a closed Leotief iput-output-model cosumptio ad productio coicide, i.e. V x = x = x Is this possible for the give techology matrix V? This is a special case

More information

BER results for a narrowband multiuser receiver based on successive subtraction for M-PSK modulated signals

BER results for a narrowband multiuser receiver based on successive subtraction for M-PSK modulated signals results for a arrowbad multiuser receiver based o successive subtractio for M-PSK modulated sigals Gerard J.M. Jasse Telecomm. ad Traffic-Cotrol Systems Group Dept. of Iformatio Techology ad Systems Delft

More information

Introduction to Signals and Systems, Part V: Lecture Summary

Introduction to Signals and Systems, Part V: Lecture Summary EEL33: Discrete-Time Sigals ad Systems Itroductio to Sigals ad Systems, Part V: Lecture Summary Itroductio to Sigals ad Systems, Part V: Lecture Summary So far we have oly looked at examples of o-recursive

More information

Estimation of Backward Perturbation Bounds For Linear Least Squares Problem

Estimation of Backward Perturbation Bounds For Linear Least Squares Problem dvaced Sciece ad Techology Letters Vol.53 (ITS 4), pp.47-476 http://dx.doi.org/.457/astl.4.53.96 Estimatio of Bacward Perturbatio Bouds For Liear Least Squares Problem Xixiu Li School of Natural Scieces,

More information

Variance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom

Variance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom Variace of Discrete Radom Variables Class 5, 18.05 Jeremy Orloff ad Joatha Bloom 1 Learig Goals 1. Be able to compute the variace ad stadard deviatio of a radom variable.. Uderstad that stadard deviatio

More information

Spectral Analysis. This week in lab. Next classes: 3/26 and 3/28. Your next experiment Homework is to prepare

Spectral Analysis. This week in lab. Next classes: 3/26 and 3/28. Your next experiment Homework is to prepare Spectral Aalysis This week i lab Your ext experimet Homework is to prepare Next classes: 3/26 ad 3/28 Aero Testig, Fracture Toughess Testig Read the Experimets 5 ad 7 sectios of the course maual Spectral

More information

Machine Learning for Data Science (CS 4786)

Machine Learning for Data Science (CS 4786) Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm

More information

Performance Investigation of a Digital Filter bank based Multicarrier Communication System Model using Different types of Prototype Filters

Performance Investigation of a Digital Filter bank based Multicarrier Communication System Model using Different types of Prototype Filters ISSN (Prit) : 30 3765 Electroics ad Istrumetatio Egieerig (A ISO 397: 007 Certified Orgaizatio) Performace Ivestigatio of a Digital Filter bak based ulticarrier Commuicatio System odel usig Differet types

More information

Distributional Similarity Models (cont.)

Distributional Similarity Models (cont.) Sematic Similarity Vector Space Model Similarity Measures cosie Euclidea distace... Clusterig k-meas hierarchical Last Time EM Clusterig Soft versio of K-meas clusterig Iput: m dimesioal objects X = {

More information

Direct Position Determination Algorithm Based on Multi-array in the Presence of Gain-phase Errors

Direct Position Determination Algorithm Based on Multi-array in the Presence of Gain-phase Errors 2016 Joit Iteratioal Coferece o Artificial Itelligece ad Computer Egieerig (AICE 2016) ad Iteratioal Coferece o Network ad Commuicatio Security (NCS 2016) ISBN: 978-1-60595-362-5 Direct Positio Determiatio

More information

A collocation method for singular integral equations with cosecant kernel via Semi-trigonometric interpolation

A collocation method for singular integral equations with cosecant kernel via Semi-trigonometric interpolation Iteratioal Joural of Mathematics Research. ISSN 0976-5840 Volume 9 Number 1 (017) pp. 45-51 Iteratioal Research Publicatio House http://www.irphouse.com A collocatio method for sigular itegral equatios

More information

EL 675 UHF Propagation for Modern Wireless Systems. Henry L. Bertoni Polytechnic University

EL 675 UHF Propagation for Modern Wireless Systems. Henry L. Bertoni Polytechnic University EL 675 UHF Propagatio for Moder Wireless Systems Hery L. Bertoi Polytechic Uiversity otext for Discussig Wireless hael haracteristics Frequecies above ~ 300 MHz (λ < m) adio liks i ma-made eviromets At

More information

Study on Coal Consumption Curve Fitting of the Thermal Power Based on Genetic Algorithm

Study on Coal Consumption Curve Fitting of the Thermal Power Based on Genetic Algorithm Joural of ad Eergy Egieerig, 05, 3, 43-437 Published Olie April 05 i SciRes. http://www.scirp.org/joural/jpee http://dx.doi.org/0.436/jpee.05.34058 Study o Coal Cosumptio Curve Fittig of the Thermal Based

More information

Complex Algorithms for Lattice Adaptive IIR Notch Filter

Complex Algorithms for Lattice Adaptive IIR Notch Filter 4th Iteratioal Coferece o Sigal Processig Systems (ICSPS ) IPCSIT vol. 58 () () IACSIT Press, Sigapore DOI:.7763/IPCSIT..V58. Complex Algorithms for Lattice Adaptive IIR Notch Filter Hog Liag +, Nig Jia

More information

A COMBINED STOCHASTIC/DECISION-AIDED SNR ESTIMATOR. Witthaya Panyaworayan and Rolf Matzner

A COMBINED STOCHASTIC/DECISION-AIDED SNR ESTIMATOR. Witthaya Panyaworayan and Rolf Matzner A COMBINED STOCHASTIC/DECISION-AIDED SNR ESTIMATOR Witthaya Payaworaya ad Rolf Matzer Istitute for Commuicatios Egieerig Federal Armed Forces Uiversity Muich, Germay tpaya@daisy.et.uibw-mueche.de, rolf@ieee.org

More information

Substantiation of the Water Filling Theorem Using Lagrange Multipliers

Substantiation of the Water Filling Theorem Using Lagrange Multipliers J.A. Crawford U048 Water Fillig o Multiple Gaussia Chaels.doc Substatiatio of the Water Fillig heorem Usig Lagrage Multipliers Shao s capacity theorem states that i the case of parallel statistically idepedet

More information

Singular value decomposition. Mathématiques appliquées (MATH0504-1) B. Dewals, Ch. Geuzaine

Singular value decomposition. Mathématiques appliquées (MATH0504-1) B. Dewals, Ch. Geuzaine Lecture 11 Sigular value decompositio Mathématiques appliquées (MATH0504-1) B. Dewals, Ch. Geuzaie V1.2 07/12/2018 1 Sigular value decompositio (SVD) at a glace Motivatio: the image of the uit sphere S

More information

THE TRANSFORMATION MATRIX OF CHEBYSHEV IV BERNSTEIN POLYNOMIAL BASES

THE TRANSFORMATION MATRIX OF CHEBYSHEV IV BERNSTEIN POLYNOMIAL BASES Joural of Mathematical Aalysis ISSN: 17-341, URL: http://iliriascom/ma Volume 7 Issue 4(16, Pages 13-19 THE TRANSFORMATION MATRIX OF CHEBYSHEV IV BERNSTEIN POLYNOMIAL BASES ABEDALLAH RABABAH, AYMAN AL

More information

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons Statistical Aalysis o Ucertaity for Autocorrelated Measuremets ad its Applicatios to Key Comparisos Nie Fa Zhag Natioal Istitute of Stadards ad Techology Gaithersburg, MD 0899, USA Outlies. Itroductio.

More information

Notes on iteration and Newton s method. Iteration

Notes on iteration and Newton s method. Iteration Notes o iteratio ad Newto s method Iteratio Iteratio meas doig somethig over ad over. I our cotet, a iteratio is a sequece of umbers, vectors, fuctios, etc. geerated by a iteratio rule of the type 1 f

More information

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE Joural of ELECTRICAL EGIEERIG, VOL. 56, O. 7-8, 2005, 200 204 OPTIMAL PIECEWISE UIFORM VECTOR QUATIZATIO OF THE MEMORYLESS LAPLACIA SOURCE Zora H. Perić Veljo Lj. Staović Alesadra Z. Jovaović Srdja M.

More information

MVDR and MPDR. Bhaskar D Rao University of California, San Diego

MVDR and MPDR. Bhaskar D Rao University of California, San Diego MVDR ad MPDR Bhaskar D Rao Uiversity of Califoria, Sa Diego Email: brao@ucsd.edu Referece Books. Optimum Array Processig, H. L. Va Trees 2. Stoica, P., & Moses, R. L. (2005). Spectral aalysis of sigals

More information

Inverse Matrix. A meaning that matrix B is an inverse of matrix A.

Inverse Matrix. A meaning that matrix B is an inverse of matrix A. Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix

More information

Tridiagonal reduction redux

Tridiagonal reduction redux Week 15: Moday, Nov 27 Wedesday, Nov 29 Tridiagoal reductio redux Cosider the problem of computig eigevalues of a symmetric matrix A that is large ad sparse Usig the grail code i LAPACK, we ca compute

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

Discrete Orthogonal Moment Features Using Chebyshev Polynomials

Discrete Orthogonal Moment Features Using Chebyshev Polynomials Discrete Orthogoal Momet Features Usig Chebyshev Polyomials R. Mukuda, 1 S.H.Og ad P.A. Lee 3 1 Faculty of Iformatio Sciece ad Techology, Multimedia Uiversity 75450 Malacca, Malaysia. Istitute of Mathematical

More information

ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1.

ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1. Yugoslav Joural of Operatios Research 1 (00), Number 1, 49-60 ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS M. JA]IMOVI], I. KRNI] Departmet of Mathematics

More information

CHAPTER I: Vector Spaces

CHAPTER I: Vector Spaces CHAPTER I: Vector Spaces Sectio 1: Itroductio ad Examples This first chapter is largely a review of topics you probably saw i your liear algebra course. So why cover it? (1) Not everyoe remembers everythig

More information

A meta-converse for private communication over quantum channels

A meta-converse for private communication over quantum channels A meta-coverse for private commuicatio over quatum chaels Mario Berta with Mark M. Wilde ad Marco Tomamichel IEEE Trasactios o Iformatio Theory, 63(3), 1792 1817 (2017) Beyod IID Sigapore - July 17, 2017

More information

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

More information

Lecture 11: Channel Coding Theorem: Converse Part

Lecture 11: Channel Coding Theorem: Converse Part EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig

More information

OBJECTIVES. Chapter 1 INTRODUCTION TO INSTRUMENTATION FUNCTION AND ADVANTAGES INTRODUCTION. At the end of this chapter, students should be able to:

OBJECTIVES. Chapter 1 INTRODUCTION TO INSTRUMENTATION FUNCTION AND ADVANTAGES INTRODUCTION. At the end of this chapter, students should be able to: OBJECTIVES Chapter 1 INTRODUCTION TO INSTRUMENTATION At the ed of this chapter, studets should be able to: 1. Explai the static ad dyamic characteristics of a istrumet. 2. Calculate ad aalyze the measuremet

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

Multivariable System Identification for Integral Controllability Computational Issues

Multivariable System Identification for Integral Controllability Computational Issues Multivariable System Idetificatio for Itegral Cotrollability Computatioal Issues Mar L. Darby, Michael Niolaou Chemical & Biomolecular Egieerig Departmet, Uiversity of Housto, Housto, X 774-44 USA (darbymar@sbcglobal.et,

More information

Properties and Tests of Zeros of Polynomial Functions

Properties and Tests of Zeros of Polynomial Functions Properties ad Tests of Zeros of Polyomial Fuctios The Remaider ad Factor Theorems: Sythetic divisio ca be used to fid the values of polyomials i a sometimes easier way tha substitutio. This is show by

More information

Probability of error for LDPC OC with one co-channel Interferer over i.i.d Rayleigh Fading

Probability of error for LDPC OC with one co-channel Interferer over i.i.d Rayleigh Fading IOSR Joural of Electroics ad Commuicatio Egieerig (IOSR-JECE) e-issn: 2278-2834,p- ISSN: 2278-8735.Volume 9, Issue 4, Ver. III (Jul - Aug. 24), PP 59-63 Probability of error for LDPC OC with oe co-chael

More information

On the Capacity of Symmetric Gaussian Interference Channels with Feedback

On the Capacity of Symmetric Gaussian Interference Channels with Feedback O the Capacity of Symmetric Gaussia Iterferece Chaels with Feedback La V Truog Iformatio Techology Specializatio Departmet ITS FPT Uiversity, Haoi, Vietam E-mail: latv@fpteduv Hirosuke Yamamoto Dept of

More information

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory The Fadig Number of Multiple-Iput Multiple-Output Fadig Chaels with Memory Stefa M. Moser Departmet of Commuicatio Egieerig Natioal Chiao Tug Uiversity NCTU Hsichu, Taiwa Email: stefa.moser@ieee.org Abstract

More information

Module 5 EMBEDDED WAVELET CODING. Version 2 ECE IIT, Kharagpur

Module 5 EMBEDDED WAVELET CODING. Version 2 ECE IIT, Kharagpur Module 5 EMBEDDED WAVELET CODING Versio ECE IIT, Kharagpur Lesso 4 SPIHT algorithm Versio ECE IIT, Kharagpur Istructioal Objectives At the ed of this lesso, the studets should be able to:. State the limitatios

More information

Grouping 2: Spectral and Agglomerative Clustering. CS 510 Lecture #16 April 2 nd, 2014

Grouping 2: Spectral and Agglomerative Clustering. CS 510 Lecture #16 April 2 nd, 2014 Groupig 2: Spectral ad Agglomerative Clusterig CS 510 Lecture #16 April 2 d, 2014 Groupig (review) Goal: Detect local image features (SIFT) Describe image patches aroud features SIFT, SURF, HoG, LBP, Group

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information