{η : η=linear combination of 1, Z 1,, Z n
|
|
- Branden Andrews
- 5 years ago
- Views:
Transcription
1 If 3. Orthogonal projection. Conditional expectation in the wide sense Let (X n n be a sequence of random variables with EX n = σ n and EX n 0. EX k X j = { σk, k = j 0, otherwise, (X n n is the sequence with orthogonal elements. For σk, this sequence forms white noise. If random variables with finite second moments are not orthogonal, they might be orthogonalize by a specific orthogonalization procedure. We start with an example. Example. Let Y and Z be random variables with EY = 0, EZ = 0 and EY <, EZ <. We introduce a projection of X on a pair (, Z by taking a linear combination of and Z: Ŷ = c + c Z. If constants c, c can be chosen such that E{Y Ŷ }Z = 0 the projection Ŷ is named the orthogonal projection. Under EZ > 0, a direct verification shows that the choice c = 0 and c = EY Z gives EZ the orthogonal projection Ŷ = EY Z Z. (3. EZ Consider now the next setting. Let Y, Z,..., Z n be random variables with finite second moments: EY < and EZk <, k =,..., n and introduce a linear space generated by, Z,, Z n : M = {η : η=linear combination of, Z,, Z n } ; if, Z,, Z n is replaced by, Z,, Z n,..., M contains not only linear combination of, Z,, Z n,... but also their limits in L -norm. The orthogonal projection Ŷ of Y on M is an element of M such that Eη ( Y Ŷ = 0 for any η M; symbolically Y Ŷ M. 3.. Computation of the orthogonal projection. Set -m = EY ; -m z k = EZ k, k =,, n; -Z vector(column with entries Z,, Z n -m Z vector(column with entries m z,, m z n -var(y = E(Y m Y ; -cov(z, Z = E(Z m Z (Z m Z T (T is the transposition symbol; -cov(y, Z = E(Y m y (Z m Z T.
2 Theorem 3.. Under nonsingular matrix cov(z, Z, the orthogonal projection of Y on M exists and is defined as follows: Ŷ = m + cov(y, Zcov (Z, Z(Z m Z. (3. Moreover, the variance of the perpendicular Y Ŷ is: E(Y Ŷ = var(y cov(y, Zcov (Z, Zcov T (Y, Z. (3.3 Proof. With an arbitrary vector(raw C, let us introduce a random variable ξ = ( Y m + C ( Z m Z. (3.4 We intend to choose C such that ξ is orthogonal to M. If it is possible, (3.4 provides the decomposition for Y (here M is the orthogonal complement to M: Y = m C ( Z m Z } {{ } M + ξ }{{} M. (3.5 Hence, if C is found, we have Ŷ = m C( Z m Z. Now, owing Z m Z M and ξ M with E ( Z m Z = 0 and Eξ = 0, we find Eξ ( Z m Z = 0. The latter and (3.4 provide cov(y, Z + Ccov(Z, Z = 0 (3.6 and, whereas cov(z, Z is assumed to be nonsingular matrix, the vector C = cov(y, Zcov (Z, Z solves (3.6. Hence, the first statement holds true. Notice now that ξ = Y Ŷ and Eξ = var(y cov(y, Zcov (Z, Zcov T (Y, Z +cov(y, Zcov (Z, Zcov T (Y, Z = var(y cov(y, Zcov (Z, Zcov T (α, Z. Remark 3.. If the matrix cov(z, Z is singular, the statement of the theorem remains valid with cov (Z, Z replaced by the Moore-Penrose pseudo-inverse matrix cov + (Z, Z = T T (T T T T, where T T T = cov(z, Z with T the rectangular matrix of the full rank so that T T T a quadratic nonsingular matrix. 3.. The conditional expectation in the wide sense. The orthogonal projection Ŷ of Y on M is referred as conditional expectation in the wide sense given M, Ŷ = Ê(Y M, and plays an important role in the optimal in the mean square sense linear estimation. We establish below the main properties of Ê(Y M.
3 . EÊ(Y M = EY, Ê(Y M = Y, if Y M Ê(Y M = 0, if Y M. Proof. The random variable Y Ŷ is orthogonal to M, particularly, orthogonal to, that is E(Y Ŷ = 0. If α M, then by the definition Ê(α M = 0.. If c is a constant, then Ê(cY M = cê(z M. Proof. It holds true since the operator of orthogonal projection is linear. 3. If c, c are constants and Y, Y are random variables, possessing second moments, then Ê(c Y + c Y M = c Ê(Y M + c Ê(Y M. Proof. It holds true since the operator of orthogonal projection is linear. 4. If M ia a linear subspace of M: M M, then Ê(Ê(Y M M = Ê( Y M. Proof. Since both Ê(Ê(Y M M and Ê( Y M are from M, it suffices to show that φ = Ê(Ê(α M M Ê( α M is orthogonal to M. To this purpose, the decomposition Y = Ê( Y M + ( Y Ê( Y M is used. Then, whereas ( Y Ê( Y M M, for any ψ M we have EψY = EψÊ( Y M. On the other side, since M M, similarly we get EψÊ (Ê( Y M M = EψÊ( Y M = EψY. Thus, Eψφ = Let M and M are orthogonal linear spaces: M M. Set M = M M. Then for Y, with EY = 0, Ê(Y M = Ê(Y M + Ê(Y M. Proof. Notice that φ = Y Ê(Y M + Ê(Y M is orthogonal to M and M, that is orthogonal to M M := M and the result holds true. 3
4 6. For every η M E(Y η E(Y Ŷ = EY EŶ. Proof. Write E(Y η = = E (Y Ŷ ( η Ŷ = E(Y Ŷ + E(η Ŷ E(Y Ŷ (η Ŷ = E(Y Ŷ + E(η Ŷ E(Y Ŷ. 7. The Cauchy-Schwarz inequality E ( Ê(Y M EY. Proof. Write 0 E ( Y Ê(Y M = EY E ( Ê(Y M Law of large numbers for sequence of orthogonal random variables. Let X, X,... be a sequence of orthogonal random variables with EX n 0 and EX n σ n. Denote by Proposition 3.3. Under n= a n = n n= σ n n X k. <, it holds lim n Ea n = 0. Proof. The result is implied by Ea n = n n σ k and the Kronecker lemma, which being adapted to the case considered, states: σn < = lim σ n n n k = 0. (3.7 We give below a sketch of the proof for (3.7. Set V n = n σ k k and notice that n σk = n (V k V k k. Now, summing by parts we find n σ k = V nn n V k ( k (k, so that σ n k = (V n ( V n k k (k. 4
5 The use of a telescopic sum n = n k ( k (k allows to transform the above equality to the following one n σk = n ( ( Vn V k k (k and evaluate the right hand side of that equality. For n > N, write n ( ( Vn V k k (k 3 = N n ( Vn V k ( k (k 3 + ( ( Vn V n k k (k 3 k=n and notice that the first summand is evaluated above by N n V = N n σ k k 0, n while the second by V V N = σk k=n 0, N. k 3.4. The martingale in the wide sense. Let X, X,... be a sequence of random variables, with EX k <, k, and M be a linear space generated by, X, X,... and M n be a linear space generated by, X, X,..., X n. For a random variable ξ, with Eξ <, set Y n = Ê(ξ M n, n. By property 4. of the conditional expectation in the wide sense, for n > m provides Ê(Y n M m = Y m. The random sequence with this property, created by the real conditional expectation, is named martingale. In our setting, the conditional expectation in the wide sense is used and so a notion of martingale in the wide sense is appropriate. The aim of this Section is to show that Y n, n, converges in L -norm to a limit Ê(ξ M: l.i.m. Ê(ξ M n = Ê(ξ M. (3.8 n Proof of (3.8. Denote y = Y, Y n = Y n Y n, n and notice that for n > m the random variable y n is orthogonal to M X m, whereas Ey n = 0 and moreover Ê(y n M m = Ê(Y n Y n M m = Y m Y m = 0. Hence, Y n = n y k is the sum of zero mean orthogonal random variables x,..., x n and thanks of that EYn = n Ey k. Then, obviously, EYn increases in n. On the other hand, since by property 7. for the 5
6 conditional expectation in the wide sense EYn for any n Eyk Eξ, Eξ, we have that that is Ey k Eξ. Then, the sequence Y n, n converges in L -norm to Y = y k, since by the Cauchy criteria (Theorem. for Lect. ( E(Y n Y m = E k=m+ y k = k=m+ Ey k 0, n, m Markov process in the wide sense. Assume Ê(X n M n = a n X n, n, (3.9 where a n is a sequence of numbers. Set ε n = X n a n X n and show that (ε n n is a sequence of zero mean orthogonal random variables: Eε n 0 Eε n ε m = { 0, n m, EX n a nex n := b b, otherwise. (3.0 In fact, whereas by property. of the conditional expectation EX n = EÊ( X n M n, the first part from (3.0 is valid. By the definition εn is orthogonal to M n and so for M m with m < n. The last property from (3.0 is provided by E ( X n a n X n = EX n a nex n. Thus, the sequence (X n n is defined by a recursion X n = a n X n + ε n subject by X as the initial value, where (ε n n is the sequence of zero mean and orthogonal random variables with Eε n = b n. Random sequences of such the type are named Markov processes in the wide sense. 6
X n = c n + c n,k Y k, (4.2)
4. Linear filtering. Wiener filter Assume (X n, Y n is a pair of random sequences in which the first component X n is referred a signal and the second an observation. The paths of the signal are unobservable
More information3. WIENER PROCESS. STOCHASTIC ITÔ INTEGRAL
3. WIENER PROCESS. STOCHASTIC ITÔ INTEGRAL 3.1. Wiener process. Main properties. A Wiener process notation W W t t is named in the honor of Prof. Norbert Wiener; other name is the Brownian motion notation
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More information2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationGaussian Processes. 1. Basic Notions
Gaussian Processes 1. Basic Notions Let T be a set, and X : {X } T a stochastic process, defined on a suitable probability space (Ω P), that is indexed by T. Definition 1.1. We say that X is a Gaussian
More informationPrinciples of forecasting
2.5 Forecasting Principles of forecasting Forecast based on conditional expectations Suppose we are interested in forecasting the value of y t+1 based on a set of variables X t (m 1 vector). Let y t+1
More informationChapter 8 Gradient Methods
Chapter 8 Gradient Methods An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Introduction Recall that a level set of a function is the set of points satisfying for some constant. Thus, a point
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 26. Estimation: Regression and Least Squares
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 26 Estimation: Regression and Least Squares This note explains how to use observations to estimate unobserved random variables.
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2
MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationOn V-orthogonal projectors associated with a semi-norm
On V-orthogonal projectors associated with a semi-norm Short Title: V-orthogonal projectors Yongge Tian a, Yoshio Takane b a School of Economics, Shanghai University of Finance and Economics, Shanghai
More informationProjection Theorem 1
Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality
More informationarxiv: v5 [math.na] 16 Nov 2017
RANDOM PERTURBATION OF LOW RANK MATRICES: IMPROVING CLASSICAL BOUNDS arxiv:3.657v5 [math.na] 6 Nov 07 SEAN O ROURKE, VAN VU, AND KE WANG Abstract. Matrix perturbation inequalities, such as Weyl s theorem
More information4 Sums of Independent Random Variables
4 Sums of Independent Random Variables Standing Assumptions: Assume throughout this section that (,F,P) is a fixed probability space and that X 1, X 2, X 3,... are independent real-valued random variables
More informationSystems of Linear Equations
Systems of Linear Equations Math 108A: August 21, 2008 John Douglas Moore Our goal in these notes is to explain a few facts regarding linear systems of equations not included in the first few chapters
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationChapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
More informationLecture 23: 6.1 Inner Products
Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such
More informationECE534, Spring 2018: Solutions for Problem Set #5
ECE534, Spring 08: s for Problem Set #5 Mean Value and Autocorrelation Functions Consider a random process X(t) such that (i) X(t) ± (ii) The number of zero crossings, N(t), in the interval (0, t) is described
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationSOLVING FUZZY LINEAR SYSTEMS BY USING THE SCHUR COMPLEMENT WHEN COEFFICIENT MATRIX IS AN M-MATRIX
Iranian Journal of Fuzzy Systems Vol 5, No 3, 2008 pp 15-29 15 SOLVING FUZZY LINEAR SYSTEMS BY USING THE SCHUR COMPLEMENT WHEN COEFFICIENT MATRIX IS AN M-MATRIX M S HASHEMI, M K MIRNIA AND S SHAHMORAD
More informationCh. 15 Forecasting. 1.1 Forecasts Based on Conditional Expectations
Ch 15 Forecasting Having considered in Chapter 14 some of the properties of ARMA models, we now show how they may be used to forecast future values of an observed time series For the present we proceed
More informationarxiv: v1 [math.na] 1 Sep 2018
On the perturbation of an L -orthogonal projection Xuefeng Xu arxiv:18090000v1 [mathna] 1 Sep 018 September 5 018 Abstract The L -orthogonal projection is an important mathematical tool in scientific computing
More informationAppendix B for The Evolution of Strategic Sophistication (Intended for Online Publication)
Appendix B for The Evolution of Strategic Sophistication (Intended for Online Publication) Nikolaus Robalino and Arthur Robson Appendix B: Proof of Theorem 2 This appendix contains the proof of Theorem
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationMAT 771 FUNCTIONAL ANALYSIS HOMEWORK 3. (1) Let V be the vector space of all bounded or unbounded sequences of complex numbers.
MAT 771 FUNCTIONAL ANALYSIS HOMEWORK 3 (1) Let V be the vector space of all bounded or unbounded sequences of complex numbers. (a) Define d : V V + {0} by d(x, y) = 1 ξ j η j 2 j 1 + ξ j η j. Show that
More informationHomework If the inverse T 1 of a closed linear operator exists, show that T 1 is a closed linear operator.
Homework 3 1 If the inverse T 1 of a closed linear operator exists, show that T 1 is a closed linear operator Solution: Assuming that the inverse of T were defined, then we will have to have that D(T 1
More informationLecture 3 - Expectation, inequalities and laws of large numbers
Lecture 3 - Expectation, inequalities and laws of large numbers Jan Bouda FI MU April 19, 2009 Jan Bouda (FI MU) Lecture 3 - Expectation, inequalities and laws of large numbersapril 19, 2009 1 / 67 Part
More informationarxiv: v1 [math.pr] 22 May 2008
THE LEAST SINGULAR VALUE OF A RANDOM SQUARE MATRIX IS O(n 1/2 ) arxiv:0805.3407v1 [math.pr] 22 May 2008 MARK RUDELSON AND ROMAN VERSHYNIN Abstract. Let A be a matrix whose entries are real i.i.d. centered
More informationRisk-Minimality and Orthogonality of Martingales
Risk-Minimality and Orthogonality of Martingales Martin Schweizer Universität Bonn Institut für Angewandte Mathematik Wegelerstraße 6 D 53 Bonn 1 (Stochastics and Stochastics Reports 3 (199, 123 131 2
More informationCHAPTER II HILBERT SPACES
CHAPTER II HILBERT SPACES 2.1 Geometry of Hilbert Spaces Definition 2.1.1. Let X be a complex linear space. An inner product on X is a function, : X X C which satisfies the following axioms : 1. y, x =
More information4 Derivations of the Discrete-Time Kalman Filter
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof N Shimkin 4 Derivations of the Discrete-Time
More informationLinear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification
page i Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification Anders Lindquist and Giorgio Picci January 28, 2007 page i ii Boo page Contents Preface vii 1 Introduction
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationESTIMATION THEORY. Chapter Estimation of Random Variables
Chapter ESTIMATION THEORY. Estimation of Random Variables Suppose X,Y,Y 2,...,Y n are random variables defined on the same probability space (Ω, S,P). We consider Y,...,Y n to be the observed random variables
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationLinear-quadratic control problem with a linear term on semiinfinite interval: theory and applications
Linear-quadratic control problem with a linear term on semiinfinite interval: theory and applications L. Faybusovich T. Mouktonglang Department of Mathematics, University of Notre Dame, Notre Dame, IN
More informationPart III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be ed to
TIME SERIES Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be emailed to Y.Chen@statslab.cam.ac.uk. 1. Let {X t } be a weakly stationary process with mean zero and let
More informationLearning Theory of Randomized Kaczmarz Algorithm
Journal of Machine Learning Research 16 015 3341-3365 Submitted 6/14; Revised 4/15; Published 1/15 Junhong Lin Ding-Xuan Zhou Department of Mathematics City University of Hong Kong 83 Tat Chee Avenue Kowloon,
More information1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?
. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in
More informationProposition 42. Let M be an m n matrix. Then (32) N (M M)=N (M) (33) R(MM )=R(M)
RODICA D. COSTIN. Singular Value Decomposition.1. Rectangular matrices. For rectangular matrices M the notions of eigenvalue/vector cannot be defined. However, the products MM and/or M M (which are square,
More information3 Integration and Expectation
3 Integration and Expectation 3.1 Construction of the Lebesgue Integral Let (, F, µ) be a measure space (not necessarily a probability space). Our objective will be to define the Lebesgue integral R fdµ
More informationBasic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = The errors are uncorrelated with common variance:
8. PROPERTIES OF LEAST SQUARES ESTIMATES 1 Basic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = 0. 2. The errors are uncorrelated with common variance: These assumptions
More information0.1 Uniform integrability
Copyright c 2009 by Karl Sigman 0.1 Uniform integrability Given a sequence of rvs {X n } for which it is known apriori that X n X, n, wp1. for some r.v. X, it is of great importance in many applications
More information5.1 Consistency of least squares estimates. We begin with a few consistency results that stand on their own and do not depend on normality.
88 Chapter 5 Distribution Theory In this chapter, we summarize the distributions related to the normal distribution that occur in linear models. Before turning to this general problem that assumes normal
More informationMathematics of Information Spring semester 2018
Communication Technology Laboratory Prof. Dr. H. Bölcskei Sternwartstrasse 7 CH-809 Zürich Mathematics of Information Spring semester 08 Solution to Homework Problem Overcomplete expansion in R a) Consider
More informationRepresentation of integrated autoregressive processes in Banach space
Representation of integrated autoregressive processes in Banach space Won-Ki Seo Department of Economics, University of California, San Diego April 2, 218 Very preliminary and incomplete. Comments welcome.
More informationV. SUBSPACES AND ORTHOGONAL PROJECTION
V. SUBSPACES AND ORTHOGONAL PROJECTION In this chapter we will discuss the concept of subspace of Hilbert space, introduce a series of subspaces related to Haar wavelet, explore the orthogonal projection
More informationI forgot to mention last time: in the Ito formula for two standard processes, putting
I forgot to mention last time: in the Ito formula for two standard processes, putting dx t = a t dt + b t db t dy t = α t dt + β t db t, and taking f(x, y = xy, one has f x = y, f y = x, and f xx = f yy
More informationLinear regression COMS 4771
Linear regression COMS 4771 1. Old Faithful and prediction functions Prediction problem: Old Faithful geyser (Yellowstone) Task: Predict time of next eruption. 1 / 40 Statistical model for time between
More informationA SYNOPSIS OF HILBERT SPACE THEORY
A SYNOPSIS OF HILBERT SPACE THEORY Below is a summary of Hilbert space theory that you find in more detail in any book on functional analysis, like the one Akhiezer and Glazman, the one by Kreiszig or
More informationOptimal stopping for non-linear expectations Part I
Stochastic Processes and their Applications 121 (2011) 185 211 www.elsevier.com/locate/spa Optimal stopping for non-linear expectations Part I Erhan Bayraktar, Song Yao Department of Mathematics, University
More informationarxiv: v2 [cs.it] 8 Apr 2013
ONE-BIT COMPRESSED SENSING WITH NON-GAUSSIAN MEASUREMENTS ALBERT AI, ALEX LAPANOWSKI, YANIV PLAN, AND ROMAN VERSHYNIN arxiv:1208.6279v2 [cs.it] 8 Apr 2013 Abstract. In one-bit compressed sensing, previous
More informationCLASSICAL PROBABILITY MODES OF CONVERGENCE AND INEQUALITIES
CLASSICAL PROBABILITY 2008 2. MODES OF CONVERGENCE AND INEQUALITIES JOHN MORIARTY In many interesting and important situations, the object of interest is influenced by many random factors. If we can construct
More informationCourse Summary Math 211
Course Summary Math 211 table of contents I. Functions of several variables. II. R n. III. Derivatives. IV. Taylor s Theorem. V. Differential Geometry. VI. Applications. 1. Best affine approximations.
More informationInner product spaces. Layers of structure:
Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationAn Introduction to Signal Detection and Estimation - Second Edition Chapter III: Selected Solutions
An Introduction to Signal Detection and Estimation - Second Edition Chapter III: Selected Solutions H. V. Poor Princeton University March 17, 5 Exercise 1: Let {h k,l } denote the impulse response of a
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationX t = a t + r t, (7.1)
Chapter 7 State Space Models 71 Introduction State Space models, developed over the past 10 20 years, are alternative models for time series They include both the ARIMA models of Chapters 3 6 and the Classical
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationDo not copy, quote, or cite without permission LECTURE 4: THE GENERAL LISREL MODEL
LECTURE 4: THE GENERAL LISREL MODEL I. QUICK REVIEW OF A LITTLE MATRIX ALGEBRA. II. A SIMPLE RECURSIVE MODEL IN LATENT VARIABLES. III. THE GENERAL LISREL MODEL IN MATRIX FORM. A. SPECIFYING STRUCTURAL
More informationNormed & Inner Product Vector Spaces
Normed & Inner Product Vector Spaces ECE 174 Introduction to Linear & Nonlinear Optimization Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 174 Fall 2016 1 / 27 Normed
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationMATH 650. THE RADON-NIKODYM THEOREM
MATH 650. THE RADON-NIKODYM THEOREM This note presents two important theorems in Measure Theory, the Lebesgue Decomposition and Radon-Nikodym Theorem. They are not treated in the textbook. 1. Closed subspaces
More informationarxiv: v1 [math.co] 17 Dec 2007
arxiv:07.79v [math.co] 7 Dec 007 The copies of any permutation pattern are asymptotically normal Milós Bóna Department of Mathematics University of Florida Gainesville FL 36-805 bona@math.ufl.edu Abstract
More informationRANDOM PROCESSES. THE FINAL TEST. Prof. R. Liptser and P. Chigansky 9:00-12:00, 18 of October, Student ID:
RANDOM PROCESSES. THE FINAL TEST. Prof. R. Liptser and P. Chigansky 9:00-12:00, 18 of October, 2001 Student ID: any supplementary material is allowed duration of the exam is 3 hours write briefly the main
More information1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,
Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional
More informationMATRICES ARE SIMILAR TO TRIANGULAR MATRICES
MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,
More informationChapter 2 Finite Element Spaces for Linear Saddle Point Problems
Chapter 2 Finite Element Spaces for Linear Saddle Point Problems Remark 2.1. Motivation. This chapter deals with the first difficulty inherent to the incompressible Navier Stokes equations, see Remark
More informationRANDOM MATRICES: LAW OF THE DETERMINANT
RANDOM MATRICES: LAW OF THE DETERMINANT HOI H. NGUYEN AND VAN H. VU Abstract. Let A n be an n by n random matrix whose entries are independent real random variables with mean zero and variance one. We
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationLinear algebra for computational statistics
University of Seoul May 3, 2018 Vector and Matrix Notation Denote 2-dimensional data array (n p matrix) by X. Denote the element in the ith row and the jth column of X by x ij or (X) ij. Denote by X j
More informationELA THE OPTIMAL PERTURBATION BOUNDS FOR THE WEIGHTED MOORE-PENROSE INVERSE. 1. Introduction. Let C m n be the set of complex m n matrices and C m n
Electronic Journal of Linear Algebra ISSN 08-380 Volume 22, pp. 52-538, May 20 THE OPTIMAL PERTURBATION BOUNDS FOR THE WEIGHTED MOORE-PENROSE INVERSE WEI-WEI XU, LI-XIA CAI, AND WEN LI Abstract. In this
More information4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan
The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan Wir müssen wissen, wir werden wissen. David Hilbert We now continue to study a special class of Banach spaces,
More information180B Lecture Notes, W2011
Bruce K. Driver 180B Lecture Notes, W2011 January 11, 2011 File:180Lec.tex Contents Part 180B Notes 0 Course Notation List......................................................................................................................
More informationOn the Local Convergence of an Iterative Approach for Inverse Singular Value Problems
On the Local Convergence of an Iterative Approach for Inverse Singular Value Problems Zheng-jian Bai Benedetta Morini Shu-fang Xu Abstract The purpose of this paper is to provide the convergence theory
More information1 Exercises for lecture 1
1 Exercises for lecture 1 Exercise 1 a) Show that if F is symmetric with respect to µ, and E( X )
More informationOn large deviations for combinatorial sums
arxiv:1901.0444v1 [math.pr] 14 Jan 019 On large deviations for combinatorial sums Andrei N. Frolov Dept. of Mathematics and Mechanics St. Petersburg State University St. Petersburg, Russia E-mail address:
More informationInstrumental Variables
Università di Pavia 2010 Instrumental Variables Eduardo Rossi Exogeneity Exogeneity Assumption: the explanatory variables which form the columns of X are exogenous. It implies that any randomness in the
More informationFall TMA4145 Linear Methods. Exercise set 10
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 207 Exercise set 0 Please justify your answers! The most important part is how you arrive at
More informationPCA with random noise. Van Ha Vu. Department of Mathematics Yale University
PCA with random noise Van Ha Vu Department of Mathematics Yale University An important problem that appears in various areas of applied mathematics (in particular statistics, computer science and numerical
More informationExercise Sheet 1.
Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giang-vien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?
More informationk=1 = e λ λ k λk 1 k! (k + 1)! = λ. k=0
11. Poisson process. Poissonian White Boise. Telegraphic signal 11.1. Poisson process. First, recall a definition of Poisson random variable π. A random random variable π, valued in the set {, 1,, }, is
More informationapproximation algorithms I
SUM-OF-SQUARES method and approximation algorithms I David Steurer Cornell Cargese Workshop, 201 meta-task encoded as low-degree polynomial in R x example: f(x) = i,j n w ij x i x j 2 given: functions
More informationA note on the σ-algebra of cylinder sets and all that
A note on the σ-algebra of cylinder sets and all that José Luis Silva CCM, Univ. da Madeira, P-9000 Funchal Madeira BiBoS, Univ. of Bielefeld, Germany (luis@dragoeiro.uma.pt) September 1999 Abstract In
More informationConvergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit
Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Evan Kwiatkowski, Jan Mandel University of Colorado Denver December 11, 2014 OUTLINE 2 Data Assimilation Bayesian Estimation
More informationAnalysis-3 lecture schemes
Analysis-3 lecture schemes (with Homeworks) 1 Csörgő István November, 2015 1 A jegyzet az ELTE Informatikai Kar 2015. évi Jegyzetpályázatának támogatásával készült Contents 1. Lesson 1 4 1.1. The Space
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationMATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.
as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations
More informationTrace Class Operators and Lidskii s Theorem
Trace Class Operators and Lidskii s Theorem Tom Phelan Semester 2 2009 1 Introduction The purpose of this paper is to provide the reader with a self-contained derivation of the celebrated Lidskii Trace
More informationANOVA: Analysis of Variance - Part I
ANOVA: Analysis of Variance - Part I The purpose of these notes is to discuss the theory behind the analysis of variance. It is a summary of the definitions and results presented in class with a few exercises.
More informationarxiv: v1 [math.pr] 22 Dec 2018
arxiv:1812.09618v1 [math.pr] 22 Dec 2018 Operator norm upper bound for sub-gaussian tailed random matrices Eric Benhamou Jamal Atif Rida Laraki December 27, 2018 Abstract This paper investigates an upper
More informationAPPLICATIONS OF DIFFERENTIABILITY IN R n.
APPLICATIONS OF DIFFERENTIABILITY IN R n. MATANIA BEN-ARTZI April 2015 Functions here are defined on a subset T R n and take values in R m, where m can be smaller, equal or greater than n. The (open) ball
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More information