Optimization Theory. Linear Operators and Adjoints

Size: px
Start display at page:

Download "Optimization Theory. Linear Operators and Adjoints"

Transcription

1 Optimization Theory Linear Operators and Adjoints

2 A transformation T. : X Y y Linear Operators y T( x), x X, yy is the image of x under T The domain of T on which T can be defined : D X The range of T { y Y xd, y T( x)} 2

3 Linear Operators T( S): the image of S in Y for S X T ( P): the inverse image of P { xx T( x) P, for PY} Linear transformation (linear operator) A, B, A x for A( x) RA ( ): The range of A : X Y R( A) Y N( A) { xx Ax }: The null space of A N( A) X 3

4 Proposition Linear Operators A linear operator on a normed space X is continuous at every point in X if it is continuous at a single point. Definition A linear operator A from a normed space X to a normed space Y is said to be bounded if a constant M such that Ax M x, x X. The smallest M which satisfies the above condition is denoted A and called the norm of A. Equivalent definition of the norm A Ax sup x x Ax sup x x 4

5 Linear Operators Proposition A linear operator is bounded iff it is continuous. If addition and scalar multiplication are defined by ( A A ) x Ax A x 2 2 ( A) x ( Ax), the linear operators from X to Y form a linear vector space. Definition The normed space of all bounded linear operators from the normed space X into the normed space Y is denoted BX (, Y). 5

6 Theorem Linear Operators Let X and Y be normed spaces with Y complete. Then the space BX (, Y) is complete. Products of operators S: X Y, T : Y Z, Define TS: X Z by TS( x) T( Sx), x X Proposition Let X, Y, Z be normed spaces and suppose S B( X, Y), T B( Y, Z). Then TS T S 6

7 Linear Operators Ex X C[0, ] A: X X A x K(, s t)() x t dt where the 0 function K is continuous on the unit square 0 s, 0 t. A is linear. Compute A Actually Ax max K( s, t) x( t) dt 0s 0 max Kst (, ) dtmax xt ( ) 0s 0 0t max Kst (, ) dt x 0s 0 A max Kst (, ) dt 0s 0 A max Kst (, ) dt 0s 0 7

8 Linear Operators Ex X n E, A : X 2 Ax [ x, AAx] X A: n n matrix A : transpose of A Denote AA Q A max [ x, Qx] x Q : symmetric, 0 Q : has nonnegative eigenvalues A ( Q) max 8

9 Inverse Operators Inverse Operators A : X Y, Consider A x y. For a given yy This equation may :. have a unique solution 2. have no solution x X 3. have more than one solution Cases 2 & 3 Optimization problem Condition holds for every y Y, iff A: X Y is one-to-one and RA ( ) Y The operator A has an inverse if Ax y, then x A y A 9

10 Inverse Operators Proposition If a linear operator A : X Y has an inverse, the A inverse is linear. The Banach Inverse Theorem A supporting role in the optimization theory Lemma (Baire) A Banach space X is not the union of countably many nowhere dense sets in X. Theorem (Banach Inverse Theorem) Let A be a continuous linear operator from a Banach space X onto a Banach space Y and suppose A A that the inverse operator exists. Then is continuous. 0

11 Adjoint Operators Adjoint Operators X Definition A transformation from a vector space into the space of real (or complex) scalars is said to be a functional on X. Ex. E n, a linear functional can be expressed f( x) kk for k where the are fixed scalars. x (, 2,..., n ), k n Ex. Ex. C[0, ], f( x) x linear functional 2 [0, ], f( x) y() t x() t dt, for y L 2 [0, ] L 2 0

12 Linear Functionals Linear functional Definition A linear functional f on normed space is bounded if there is a constant M f( x) M x, x X. The smallest such constant M is called the norm of f and is denoted f. Thus, Alternative definitions f inf { M : f( x) M x, xx} f sup x sup f( x) x sup f( x) x f( x) x 2

13 Linear Functionals and Dual Space Definition Let X be a normed linear vector space. The space of all bounded linear functionals on X is called the normed dual of X and is denoted X. The norm of an element f X is x, x 2 f sup f( x) x bounded linear functionals on X elements of X The value of the linear functional at the point is denoted by x ( x) or x, x x X x X 3

14 Linear Functionals and Dual Spaces Dual of E n : E n Dual of l, p pq p Dual of Dual space of p Dual space of l L [0, ], p, p q L p p : : l q L q Theorem (Riesz-Frechet, Riesz representation theorem) If f is a bounded linear functional on a Hilbert space H, there exists a unique vector y H such that for all x H, f( x) [ x, y]. Furthermore, we have f y and every y determines a unique bounded linear functional in this way. 4

15 Adjoint Operators Definition Let X and Y be normed spaces and let A BX (, Y). The adjoint operator A : Y X is defined by the equation x, A y Ax, y Given a fixed y Y, the quantity Ax, y is a linear functional on X Ax, y y Ax y A x bounded an element x of Define A y x, unique, linear X. 5

16 Adjoint Operators A : Y X X A Y Dual space X A Y An operator and its adjoints y Ax A y x x X ( ) ( )( ). We may write yaay functional on X functional on Y 6

17 Adjoint Operators Theorem The adjoint operator A of linear operator A BX (, Y) is linear and bounded with A A. Proposition Adjoints satisfy the following properties. If I is the identity operator on a normed space X, then 2. If A, A B( X, Y), then 2 3. If A BX (, Y) and is a real scalar, then 4. If A BX (, Y), ABY (, Z), then 2 ( A A ) A A 2 2 ( A A) ( A) A A If A BX (, Y) and A has a bounded inverse, then A I I ( A ) ( A ) 7

18 An important special case Adjoint Operators A: H G, H, G Hilbert spaces If H, G are real, [ Ax, y] [ x, A y] H H, G G Definition of the adjoint operator on Hilbert spaces In Hilbert space, we have Definition A A A bounded linear operator A mapping a real Hilbert space into itself is said to be self-adjoint if A A. 8

19 Definition Self-Adjoint Operators A self-adjoint linear operator A on a Hilbert space H is said to be positive semidefinite if [ x, Ax] 0 xh. Ex. Let X Y E n. Then A : X X is represented by an n n matrix. n Thus the i-th component of Ax is ( Ax) i aijxj j 9

20 Compute A. For A A Ex. Where Then, is the matrix with elements: is the transpose of Adjoint Operators n n i j ij aji yy,[ Ax, y] ya x A a i ij j j ij i j i [, ] Let X Y L 2 [0, ] and define Ax K(, t s) x( s) ds, t [0,] 0 0 Kt (, s) dsdt [ Ax, y] y( t) K( t, s) x( s) dsdt xs () Kt (, sytdtds ) () n n x a y x A y 20

21 Adjoint Operators Or, by interchanging the role of and s t [ A x, y] x( t) K( s, t) y( s) dsdt 0 0 [ x, A y] where A y K(, s t)() y s ds 0 2

22 Ex Let X Y L 2 [0, ] and define t 0 Adjoint Operators Ax K(, t s) x( s) ds, t [0,] with Then 2 Kt (, s) dtds. 0 0 [ Ax, y] y( t) K( t, s) x( s) dsdt 0 0 t 0 0 t ytkt () (, sxsdsdt ) () 22

23 Adjoint Operators s s s t s t t t Integration vertically Integration horizontally and then horizontally and then vertically 23

24 Adjoint Operators [ Ax, y] y( t) K( t, s) x( s) dtds 0 s xs ()( Ktsytdt (,) () ) ds 0 s Or, interchanging the role of and t s [ Ax, y] x( t) ( K( s, t) y( s) ds) dt 0 t [ x, A y] where A y K(, s t) y() s ds t 24

25 Range and Null spaces Relations between Range and Null space Theorem Let X and Y be normed spaces and let A BX (, Y). Then [ RA ( )] NA ( ) Proof Let y N( A ) and y R( A). Then y Ax for some x X. y, y Ax, y x, A y 0 N( A ) [ R( A)] Assume y [ R( A)]. Then for every x X, x, A y 0 RA NA [ ( )] ( ) Ax y, 0 25

26 Range and Null spaces Theorem Let A be a bounded linear operator acting between two real Hilbert spaces. Then [ RA ( )] NA ( ) RA ( ) [ NA ( )] [ RA ( )] NA ( ) RA ( ) [ ( )] NA 26

27 Solving Equation Optimization in Hilbert Space A: G H, G, H : Hilbert spaces Linear equation A x y, may for a given y H. possess a unique solution x G 2. possess no solution 3. possess more than one solution Cases 2 & 3 : Optimization problem Normal equation Case 2 an approximate solution 27

28 Theorem Let G and H be Hilbert spaces and let A BG (, H). Then for a fixed y H the vector xg minimizes y Ax if and only if A Ax A y Proof Problem min y yˆ where yˆ R( A). By P.T. ŷ is a minimizing vector iff y yˆ R( A) ˆ [ ( )] ( ). y y R A N A Solving Equation Or ( ˆ) A y y A ya Ax y y R( A) RA ( ) ŷ ŷ Ax 28

29 If A A is invertible x ( A A) A y Ex. Basic approximation problem in Hilbert space { x, x2,, x n }: independent set of vectors M f { x, x2,, x n }. Find the best approximation to y H onto M. n yˆ aixi i n Define the operator A : E H by the equation n Aa ax i i where a i a a n Solving Equation T By Theorem minimize y Aa. AAa=A y 29

30 Solving Equation Find the operator A. n [ x, Aa] [ x, a x ] a[ x, x ] [ ] E n n A : H i i i i i i xx xx n z,a z,, 2 T n n A x [ x, x ][ x, x ] [ x, x n ] AA: E E nn E n T matrix AAa=A y [ x, x] [ x2, x] [ xn, x] a [ y, x] [, 2] [ 2, 2] [, 2] 2 [, 2] x x x x xn x a y x [ x, x ] [ x, x ] [ x, x ] a [ y, x ] n 2 n n n n n Normal equations a=(a A) A - y 30

31 Solving Equation Dual Problem Case 3 : the equation Ax y has more than one solution minimum norm problem N( A) {0} Theorem Let G and H be Hilbert spaces and let AB( GH, ) with range closed in H. Then the vector x of minimum norm satisfying Ax y is given by x A z where z is any solution of AAzy. Proof If x is a solution of Ax y, the general solution is x x u where u N( A) a unique vector ˆx of minimum norm satisfying Axˆ y, xˆ N( A) 3

32 Solving Equation xx N( A) V : linear variety ˆx xˆ N( A) N( A) ˆ [ ( )] ( ): x N A R A closed Hence xˆ A z for some z H. Axˆ y AA z y. If ( ), AA ˆ ( ) x A AA y 32

33 Ex. A linear dynamic system governed by a set of differential eqs: x () t Fx() t bu() t x : n vector F : n n matrix b : n vector u : scalar control function Find the minimum energy solution s.t. x(0) θ, x( T ) x x ( ) ( ) T F T t b ( ) 0 T e u t dt Define A : L [0, T] E A 2 T F( Tt) u e bu() t dt 0 Solving Equation n T 2 min ( ) 0 u u t dt 33

34 The problem: Find the u of minimum norm satisfying Au x. R( A) is finite dimensional closed. By Theorem u Az where AA z x Calculation of the operators A and AA For any ul y E where AA 2, n is the n nmatrix AA T F( Tt) [ ya, u] n y e bu( t) dt E 0 T F( Tt) F ( T t) e Ayb y T T t T t e bbe dt F( ) F( ) 0 If the matrix AA is invertible, Solving Equation 0 ye bu() t dt [ A y, u] u A ( AA ) x L 2 34

35 Definition Among all vectors Pseudoinverse Operators More general and more complete approach to the problem of finding approximate or minimum norm solutions to Ax y. pseudoinverse operator Suppose A BGH (, ) with R( A) closed x 0 x G Ax y min Ax y, x satisfying Let be the unique vector of minimum norm. The pseudoinverse A of A is the operator mapping y into its corresponding x 0 as y vaires over H. Discussion. min Ax y approximation of y on R( A) yˆ Ax is x unique, although x may not be. Ax yˆ is a linear variety, a translation of N( A) closed a unique x0 of minimum norm. A is well defined. 35

36 Geometric interpretation of A G N( A) N( A) H R( A) R( A) Pseudoinverse Operators ( R( A) is closed) A : The operator A restricted to N( A) N ( A) A : N( A) R( A) N ( A) onto one-to-one : has a linear inverse, bounded ( By the Banach inverse theorem) Definition We say that a vector space X is the direct sum of two subspaces M and N if every vector x X has a unique representation of the form x m n where m M and n N. We describe this situation by notation X M N. 36

37 Pseudoinverse Operators Inverse operator defines A on R( A). Its domain is extended to all of by defining Ayθ for y R( A) A H N( A ) R( A) A N( A ) R( A) A A θ θ The pseudo inverse operator 37

38 Verification of This Definition For y H, y yˆ y uniquely where yˆ R( A), y ˆ is the best approximation to y in R( A). y R( A) Ay A( yˆ y ˆ ) Ay. Define x0 A yˆ, then by definition Ax0 yˆ. Furthermore x0 N( A) minimum norm solution of Ax ˆ y x x 0 θ Linear variety of x s with Ax yˆ translation of N( A) A minimum norm sol y ŷ θ R( A) yˆ R( A) 38

39 Verification of This Definition Inverse of A : Ax y y R( A) no sol approximate ŷ For Ax yˆ, N( A) more than one solution minimum norm solution x 0 x A y, A y A yˆ A y 0 0 R( A) 39

40 Algebraic Properties of Pseudoinverse Proposition Let A be a bounded linear operator with closed range and A denotes its pseudoinverse. Then. A is linear 2. A is bounded 3. ( A ) A 4. ( A ) ( A ) 5. AAA A 6. AA A A 7. ( AA) AA 8. A ( A A) A 9. A A ( AA ) If AA is invertible, then If AA is invertible, then A ( A A) A A A ( AA ) 40

1 Definition and Basic Properties of Compa Operator

1 Definition and Basic Properties of Compa Operator 1 Definition and Basic Properties of Compa Operator 1.1 Let X be a infinite dimensional Banach space. Show that if A C(X ), A does not have bounded inverse. Proof. Denote the unit ball of X by B and the

More information

2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers.

2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers. Chapter 3 Duality in Banach Space Modern optimization theory largely centers around the interplay of a normed vector space and its corresponding dual. The notion of duality is important for the following

More information

Pseudoinverse and Adjoint Operators

Pseudoinverse and Adjoint Operators ECE 275AB Lecture 5 Fall 2008 V1.1 c K. Kreutz-Delgado, UC San Diego p. 1/1 Lecture 5 ECE 275A Pseudoinverse and Adjoint Operators ECE 275AB Lecture 5 Fall 2008 V1.1 c K. Kreutz-Delgado, UC San Diego p.

More information

ECE 275A Homework #3 Solutions

ECE 275A Homework #3 Solutions ECE 75A Homework #3 Solutions. Proof of (a). Obviously Ax = 0 y, Ax = 0 for all y. To show sufficiency, note that if y, Ax = 0 for all y, then it must certainly be true for the particular value of y =

More information

Analysis Preliminary Exam Workshop: Hilbert Spaces

Analysis Preliminary Exam Workshop: Hilbert Spaces Analysis Preliminary Exam Workshop: Hilbert Spaces 1. Hilbert spaces A Hilbert space H is a complete real or complex inner product space. Consider complex Hilbert spaces for definiteness. If (, ) : H H

More information

Normed & Inner Product Vector Spaces

Normed & Inner Product Vector Spaces Normed & Inner Product Vector Spaces ECE 174 Introduction to Linear & Nonlinear Optimization Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 174 Fall 2016 1 / 27 Normed

More information

INF-SUP CONDITION FOR OPERATOR EQUATIONS

INF-SUP CONDITION FOR OPERATOR EQUATIONS INF-SUP CONDITION FOR OPERATOR EQUATIONS LONG CHEN We study the well-posedness of the operator equation (1) T u = f. where T is a linear and bounded operator between two linear vector spaces. We give equivalent

More information

Analysis Comprehensive Exam Questions Fall 2008

Analysis Comprehensive Exam Questions Fall 2008 Analysis Comprehensive xam Questions Fall 28. (a) Let R be measurable with finite Lebesgue measure. Suppose that {f n } n N is a bounded sequence in L 2 () and there exists a function f such that f n (x)

More information

There are six more problems on the next two pages

There are six more problems on the next two pages Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with

More information

Projection Theorem 1

Projection Theorem 1 Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

Problem 1 (Exercise 2.2, Monograph)

Problem 1 (Exercise 2.2, Monograph) MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma

More information

STAT200C: Review of Linear Algebra

STAT200C: Review of Linear Algebra Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose

More information

The goal of this chapter is to study linear systems of ordinary differential equations: dt,..., dx ) T

The goal of this chapter is to study linear systems of ordinary differential equations: dt,..., dx ) T 1 1 Linear Systems The goal of this chapter is to study linear systems of ordinary differential equations: ẋ = Ax, x(0) = x 0, (1) where x R n, A is an n n matrix and ẋ = dx ( dt = dx1 dt,..., dx ) T n.

More information

Orthogonality and Least Squares

Orthogonality and Least Squares 6 Orthogonality and Least Squares 6.1 INNER PRODUCT, LENGTH, AND ORTHOGONALITY INNER PRODUCT If u and v are vectors in, then we regard u and v as matrices. n 1 n The transpose u T is a 1 n matrix, and

More information

Functional Analysis. Martin Brokate. 1 Normed Spaces 2. 2 Hilbert Spaces The Principle of Uniform Boundedness 32

Functional Analysis. Martin Brokate. 1 Normed Spaces 2. 2 Hilbert Spaces The Principle of Uniform Boundedness 32 Functional Analysis Martin Brokate Contents 1 Normed Spaces 2 2 Hilbert Spaces 2 3 The Principle of Uniform Boundedness 32 4 Extension, Reflexivity, Separation 37 5 Compact subsets of C and L p 46 6 Weak

More information

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )

More information

SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS

SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS G. RAMESH Contents Introduction 1 1. Bounded Operators 1 1.3. Examples 3 2. Compact Operators 5 2.1. Properties 6 3. The Spectral Theorem 9 3.3. Self-adjoint

More information

16 1 Basic Facts from Functional Analysis and Banach Lattices

16 1 Basic Facts from Functional Analysis and Banach Lattices 16 1 Basic Facts from Functional Analysis and Banach Lattices 1.2.3 Banach Steinhaus Theorem Another fundamental theorem of functional analysis is the Banach Steinhaus theorem, or the Uniform Boundedness

More information

2.3 Variational form of boundary value problems

2.3 Variational form of boundary value problems 2.3. VARIATIONAL FORM OF BOUNDARY VALUE PROBLEMS 21 2.3 Variational form of boundary value problems Let X be a separable Hilbert space with an inner product (, ) and norm. We identify X with its dual X.

More information

1.2 Fundamental Theorems of Functional Analysis

1.2 Fundamental Theorems of Functional Analysis 1.2 Fundamental Theorems of Functional Analysis 15 Indeed, h = ω ψ ωdx is continuous compactly supported with R hdx = 0 R and thus it has a unique compactly supported primitive. Hence fφ dx = f(ω ψ ωdy)dx

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information

THEOREMS, ETC., FOR MATH 515

THEOREMS, ETC., FOR MATH 515 THEOREMS, ETC., FOR MATH 515 Proposition 1 (=comment on page 17). If A is an algebra, then any finite union or finite intersection of sets in A is also in A. Proposition 2 (=Proposition 1.1). For every

More information

FUNCTIONAL ANALYSIS-NORMED SPACE

FUNCTIONAL ANALYSIS-NORMED SPACE MAT641- MSC Mathematics, MNIT Jaipur FUNCTIONAL ANALYSIS-NORMED SPACE DR. RITU AGARWAL MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY JAIPUR 1. Normed space Norm generalizes the concept of length in an arbitrary

More information

Functional Analysis I

Functional Analysis I Functional Analysis I Course Notes by Stefan Richter Transcribed and Annotated by Gregory Zitelli Polar Decomposition Definition. An operator W B(H) is called a partial isometry if W x = X for all x (ker

More information

A Brief Introduction to Functional Analysis

A Brief Introduction to Functional Analysis A Brief Introduction to Functional Analysis Sungwook Lee Department of Mathematics University of Southern Mississippi sunglee@usm.edu July 5, 2007 Definition 1. An algebra A is a vector space over C with

More information

Introduction to Empirical Processes and Semiparametric Inference Lecture 22: Preliminaries for Semiparametric Inference

Introduction to Empirical Processes and Semiparametric Inference Lecture 22: Preliminaries for Semiparametric Inference Introduction to Empirical Processes and Semiparametric Inference Lecture 22: Preliminaries for Semiparametric Inference Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics

More information

CHAPTER V DUAL SPACES

CHAPTER V DUAL SPACES CHAPTER V DUAL SPACES DEFINITION Let (X, T ) be a (real) locally convex topological vector space. By the dual space X, or (X, T ), of X we mean the set of all continuous linear functionals on X. By the

More information

Lecture 1 Operator spaces and their duality. David Blecher, University of Houston

Lecture 1 Operator spaces and their duality. David Blecher, University of Houston Lecture 1 Operator spaces and their duality David Blecher, University of Houston July 28, 2006 1 I. Introduction. In noncommutative analysis we replace scalar valued functions by operators. functions operators

More information

3.2 Real and complex symmetric bilinear forms

3.2 Real and complex symmetric bilinear forms Then, the adjoint of f is the morphism 3 f + : Q 3 Q 3 ; x 5 0 x. 0 2 3 As a verification, you can check that 3 0 B 5 0 0 = B 3 0 0. 0 0 2 3 3 2 5 0 3.2 Real and complex symmetric bilinear forms Throughout

More information

Inverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1

Inverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1 Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is

More information

3 Gramians and Balanced Realizations

3 Gramians and Balanced Realizations 3 Gramians and Balanced Realizations In this lecture, we use an optimization approach to find suitable realizations for truncation and singular perturbation of G. It turns out that the recommended realizations

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

If Y and Y 0 satisfy (1-2), then Y = Y 0 a.s.

If Y and Y 0 satisfy (1-2), then Y = Y 0 a.s. 20 6. CONDITIONAL EXPECTATION Having discussed at length the limit theory for sums of independent random variables we will now move on to deal with dependent random variables. An important tool in this

More information

AN EFFECTIVE METRIC ON C(H, K) WITH NORMAL STRUCTURE. Mona Nabiei (Received 23 June, 2015)

AN EFFECTIVE METRIC ON C(H, K) WITH NORMAL STRUCTURE. Mona Nabiei (Received 23 June, 2015) NEW ZEALAND JOURNAL OF MATHEMATICS Volume 46 (2016), 53-64 AN EFFECTIVE METRIC ON C(H, K) WITH NORMAL STRUCTURE Mona Nabiei (Received 23 June, 2015) Abstract. This study first defines a new metric with

More information

A Spectral Characterization of Closed Range Operators 1

A Spectral Characterization of Closed Range Operators 1 A Spectral Characterization of Closed Range Operators 1 M.THAMBAN NAIR (IIT Madras) 1 Closed Range Operators Operator equations of the form T x = y, where T : X Y is a linear operator between normed linear

More information

7. Symmetric Matrices and Quadratic Forms

7. Symmetric Matrices and Quadratic Forms Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

More information

Math 209B Homework 2

Math 209B Homework 2 Math 29B Homework 2 Edward Burkard Note: All vector spaces are over the field F = R or C 4.6. Two Compactness Theorems. 4. Point Set Topology Exercise 6 The product of countably many sequentally compact

More information

CHAPTER X THE SPECTRAL THEOREM OF GELFAND

CHAPTER X THE SPECTRAL THEOREM OF GELFAND CHAPTER X THE SPECTRAL THEOREM OF GELFAND DEFINITION A Banach algebra is a complex Banach space A on which there is defined an associative multiplication for which: (1) x (y + z) = x y + x z and (y + z)

More information

Matrices A brief introduction

Matrices A brief introduction Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino September 2013 Basilio Bona (DAUIN) Matrices September 2013 1 / 74 Definitions Definition A matrix is a set of N real or complex numbers

More information

A Concise Course on Stochastic Partial Differential Equations

A Concise Course on Stochastic Partial Differential Equations A Concise Course on Stochastic Partial Differential Equations Michael Röckner Reference: C. Prevot, M. Röckner: Springer LN in Math. 1905, Berlin (2007) And see the references therein for the original

More information

1.4 The Jacobian of a map

1.4 The Jacobian of a map 1.4 The Jacobian of a map Derivative of a differentiable map Let F : M n N m be a differentiable map between two C 1 manifolds. Given a point p M we define the derivative of F at p by df p df (p) : T p

More information

x n x or x = T -limx n, if every open neighbourhood U of x contains x n for all but finitely many values of n

x n x or x = T -limx n, if every open neighbourhood U of x contains x n for all but finitely many values of n Note: This document was written by Prof. Richard Haydon, a previous lecturer for this course. 0. Preliminaries This preliminary chapter is an attempt to bring together important results from earlier courses

More information

CHAPTER VIII HILBERT SPACES

CHAPTER VIII HILBERT SPACES CHAPTER VIII HILBERT SPACES DEFINITION Let X and Y be two complex vector spaces. A map T : X Y is called a conjugate-linear transformation if it is a reallinear transformation from X into Y, and if T (λx)

More information

Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space. Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

More information

Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016

Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016 Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016 1. Let V be a vector space. A linear transformation P : V V is called a projection if it is idempotent. That

More information

Real Analysis Notes. Thomas Goller

Real Analysis Notes. Thomas Goller Real Analysis Notes Thomas Goller September 4, 2011 Contents 1 Abstract Measure Spaces 2 1.1 Basic Definitions........................... 2 1.2 Measurable Functions........................ 2 1.3 Integration..............................

More information

Real Analysis, 2nd Edition, G.B.Folland Elements of Functional Analysis

Real Analysis, 2nd Edition, G.B.Folland Elements of Functional Analysis Real Analysis, 2nd Edition, G.B.Folland Chapter 5 Elements of Functional Analysis Yung-Hsiang Huang 5.1 Normed Vector Spaces 1. Note for any x, y X and a, b K, x+y x + y and by ax b y x + b a x. 2. It

More information

Matrices A brief introduction

Matrices A brief introduction Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex

More information

Problem Set 6: Solutions Math 201A: Fall a n x n,

Problem Set 6: Solutions Math 201A: Fall a n x n, Problem Set 6: Solutions Math 201A: Fall 2016 Problem 1. Is (x n ) n=0 a Schauder basis of C([0, 1])? No. If f(x) = a n x n, n=0 where the series converges uniformly on [0, 1], then f has a power series

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

RIESZ BASES AND UNCONDITIONAL BASES

RIESZ BASES AND UNCONDITIONAL BASES In this paper we give a brief introduction to adjoint operators on Hilbert spaces and a characterization of the dual space of a Hilbert space. We then introduce the notion of a Riesz basis and give some

More information

Large Scale Data Analysis Using Deep Learning

Large Scale Data Analysis Using Deep Learning Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset

More information

4.6. Linear Operators on Hilbert Spaces

4.6. Linear Operators on Hilbert Spaces 4.6. Linear Operators on Hilbert Spaces 1 4.6. Linear Operators on Hilbert Spaces Note. This section explores a number of different kinds of bounded linear transformations (or, equivalently, operators

More information

Lecture 3: Linear Algebra Review, Part II

Lecture 3: Linear Algebra Review, Part II Lecture 3: Linear Algebra Review, Part II Brian Borchers January 4, Linear Independence Definition The vectors v, v,..., v n are linearly independent if the system of equations c v + c v +...+ c n v n

More information

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear

More information

DS-GA 1002 Lecture notes 10 November 23, Linear models

DS-GA 1002 Lecture notes 10 November 23, Linear models DS-GA 2 Lecture notes November 23, 2 Linear functions Linear models A linear model encodes the assumption that two quantities are linearly related. Mathematically, this is characterized using linear functions.

More information

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability... Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................

More information

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly.

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly. C PROPERTIES OF MATRICES 697 to whether the permutation i 1 i 2 i N is even or odd, respectively Note that I =1 Thus, for a 2 2 matrix, the determinant takes the form A = a 11 a 12 = a a 21 a 11 a 22 a

More information

This property turns out to be a general property of eigenvectors of a symmetric A that correspond to distinct eigenvalues as we shall see later.

This property turns out to be a general property of eigenvectors of a symmetric A that correspond to distinct eigenvalues as we shall see later. 34 To obtain an eigenvector x 2 0 2 for l 2 = 0, define: B 2 A - l 2 I 2 = È 1, 1, 1 Î 1-0 È 1, 0, 0 Î 1 = È 1, 1, 1 Î 1. To transform B 2 into an upper triangular matrix, subtract the first row of B 2

More information

Lecture II: Linear Algebra Revisited

Lecture II: Linear Algebra Revisited Lecture II: Linear Algebra Revisited Overview Vector spaces, Hilbert & Banach Spaces, etrics & Norms atrices, Eigenvalues, Orthogonal Transformations, Singular Values Operators, Operator Norms, Function

More information

MAT 578 FUNCTIONAL ANALYSIS EXERCISES

MAT 578 FUNCTIONAL ANALYSIS EXERCISES MAT 578 FUNCTIONAL ANALYSIS EXERCISES JOHN QUIGG Exercise 1. Prove that if A is bounded in a topological vector space, then for every neighborhood V of 0 there exists c > 0 such that tv A for all t > c.

More information

Weak Topologies, Reflexivity, Adjoint operators

Weak Topologies, Reflexivity, Adjoint operators Chapter 2 Weak Topologies, Reflexivity, Adjoint operators 2.1 Topological vector spaces and locally convex spaces Definition 2.1.1. [Topological Vector Spaces and Locally convex Spaces] Let E be a vector

More information

2. Review of Linear Algebra

2. Review of Linear Algebra 2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

More information

Homework If the inverse T 1 of a closed linear operator exists, show that T 1 is a closed linear operator.

Homework If the inverse T 1 of a closed linear operator exists, show that T 1 is a closed linear operator. Homework 3 1 If the inverse T 1 of a closed linear operator exists, show that T 1 is a closed linear operator Solution: Assuming that the inverse of T were defined, then we will have to have that D(T 1

More information

University of Luxembourg. Master in Mathematics. Student project. Compressed sensing. Supervisor: Prof. I. Nourdin. Author: Lucien May

University of Luxembourg. Master in Mathematics. Student project. Compressed sensing. Supervisor: Prof. I. Nourdin. Author: Lucien May University of Luxembourg Master in Mathematics Student project Compressed sensing Author: Lucien May Supervisor: Prof. I. Nourdin Winter semester 2014 1 Introduction Let us consider an s-sparse vector

More information

4 Linear operators and linear functionals

4 Linear operators and linear functionals 4 Linear operators and linear functionals The next section is devoted to studying linear operators between normed spaces. Definition 4.1. Let V and W be normed spaces over a field F. We say that T : V

More information

A Review of Linear Algebra

A Review of Linear Algebra A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose

More information

Examples include: (a) the Lorenz system for climate and weather modeling (b) the Hodgkin-Huxley system for neuron modeling

Examples include: (a) the Lorenz system for climate and weather modeling (b) the Hodgkin-Huxley system for neuron modeling 1 Introduction Many natural processes can be viewed as dynamical systems, where the system is represented by a set of state variables and its evolution governed by a set of differential equations. Examples

More information

Compact operators on Banach spaces

Compact operators on Banach spaces Compact operators on Banach spaces Jordan Bell jordan.bell@gmail.com Department of Mathematics, University of Toronto November 12, 2017 1 Introduction In this note I prove several things about compact

More information

Notes for Functional Analysis

Notes for Functional Analysis Notes for Functional Analysis Wang Zuoqin (typed by Xiyu Zhai) September 29, 2015 1 Lecture 09 1.1 Equicontinuity First let s recall the conception of equicontinuity for family of functions that we learned

More information

Quantum Mechanics for Scientists and Engineers. David Miller

Quantum Mechanics for Scientists and Engineers. David Miller Quantum Mechanics for Scientists and Engineers David Miller Vector spaces, operators and matrices Vector spaces, operators and matrices Vector space Vector space We need a space in which our vectors exist

More information

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces. Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

More information

Linear algebra review

Linear algebra review EE263 Autumn 2015 S. Boyd and S. Lall Linear algebra review vector space, subspaces independence, basis, dimension nullspace and range left and right invertibility 1 Vector spaces a vector space or linear

More information

Inequalities in Hilbert Spaces

Inequalities in Hilbert Spaces Inequalities in Hilbert Spaces Jan Wigestrand Master of Science in Mathematics Submission date: March 8 Supervisor: Eugenia Malinnikova, MATH Norwegian University of Science and Technology Department of

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Multiplication Operators with Closed Range in Operator Algebras

Multiplication Operators with Closed Range in Operator Algebras J. Ana. Num. Theor. 1, No. 1, 1-5 (2013) 1 Journal of Analysis & Number Theory An International Journal Multiplication Operators with Closed Range in Operator Algebras P. Sam Johnson Department of Mathematical

More information

Spectral Theory, with an Introduction to Operator Means. William L. Green

Spectral Theory, with an Introduction to Operator Means. William L. Green Spectral Theory, with an Introduction to Operator Means William L. Green January 30, 2008 Contents Introduction............................... 1 Hilbert Space.............................. 4 Linear Maps

More information

Second Order Elliptic PDE

Second Order Elliptic PDE Second Order Elliptic PDE T. Muthukumar tmk@iitk.ac.in December 16, 2014 Contents 1 A Quick Introduction to PDE 1 2 Classification of Second Order PDE 3 3 Linear Second Order Elliptic Operators 4 4 Periodic

More information

FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM. F (m 2 ) + α m 2 + x 0

FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM. F (m 2 ) + α m 2 + x 0 FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM If M is a linear subspace of a normal linear space X and if F is a bounded linear functional on M then F can be extended to M + [x 0 ] without changing its norm.

More information

Exercise Solutions to Functional Analysis

Exercise Solutions to Functional Analysis Exercise Solutions to Functional Analysis Note: References refer to M. Schechter, Principles of Functional Analysis Exersize that. Let φ,..., φ n be an orthonormal set in a Hilbert space H. Show n f n

More information

Chapter 1. Matrix Algebra

Chapter 1. Matrix Algebra ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface

More information

The best generalised inverse of the linear operator in normed linear space

The best generalised inverse of the linear operator in normed linear space Linear Algebra and its Applications 420 (2007) 9 19 www.elsevier.com/locate/laa The best generalised inverse of the linear operator in normed linear space Ping Liu, Yu-wen Wang School of Mathematics and

More information

LECTURE 7. k=1 (, v k)u k. Moreover r

LECTURE 7. k=1 (, v k)u k. Moreover r LECTURE 7 Finite rank operators Definition. T is said to be of rank r (r < ) if dim T(H) = r. The class of operators of rank r is denoted by K r and K := r K r. Theorem 1. T K r iff T K r. Proof. Let T

More information

Elements of Linear Algebra, Topology, and Calculus

Elements of Linear Algebra, Topology, and Calculus Appendix A Elements of Linear Algebra, Topology, and Calculus A.1 LINEAR ALGEBRA We follow the usual conventions of matrix computations. R n p is the set of all n p real matrices (m rows and p columns).

More information

Lecture 2: Linear Algebra Review

Lecture 2: Linear Algebra Review EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1

More information

May 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions

May 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions May 9, 24 MATH 48 MIDTERM EXAM OUTLINE This exam will consist of two parts and each part will have multipart questions. Each of the 6 questions is worth 5 points for a total of points. The two part of

More information

Stabilization of Distributed Parameter Systems by State Feedback with Positivity Constraints

Stabilization of Distributed Parameter Systems by State Feedback with Positivity Constraints Stabilization of Distributed Parameter Systems by State Feedback with Positivity Constraints Joseph Winkin Namur Center of Complex Systems (naxys) and Dept. of Mathematics, University of Namur, Belgium

More information

Synopsis of Numerical Linear Algebra

Synopsis of Numerical Linear Algebra Synopsis of Numerical Linear Algebra Eric de Sturler Department of Mathematics, Virginia Tech sturler@vt.edu http://www.math.vt.edu/people/sturler Iterative Methods for Linear Systems: Basics to Research

More information

Chapter 1. Introduction

Chapter 1. Introduction Chapter 1 Introduction Functional analysis can be seen as a natural extension of the real analysis to more general spaces. As an example we can think at the Heine - Borel theorem (closed and bounded is

More information

NOTES ON FRAMES. Damir Bakić University of Zagreb. June 6, 2017

NOTES ON FRAMES. Damir Bakić University of Zagreb. June 6, 2017 NOTES ON FRAMES Damir Bakić University of Zagreb June 6, 017 Contents 1 Unconditional convergence, Riesz bases, and Bessel sequences 1 1.1 Unconditional convergence of series in Banach spaces...............

More information

Chapter 2 Subspaces of R n and Their Dimensions

Chapter 2 Subspaces of R n and Their Dimensions Chapter 2 Subspaces of R n and Their Dimensions Vector Space R n. R n Definition.. The vector space R n is a set of all n-tuples (called vectors) x x 2 x =., where x, x 2,, x n are real numbers, together

More information

Transpose & Dot Product

Transpose & Dot Product Transpose & Dot Product Def: The transpose of an m n matrix A is the n m matrix A T whose columns are the rows of A. So: The columns of A T are the rows of A. The rows of A T are the columns of A. Example:

More information

TOPIC III LINEAR ALGEBRA

TOPIC III LINEAR ALGEBRA [1] Linear Equations TOPIC III LINEAR ALGEBRA (1) Case of Two Endogenous Variables 1) Linear vs. Nonlinear Equations Linear equation: ax + by = c, where a, b and c are constants. 2 Nonlinear equation:

More information

MATH2210 Notebook 2 Spring 2018

MATH2210 Notebook 2 Spring 2018 MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................

More information

Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

More information

TD 1: Hilbert Spaces and Applications

TD 1: Hilbert Spaces and Applications Université Paris-Dauphine Functional Analysis and PDEs Master MMD-MA 2017/2018 Generalities TD 1: Hilbert Spaces and Applications Exercise 1 (Generalized Parallelogram law). Let (H,, ) be a Hilbert space.

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Intertibility and spectrum of the multiplication operator on the space of square-summable sequences

Intertibility and spectrum of the multiplication operator on the space of square-summable sequences Intertibility and spectrum of the multiplication operator on the space of square-summable sequences Objectives Establish an invertibility criterion and calculate the spectrum of the multiplication operator

More information

Solutions: Problem Set 3 Math 201B, Winter 2007

Solutions: Problem Set 3 Math 201B, Winter 2007 Solutions: Problem Set 3 Math 201B, Winter 2007 Problem 1. Prove that an infinite-dimensional Hilbert space is a separable metric space if and only if it has a countable orthonormal basis. Solution. If

More information