Real Numbers The least upper boud - Let B be ay subset of R B is bouded above if there is a k R such that x k for all x B - A real umber, k R is a uique least upper boud of B, ie k = LUB(B), if () k is a upper boud of B (2) For every y < k, y is ot a upper boud of B - LUB axiom says that every oempty subset of R that is bouded above has a least upper boud - LUB(B) may or may ot belog to B (Ex; B= { y: y = x, x + } - Note that A B LUB( A) LUB( B) R ) The greatest lower boud - Let B be ay subset of R If B is bouded below, the greatest lower boud, GLB(B) is similarly defied Supremum ad ifimum - For ay subset B of R, the supremum is defied as LUB( B), B ad bouded above sup B: = +, B ad ot bouded above, B = - For ay subset C of R, the ifimum is defied as GLB( C), C ad bouded below if C: =, C ad ot bouded below +, C = Bolzao-Weierstrass theorem - If x is a bouded sequece of real umbers, ie < a x b<+, the there is a covergig subsequece, x whose limit lies i [a, b] k - - BME, KHU
Vector Space Field - A field is a set F o which two operatios of additio ad multiplicatio are defied with the usual properties - A ordered field is a field F with a relatio < - Example: ratioal umbers, real umbers, complex umbers Vector space ad subspace - A oempty set V is a vector space over a field F if the followig properties hold: There is a operatio called vector additio, + such that () Closure: uv, V, u+ v V (2) Commutative law: uv, V, u+ v= v+ u (3) Associative law: uvw,, V,( u+ v) + w= u+ ( v+ w ) (4) Additive idetity: 0 V u V, u+ 0= u (5) Additive iverse: u V, ( u) u+ ( u) = 0 ad ( u ) is uique There is a operatio called scalar multiplicatio such that () Closure: a Fad u V, au V (2) Associative law: ab, Fad u V, ab ( u) = ( ab) u (3) First distributive law: a Fad uv, V, a( u+ v) = au+ av (4) Secod distributive law y: ab, Fad u V, ( a+ b) u= au+ bu (5) Multiplicative idetity of F: u V,u= u - A subset W of a vector space V over F is a subspace of V iff a Fad uv, W, au+ v W W itself is a vector space Spa, liear idepedece, ad basis - Let V be a vector space over a field F Suppose G V ad G may ot be a subspace ad may ot be a fiite set The set of all liear combiatios of elemets of G is deoted by spa G, ie, spa G := akvk : is ay positive iteger, v k G, ad ak F k= Note that () G spa G (2) spa G is a subspace of V (3) If a subspace W cotais G, the W cotais spa G - For a arbitrary subset G of V, G is liearly idepedet if - 2 - BME, KHU
v G, a v = 0 implies a = a = = a = 0 k k k 2 k= If G is ot liearly idepedet, G is liearly depedet Note that if liearly depedet 0 G, G is - If { } v are liaerly idepedet, o vector v k ca be expressed as a liear k k= combiatio of other vectors i the set - Let W be a subspace of V If there exists a fiite subset G W, such that spa G = W, the W is fiite-dimesioal If spa G = W ad G is liearly idepedet, G is a basis for W - If G ={ } v is a basis for W, k k= x= akvk x W ad { a k} k= k= is uique - If W is fiite-dimesioal, the ay basis of W cotais the same umber, of liearly idepedet vectors We say that is the dimesio of W (ie dimw = ) If dimw = ad { t, t,, t} W are liearly idepedet, the { t t t } 2 spa, 2,, = W - 3 - BME, KHU
Mappig Fuctio ad mappig - A fuctio is a triple (X, Y, f), also deoted by f: X Y, where X ad Y are specified sets of iputs ad outputs, respectively - f is a rule or mappig that associates to each x X, a uique elemet f(x) Y - The set X is the domai of f ad the set Y is the co-domai of f - The rage of f is the set { f ( x): x X} - Two fuctios ( X, Y, f ) ad ( X2, Y2, f 2) are equal iff X = X, Y = Y, ad f ( x) = f ( x) x X = X 2 2 2 2 Vector space of mappigs Let V be a vector space over F ad U be a arbitrary set - x: U V is a mappig if there is a rule that assigs to each u U, a elemet x(u) V - We let X be the set of all mappigs from U ito V Two mappigs, x ad y i X are equal iff x( u) = y( u) u U - X is itself a vector space with the followig defiitios () Additio of mappigs is defied as ( x + y)( u) : = x( u) + y( u) x, y X ad u U, (2) Additive idetity, z( u): = 0 u U, (3) Additive iverse, ( x)( u): = x( u) u U, (4) Scalar multiplicatio, ( ax)( u) : = a x( u) u U ad a F Liear fuctioal - Let V be a vector spave over F A mappig β: V F is called a liear fuctioal if β( av + v ) = aβ( v ) + β( v ), a F, v, v V 2 2 2 - Give a set of vectors, { t t t },,, V, if there exists a set of liear fuctioals, 2 { β β β },,, such that 2 idepedet, if i = j β j( i) = δij = 0, if i j t, the { t t t },,, is liearly 2-4 - BME, KHU
Metric Space Metric space - Let X be a oempty set ad defie a mappig ρ: X X [0, ) with the follwoig properties: () ρ(x, y) 0 ad ρ(x, y) = 0 iff x = y (2) ρ(x, y) = ρ(y, x) (3) ρ(x, z) ρ(x, y) + ρ(y, z) The, ρ is called a metric The pair (X, ρ) or X is a metric space - We defie a ball as { ρ } B( xr, ) = B( x): = y X: ( xy, ) < r, for some x X r INSERT TOPOLOGY Covergece - A sequece x X coverges to x X if ε > 0, ρ( x, x) < ε for all sufficietly large (ie, there exists a iteger N such that the coditio holds for all > N) We deote this as x x or lim x = x - A sequece x large X coverges to x X if x B( x, ε), ε > 0 for all sufficietly - A set E i a metric space is closed iff every covergig sequece of poits i E coverges to a poit i E - (Approximatio) If x E, there is a sequece x E ad x x I orther words, if x E, the there is a poit y E such that ρ(x, y) < ε for ay ε > 0 Subsequece - Let, 2, be itegers such that k as k - If x X is a sequece, x is a subsequece of k x Sequetial compactess - 5 - BME, KHU
- A subset D is sequetially compact if for every sequece x D, there is a covergig subsequece x whose limit lies i D k - From Bolzao-Weierstrass, [a, b] with < a< b< is sequetially compact - Sequetially compact subset of a metric space must be closed Cauchy sequece - A sequece x i a metric space is Cauchy if ρ( x, xm) < ε, ε > 0 ad for all sufficietly large ad m - I a Cauchy sequece, all the poits i the tail of the sequece are close together - Every covergig sequece is Cauchy The coverse is ot true - A Cauchy sequece is bouded Complete space - If every Cauchy sequece of a metric space coverges to a poit i the space, the space is complete - If x is a Cuachy sequece i a metric space, ad if x is a covergig subsequece k of x, the x coverges to the same limit as x k - The real umbers with the metric ρ ( x, y) = x y is a complete metric space - The space d is complete uder the usual Euclidia distace, ie d i= () i () i 2 ρ( xy, ) = x y - Ay closed ad bouded subset of d is sequetially compact - The spaces of complex umbers ad d are complete Ay closed ad bouded subset of d is sequetially compact Cotiuity - Let (X, ρ) ad (Y, m) be metric spaces Let f: X Y be a fuctio - (Cotiuity of a poit) A fuctio f is cotiuous at a poit x 0 if ε > 0, δ = δ( x, ε) x X, ρ( x, x ) < δ m( f( x), f( x )) < ε, or 0 0 0 ε > 0, δ = δ( x, ε) x X, x B ( x, δ) f( x) B ( f( x ), ε) 0 ρ 0 m 0-6 - BME, KHU
- (Cotiuity o a set) A fuctio f is cotiuous o a subset D X if f is cotiuous at each poit x 0 D - A fuctio f is cotiuous at a poit x 0 for every sequece x x0, f( x ) f( x 0) I order words, f is covergece preservig iff f is cotiuous - (Uiform cotiuity) A fuctio f is uiformly cotiuous o a subset D X if ε > 0, δ = δ( ε) > 0 xx, D, ρ( xx, ) < δ m( f( x), f( x )) < ε 0 0 0 Compact sets - 7 - BME, KHU
Topology Let X be a metric space with a metric ρ Ball - A ball is defied as { ρ } B( xr, ) = B( x): = y X: ( xy, ) < r, x X r Ope set - A set U X is ope if x U, ε > 0 with B( x, ε) U - A set U X is ot ope if ε > 0, x U with B( x, ε) U - The whole space X ad are both ope - The set B(x, r) is ope, ie it is a ope ball Closed set c - A set F X is closed if its complemet F : { x X : x F} - X,, ad B( xr, ) c { y X: ρ( xy, ) r} = is ope = are all closed sets - Every (possibly ifiite) uio of ope sets is a ope set - Every itersectio of fiite umber of ope sets is a ope set Topological space - Let X be a oempty set ad I be a collectio of subsets of X I is called a topology for X if () I ad X I (2) If U α I, the α U α I (3) If U I ad U 2 I, the U U 2 I - The pair (X, I ) or X is called a topological space - The elemets of I are ope sets - A set F is closed if F c I Properties of topological space - A set U is ope for every x U, there is a ope set cotaiig x, say O x, with O x U - 8 - BME, KHU
- The closure of a set E is E : = C ad E E E is the smallest closed set CE : C ad C is closed cotaiig E - A set E is closed E = E - A poit x is a accumulatio poit (or cluster poit or limit poit) of a set E if for every ope set cotaiig x, say O x, there is a poit y x with y O x E We let E' deote the set of accumulatio poits of E The poit x may or may ot be i E - E is closed E' E ' - E = E E c - The boudary of E is E ad E : = E E - The iterior of E is o o c E E = E E = E o c E ad E : ( E ) c = o E is a ope set with E o E ad - 9 - BME, KHU
Normed Vector Space Let F deote or ad V be a vector space over F Norm - is a orm if () 0 v <, v V ad v = 0 iff v = 0, (2) av = a v, v V, a F, ad (3) v+ w v + w, v, w V (triagular iequality) - Every ormed vector space is a metric space with ρ (, vw) = v w - A sequece v coverges to v (ie, v v) iff v v 0 - v w v w v + w Baach space - A complete ormed vector space is called Baach space Examples of orm - The p-orm o V = or Let v = ( v, v2,, v ), the p p vk, p< k= v : = p max vk, p = k - Whe p = 2, we call it Euclidea orm - The uiform orm Let U be ay set ad let F = or Let X deote the vector space of mappigs from U ito F Let X b deote the set of bouded mappigs, ie Xb : = x X : sup x( u) < u U Note that if U is a fiite set, the X = X b The uiform orm of x X b is x : = sup xu ( ) X b with the uiform orm is a Baach u U - 0 - BME, KHU
space The p spaces - Let U = {, 2, 3, } For k U, we write x k istead of x(k) The, X deotes the set - of all real- or complex-valued sequeces For p <, let p p : = x X : xk <, k= ad set : = { x X : sup x k < } k p spaces is equipped with the correspodig p-orm Projectios - Let V be a ormed vector space ad G be a subset of V If there exists a vector v ˆ G such that v vˆ v w, w G, v V, the ˆv is a projectio of v oto G - A projectio may ot exist (for example, if G is ope) ad may ot be uique (for example, if G is ot covex) - Projectios exist whe G is a closed ball i a arbitrary, possibly ifiite-dimesioal, ormed vector space Fiite-dimesioal subspaces - Let W be a fiite-dimesioal ormed vector space or a fiite-dimesioal subspace of a ormed vector space W may be a subspace of a larger ifiite-dimesioal space V The, () W is complete, ie, W is a Baach space (2) Every closed ad bouded subset G of W is (sequetially) compact Projectios oto closed fiite-dimesioal subsets - If G is a oempty closed ad bouded subset of a fiite-dimesioal subspace W of a larger ormed vector space V, the the projectio of every v V oto G always exists - If W is a fiite-dimesioal subspace of a larger ormed vector space V, the the projectio of ay v V oto W always exists - - BME, KHU
Ier Product Spaces Let F deote or ad V be a vector space over F For a, a deotes the complex cojugate of a Ier product space (pre-hilbert space) -, is a ier product o V if the followig properties hold: () 0 vv, <, v V ad vv, = 0 iff v= 0, (2) vw, = wv,, vw, V (3) au+ bv, w = a u, w + b v, w, a, b F, u, v, w V - vw, is i geeral complex umber but vv, is always real - w, au+ bv = a w, u + b w, v - v0, = 0 If vw, = 0, w V, the v = 0 Hilbert space - A complete ier product space is Hilbert space Norm o a ier product space - Give ay ier product, 2 v : = v, v defies a orm o V Parallogram equality - u+ v 2 + u v 2 = 2( u 2 + v 2 ) Cauchy-Schwarz iequality - uv, u v - If v 0, the equality holds iff u = av for some a F - 2 - BME, KHU
- Agle betwee u ad v, θ = ( uv, ) = cos uv, u v ad uv, = u v cosθ (a) θ = 0 u ad v are aliged uv, = u v, v = αu for some α 0 (b) θ = π u ad v are opposed uv, = u v, v = αu for some α < 0 (c) θ =± π 2 u ad v are orthogoal uv, = 0, v u Orthogoality - A collectio of vectors G is (mutually) orthogoal if uv, = 0, uv, G with u v - If, i additio, u =, u G, the they are orthoormal - Orthoormal set of vectors are liearly idepedet The coverse may ot be true Some idetities - (Parallelogram law) I ay ier product space, + 2 + 2 = 2( 2 + 2 ) - (Polarizatio idetity) I a complex ier product space, 2 2 2 2 4 uv, = u+ v u v + j u+ jv j u jv u v u v u v The orthogoality priciple (OP) - Let V be a ier product space Let W be a subspace of V Fix ay v V The, a vector v W has the property that v v v w, w W iff v v, w = 0, w W Furthermore, there is at most oe elemet v W satisfyig the coditio - If v W exists, it is uique But it may ot exist - If v W exists, the v is the orthogoal projectio of v oto W - Note that () 2 2 2 v = v v + v - 3 - BME, KHU
(2) 2 2 2 v v = v v (3) v v Projectios oto fiite-dimesioal spaces - Let V be a ier product space Let W be a fiite-dimesioal subspace of V The, { w, w,, w } spa { w, w,, w } = ad OP is as follows 2 2 W v v v w, w W iff v v, wi = 0, i=,2,, - If v exists, - Note that () v = c jw j (ie v W ) j= vw, = w, w c, i =,2,,, or equivaletly (2) = i j i j j= Ac b where A : = w, w, b: = v, w,, v, w T, c: = [ c,, c ] Ad, A is osigular if { w w w } ij j i,,, is liearly idepedet 2 T - If { w w w },,, is orthoormal, the A = I ad c i = vw, 2 i, ad thus v = v, w j w j ad j= v 2 = j= v, w - Bessel's iequality for a orthoomal basis is - Sice v = v iff v W, 2 v = v, w, v W j= j j 2 2 2 2 vw, j v < j= Orthogoal complemet - For ay subset W of a ier product space V, we defie the orthogoal complemet of W as { v w v w } W : = V :, = 0, W - W is a subspace of V - W is a closed set - 4 - BME, KHU
- W ( W ) If W is a closed subspace of a Hilbert space, ( ) - If W is a arbitrary subset of a Hilbert space, ( W ) spa = W W = W Covex set - Let X be a arbitrary vector space over or A subset C X is covex if λx+ ( λ) y C, x, y C, λ [0, ] - I a ormed vector space, ope balls are covex - A subspace is a covex set Projectio theorem - Let C be a closed, covex subset of a Hilbert space X The, for every x X, there exists the uique x C such that x x x y, y C - If M is a closed subspace of a Hilbert space X, the x= x + ( x x ), x X where x M ad x x M Sums ad direct sums of subspaces - If U ad W are two subspaces of a vector space V, their sum is { } U + W : = u+ w: u U ad w W - If every elemet i U + W has a uique represetatio, their sum becomes the direct sum as U W - U W U W iff U W { } + = = 0 - If M is a closed subspace of a Hilber space X, the X = M M - 5 - BME, KHU