Outline. Linear maps. 1 Vector addition is commutative: summands can be swapped: 2 addition is associative: grouping of summands is irrelevant:

Transcription

1 Outline Wiskunde : Vector Spaces and Linear Maps B Jacobs Institute for Computing and Information Sciences Digital Security Version: spring 0 B Jacobs Version: spring 0 Wiskunde / 55 Points in plane The set of points in a plane is usually written as R {x, y x, y R} or as R x { x, y R} y Two points can be added, as in: x, y + x, y x + x, y + y What is this geometrically? Also, points can be multiplied by a number scalar : a x, y a x, a y Several nice properties hold, like: a x, y + x, y a x, y + a x, y B Jacobs Version: spring 0 Wiskunde / 55 Points in space Points in 3-dimensional are described as: x R 3 {x, y, z x, y, z R} or as R 3 { y x, y, z R} z Again such 3-dimensional points can be added and multiplied: x, y, z + x, y, z x + x, y + y, z + z a x, y, z a x, a y, a z And similar nice properties hold Mathematicians like to capture such similarities in an abstract definition sometimes the definition is so abstract one gets lost but then it is good to keep the main examples in mind B Jacobs Version: spring 0 Wiskunde 4 / 55 Vector space A vector space consists of a set V, whose elements are called vectors can be added can be multiplied with a real number The precise requirements are in the next two slides For each n N, n-dimensional space R n is a vector space, where R n {x, x,, x n x,, x n R} This includes points in a plane n and points in space n 3 B Jacobs Version: spring 0 Wiskunde 6 / 55 B Jacobs Version: spring 0 Wiskunde 5 / 55 Addition for vectors: precise requirements Vector addition is commutative: summands can be swapped: v + w w + v addition is associative: grouping of summands is irrelevant: u + v + w u + v + w 3 there is a zero vector 0 such that: v + 0 v, and hence by also: 0 + v v 4 each vector v has an additive inverse minus v such that: v + v 0 B Jacobs Version: spring 0 Wiskunde 7 / 55

2 Scalar multiplication for vectors: precise requirements Base in space R is unit for scalar multiplication: v v two scalar multiplications can be done as one: 3 distributivity a b v }{{} twice scalar mult ab v }{{} mult in R a v + w a v + a w a + b v a v + b v Exercise Check for yourself that all these properties hold for R n B Jacobs Version: spring 0 Wiskunde 8 / 55 Independence Vectors v,, v n in a vector space V are called independent if for all scalars a,, a n R one has: a v + + a n v n 0 in V implies a a a n 0 The 3 vectors, 0, 0, 0,, 0, 0, 0, R 3 are independent, since if a, 0, 0 + a 0,, 0 + a 3 0, 0, 0, 0, 0 then, using the computation from the previous slide, a, a, a 3 0, 0, 0, so that a a a 3 0 B Jacobs Version: spring 0 Wiskunde / 55 Proving independence via equation solving II In R 3 we can distinguish three special vectors:, 0, 0 R 3 0,, 0 R 3 0, 0, R 3 These vectors form a basis: each vector x, y, z can be expressed in terms of these three special vectors: x, 0, 0 + y 0,, 0 + z 0, 0, x, 0, 0 + 0, y, 0 + 0, 0, z x, y, z Moreover, these three special vectors are independent It is a challenge to capture this intuitive notion of basis abstractly in terms of the notion of vector space it can then be used for all examples of vector spaces B Jacobs Version: spring 0 Wiskunde 0 / 55 Proving independence via equation solving I 0 Investigate independence of,, and Thus we ask: are there any non-zero a, a, a 3 R with: 0 0 a + a + a If there is a non-zero solution, the vectors are dependent, and if a a a 3 0 is the only solution, they are independent B Jacobs Version: spring 0 Wiskunde / 55 Proving independence via equation solving III Our question involves the systems of equations / matrix: a + a 0 0 a a + 5a 3 0 corresponding to 5 3a + 4a + a This one has non-zero solutions a, a, a 3 compute and check for yourself! Thus the original vectors are dependent Explicitly: Same independence question for:,, 3 With corresponding matrix: 5 0 reducing to Thus the only solution is a a a 3 0 The vectors are independent! B Jacobs Version: spring 0 Wiskunde 3 / 55 B Jacobs Version: spring 0 Wiskunde 4 / 55

3 Non-independence / dependence Basis In the plane two vectors v, w R are dependent if and only if: they are on the same line ie v a w, for some scalar a : for v, and w, 4 we have: v w, so they are on the same line a v + a w 0, eg for a 0 and a 0 In space, three vectors u, v, w R 3 are dependent if they are in the same plane or even line In general, if v,, v n V are dependent, if and only if each v i can be expressed as linear combination of the others the v j with j i B Jacobs Version: spring 0 Wiskunde 5 / 55 The standard basis for R n Vectors v,, v n V form a basis for a vector space V if these v,, v n are independent, and span V in the sense that each w V can be written as linear combination of these v,, v n, namely as: w a v + + a n v n for certain a,, a n R These scalars a i are uniquely determined by w V see below a space V may have several bases, but the number of elements of such a basis for V is always the same; it is called the dimension of V, sometimes written as dimv N B Jacobs Version: spring 0 Wiskunde 6 / 55 An alternative basis for R For the space R n {x,, x n x i R} there is a standard choice of base vectors:, 0, 0, 0, 0,, 0,, 0, 0,, 0, We have already seen that they are independent; it is easy to see that they span R n This enables us to state precisely that R n has n dimensions The standard basis for R is, 0, 0, But many other choices are possible, eg,,, independence: if a, + b, 0, 0, then: { { a + b 0 a 0 and thus a b 0 b 0 spanning: each point x, y can written in terms of,,,, namely: x, y x+y x y, +, B Jacobs Version: spring 0 Wiskunde 7 / 55 Uniqueness of representations Lemma Suppose V is a vector space, with basis v,, v n assume x V can be represented in two ways: x a v + + a n v n and also x b v + + b n v n Then: a b and and a n b n Proof: This follows from independence of v,, v n since: 0 x x a v + + a n v n b v + + b n v n a b v + + a n b n v n Hence a i b i 0, by independence, and thus a i b i B Jacobs Version: spring 0 Wiskunde 8 / 55 Functions A function f is an operation that sends elements of one set X to another set Y in that case we write f : X Y or sometimes X f Y this f sends x X to f x Y X is called the domain and Y the codomain of the function f f n n can be seen as function N Q, that is from the natural numbers N to the rational numbers Q A function is sometimes also called a map or a mapping On each set X there is the identity function id : X X that does nothing: idx x Also one can compose functions X f Y g Z to a function: g f : X Z given by g f x gf x B Jacobs Version: spring 0 Wiskunde 9 / 55 B Jacobs Version: spring 0 Wiskunde / 55

4 Linear functions We have seen that the two relevant operations of a vector space are addition and scalar multiplication A linear function is required to preserve these two Let V, W be two vector spaces, and f : V W a function between them; f is called linear if it preserves both: addition: for all v, v V, f v } + {{ v } f v + f v }{{} in V in W scalar multiplication: for each v V and a R, f a }{{} v a f v }{{} in V in W B Jacobs Version: spring 0 Wiskunde / 55 Linear map examples I First we consider functions f : R R Most of them are not linear, like, for instance: f x + x, since f 0 0 f x x, since f f f These linear maps R R can only be very simple Lemma Each linear map f : R R is of the form f x c x, for some c R this constant c depends on f Proof: Scalar multiplication on R is ordinary multiplication Hence: f x f x x f f x c x, for c f preserve zero Lemma Each linear map f : V W preserves: zero: f 0 0 minus: f v f v Proof: Nice illustration of axiomatic reasoning: 0 f 0 f 0 f v f v + 0 f f 0 f v + f v f v f 0 + f 0 f 0 f v + f v f v f 0 + f 0 f 0 f v + v f v f f 0 f v f 0 0 f v f v B Jacobs Version: spring 0 Wiskunde 3 / 55 Linear map examples II Consider the map f : R 3 R given by f x, x, x 3 x x, x + x 3 We show in detail that this f is linear, following the definition Preservation of scalar multiplication from R 3 to R : f a x, x, x 3 f a x, a x, a x 3 a x a x, a x + a x 3 a x x, a x + x 3 a x x, x + x 3 a f x, x, x 3 B Jacobs Version: spring 0 Wiskunde 4 / 55 Linear map examples II cntd Preservation of addition from R 3 to R : f x, x, x 3 + y, y, y 3 f x + y, x + y, x 3 + y 3 x + y x + y, x + y + x 3 + y 3 x x + y y, x + x 3 + y + y 3 x x, x + x 3 + y y, y + y 3 f x, x, x 3 + f y, y, y 3 B Jacobs Version: spring 0 Wiskunde 5 / 55 Linear map examples III Consider the map f : R R given by f x, y x cosϕ y sinϕ, x sinϕ + y cosϕ In the same way one can show that f is linear [Do it yourself!] This map described rotation in the plane, with angle ϕ: B Jacobs Version: spring 0 Wiskunde 6 / 55 B Jacobs Version: spring 0 Wiskunde 7 / 55

5 and bases, example I Recall the linear map f x, x, x 3 x x, x + x 3 Claim: this map is entirely determined by what it does on the base vectors, 0, 0, 0,, 0, 0, 0, R 3, namely: f, 0, 0, 0 f 0,, 0, f 0, 0, 0, Indeed, by linearity: f x, x, x 3 f x, 0, 0 + 0, x, 0 + 0, 0, x 3 f x, 0, 0 + x 0,, 0 + x 3 0, 0, f x, 0, 0 + f x 0,, 0 + f x f, 0, 0 + x f 0,, 0 + x 3 f 0, 0, x, 0 + x, + x 3 0, x x, x + x 3 x 3 0, 0, B Jacobs Version: spring 0 Wiskunde 9 / 55 The general case The aim is to obtain a matrix for an arbitrary linear map Assume a linear map f : V W, where: the vector space V has basis {v,, v n } V ; W has basis {w,, w m } Each x V can be written as x a v + + a n v n Hence: f x f a v + + a n v n a f v + + a n f v n by linearity of f Thus, f is determined by its values f v,, f v n on base vectors v j V By writing f v j b j w + + b mj w m we obtain an m n matrix with entries b ij i m,j n and bases, example I cntd Our f x, x, x 3 x x, x + x 3 is thus determined by: f, 0, 0, 0 f 0,, 0, f 0, 0, 0, We can organise these data in a 3 matrix: 0 0 The f v i, for base vector v i, appears as the i-the column Applying f can be done by a new kind of multiplication: x 0 x 0 def x + x + 0 x 3 x x 0 x x + x + x 3 x + x 3 3 B Jacobs Version: spring 0 Wiskunde 30 / 55 Towards matrix-vector multiplication In this setting, we have: f x f a v + + a n v n a f v + + a n f v n a b w + + b m w m + + an bn w + + b mn w m a b + + a n b n w + + a b m + + a n b mn wm b a + + b n a n w + + b m a + + b mn a n wm This motivates the definition of matrix-vector multiplication: b b n a b a + + b n a n b m b mn a n b m a + + b mn a n B Jacobs Version: spring 0 Wiskunde 3 / 55 Matrix-vector multiplication, concretely Recall f x, x, x 3 x x, x + x 3 with matrix: 0 0 We can directly calculate f,,,, We can also get the same result by matrix-vector multiplication: This multiplication can be understood as: putting the argument values x, x, x 3 in variables of the underlying equations, and computing the outcome B Jacobs Version: spring 0 Wiskunde 33 / 55 B Jacobs Version: spring 0 Wiskunde 3 / 55 Another example, to learn the mechanics B Jacobs Version: spring 0 Wiskunde 34 / 55

6 Linear function from matrix Matrix summary We have seen how a linear function can be described via a matrix One can also read each matrix as a linear function Consider the matrix It has 3 columns/inputs and two rows/outputs Hence it describes a function f : R 3 R Namely: f x, x, x 3 x x 3, 5x + x 3x 3 Assume bases {v,, v n } V and {w,, w m } W Each linear map f : V W corresponds to an m n matrix, and vice-versa We often write the matrix of f as M f The i-th column in this matrix M f is given by the coefficients of f v i, wrt the basis w,, w m of W Matrix-vector multiplication corresponds to function application: f v is the same as M f v This matrix M f of f depends on the choice of basis: for different bases of V and W a different matrix is obtained Matrix-vector multiplication forms itself a linear function B Jacobs Version: spring 0 Wiskunde 35 / 55 The identity matrix Consider the following n n identity matrix with diagonal of s: 0 0 I n To which function does I n correspond? The identity function R n R n To which system of equations does I n correspond? x 0 x n 0 B Jacobs Version: spring 0 Wiskunde 37 / 55 Matrices as vectors II: example B Jacobs Version: spring 0 Wiskunde 36 / 55 Matrices as vectors I Write Mat m,n {M M is an m n matrix} Thus each M Mat m,n can be written as M a ij, for i m and j n We can add two such matrices M, N Mat m,n, giving M + N Mat m,n the matrices are added entry-wise, that is: if M a ij, N b ij, M + N c ij, then c ij a ij + b ij Similarly, matrices can be multiplied by a scalar s R s M Mat m,n has entries s a ij Finally, there is a zero matrix 0 m,n Mat m,n, with only zeros as entries Mat m,n is a vector space B Jacobs Version: spring 0 Wiskunde 38 / 55 Matrices as vectors III: transpose Addition: For a matrix M Mat m,n write M T Mat n,m for the transpose of M It is obtained by mirroring: if M a ij then M T has entries a ji Scalar multiplication: Transposition is a linear map T : Mat m,n Mat n,m That is: M + N T M T + N T a M T a M T B Jacobs Version: spring 0 Wiskunde 39 / 55 B Jacobs Version: spring 0 Wiskunde 40 / 55

7 definitions Subspace definition Let f : V W be a linear map the kernel of f is the subset of V given by: kerf {x V f x 0} the image of f is the subset of W given by: imf {f x x V } Consider the function f : R R given by f x 0, x the kernel is {x R 0, x 0, 0} {0} the image is y-axis {0, x x R} B Jacobs Version: spring 0 Wiskunde 4 / 55 Subspace examples A subset S V of a vector space V is called a linear subspace if S is closed under addition and scalar multiplication: 0 S v, v S implies v + v S v S and a R implies a v S Note A subspace S V is a vector space itself, and can thus also have a basis Also S can have its own dimension, where dims dimv B Jacobs Version: spring 0 Wiskunde 43 / 55 Basis for subspaces Earlier we saw that the subset of solutions of a system of equations is closed under addition and scalar multiplication, and thus is a linear subspace The diagonal D {x, x x R} R is a linear subspace: if x, x, x, x D, then also x, x + x, x x + x, x + x D if x, x D and a R, also a x, x a x, a x D Also: D has a single vector as basis, for example, thus, D has dimension Let the space V have dimension n, and a subspace S V dimension p, where p n Then: any set of > p vectors in S is linearly dependent any set of < p vectors in S does not span S any set of p independent vectors in S is a basis for S any set of p vectors that spans S is a basis for S B Jacobs Version: spring 0 Wiskunde 44 / 55 Kernels and images are subspaces For a linear map f : V W, kerf {x f x 0} V is a linear subspace imf {f x x V } W is a linear subspace As subspaces kerf and imf also have dimensions see later Proof: We check two cases do the others yourself! Closure of kerf under addition: if x, x kerf, then f x 0 and f x 0 By linearity of f, f x + x f x + f x , so x + x kerf Closure of imf under scalar multiplication: Assume y imf, say y f x, and a R Again by linearity: a y a f x f a x, so a y imf B Jacobs Version: spring 0 Wiskunde 46 / 55 B Jacobs Version: spring 0 Wiskunde 45 / 55 The kernel as solution space With this kernel and image terminology we can connect some previous notions Suppose a linear map f : V W has matrix M f Then: v kerf f v 0 M f v 0 v is solution of the equations given by M f Moreover, the dimension of the kernel dimkerf is the same as the number of columns without pivots in the echelon form of M f B Jacobs Version: spring 0 Wiskunde 47 / 55

8 Independence and kernels More concretely Assume a vector space V with vectors v,, v n V We define a function f : R n V by Then: f a,, a n a v + + a n v n You can check for yourself that f is linear the vectors v,, v n are independent a v + + a n v n 0 implies a a n 0 f a,, a n 0 implies a a n 0 kerf {0,, 0} the matrix M f of f has only 0,, 0 as solution the echelon form of M f has n pivots B Jacobs Version: spring 0 Wiskunde 48 / 55 Kernel-image-dimension theorem aka rank-nullity For a linear map f : V W one has: dimkerf + dimimf dimv Pictorial proof: Consider echelon forms of the matrix M f of f : }{{}}{{}}{{} dimimf dimkerf dimimf, where dimkerf 0 B Jacobs Version: spring 0 Wiskunde 50 / 55 ctd We now know: dimimf 3 dimkerf 3 Thus the image of f : R 3 R is the whole codomain R Hence f is surjective 0 In the echelon form 0 we can see the basis of imf, namely the first two columns 0 That is: and, the standard basis for imf R 0 These base vectors are indeed in imf, since: Consider the 3 vectors 0,, and 0 they are independent because the associated matrix 0 0 reduces to and this echelon form has 3 pivots, that is, only 0 as solution B Jacobs Version: spring 0 Wiskunde 49 / 55 Consider the map f : R 3 R given by M f 0 The echelon form of M f is 0 { Thus: x, x, x 3 kerf x x 3 0 { 5x + x 3x 3 0 x x 3 0 x x 3 0 Thus kerf is -dimenionsional, say with base vector Check: Indeed, M f 0 B Jacobs Version: spring 0 Wiskunde 5 / 55 Main points, so far, I We have seen matrices as abstractions of systems of equations by removing variables: a x + + a n x n b a a n x b is a m x m + + a mn x mn b m a m a mn matrices capture the behaviour of linear maps on base vectors: M f f v f v n where f v i x n b n B Jacobs Version: spring 0 Wiskunde 5 / 55 B Jacobs Version: spring 0 Wiskunde 54 / 55

9 Main points, so far, II These two points are connected: solutions of a homogeneous system M f kerf x x n 0 0 Also: basic solutions are base vectors of this kernel Solutions of a non-homogeneous system M f b exist if and only if imf b n x x n are in b b n Also: these solutions are of the form P + H i, with H i kerf B Jacobs Version: spring 0 Wiskunde 55 / 55