1. Foundations of Numerics from Advanced Mathematics. Linear Algebra

Transcription

1 Foundations of Numerics from Advanced Mathematics Linear Algebra Linear Algebra, October 23, 22

2 Linear Algebra Mathematical Structures a mathematical structure consists of one or several sets and one or several operations defined on the set(s) special elements: neutral element (of an operation) inverse element (of some element x) a group: a structure to add and subtract a field: a structure to add, subtract, multiply, and divide a vector space: a set of vectors over a field with two operations: scalar multiplication, addition of vectors, obeying certain axioms (which?) note: sometimes, the association with classical (geometric) vectors is helpful, sometimes it is more harmful Linear Algebra, October 23, 22 2

3 Exercise Mathematical Structures Show that the possible manipulations of the Rubik s Cube with the operation execute after are a group Linear Algebra, October 23, 22 3

4 Exercise Mathematical Structures Solution Show that the possible manipulations of the Rubik s Cube with the operation execute after are a group Closure: executing any two manipulations after one another is a Cube minipulation, again Associativity: the result of a sequence of three manipulations is obviously always the same no matter how you group them (the first two or the last two together) Identity: obviously included (just do nothing ) Invertibility: execute a manipulation in backward direction Linear Algebra, October 23, 22 4

5 Exercise Mathematical Structures Show that the rational numbers with the operations + (add) and (multiply) are a field Linear Algebra, October 23, 22 5

6 Exercise Mathematical Structures Solution Show that the rational numbers with the operations + (add) and (multiply) are a field Closure: obviously closed under + and Identity: for +, for Invertibility: each element q has an inverse q under + and q under The latter holds for all elements except from the neutral element of +, ie, Associativity: well-known for both + and Commutativity: also known from school (a + b = b + a, a b = b a) Distributivity: dito (a (b + c) = a b + a c) Linear Algebra, October 23, 22 6

7 Exercise Mathematical Structures Is the set of N N matrices (N N) matrices with real numbers as entries over the field of real numbers a vector space? Linear Algebra, October 23, 22 7

8 Exercise Mathematical Structures Solution Is the set of N N matrices (N N) matrices with real numbers as entries over the field of real numbers a vector space? The answer is yes Look up the axioms and show that they hold for the xample on your own Linear Algebra, October 23, 22 8

9 Vector Spaces a linear combination of vectors linear (in)dependence of a set of vectors the span of a set of vectors a basis of a vector space definition? why do we need a basis? is a vector s basis representation unique? is there only one basis for a vector space? the dimension of a vector space does infinite dimensionality exist? important applications: (analytic) geometry numerical and functional analysis: function spaces are vector spaces (frequently named after mathematicians: Banach spaces, Hilbert spaces, Sobolev spaces, ) Linear Algebra, October 23, 22 9

10 Exercise Vector Spaces {( Is the set of vectors independent? ) (, ) (, 3 )} linearly Linear Algebra, October 23, 22

11 Exercise Vector Spaces Solution {( Is the set of vectors independent? ) (, ) (, 3 )} linearly The set of vectors is not linearly independent, since the third element can easily be written as a linear combination of the first two: ( 3 ) ( = ) ( + 3 ) Linear Algebra, October 23, 22

12 Exercise Vector Spaces span{( ), ( )} =? Linear Algebra, October 23, 22 2

13 Exercise Vector Spaces Solution span{( ), ( )} = {( a b ) ; a, b R } Linear Algebra, October 23, 22 3

14 Exercise Vector Spaces Consider the set of all possible polynomials with real coefficients as a vector space over the field of real numbers What s the dimension of this space? Give a basis Linear Algebra, October 23, 22 4

15 Exercise Vector Spaces Solution Consider the set of all possible polynomials with real coefficients as a vector space over the field of real numbers What s the dimension of this space? Give a basis The space is infinite dimensional, a basis is for example {, x, x 2, x 3, } Linear Algebra, October 23, 22 5

16 Linear Mappings definition in the vector space context; notion of a homomorphism image and kernel of a homomorphism matrices, transposed and Hermitian of a matrix relations of matrices and homomorphisms meaning of injective, surjective, and bijective for a matrix; rank of a matrix meaning of the matrix columns for the underlying mapping matrices and systems of linear equations basis transformation and coordinate transformation mono-, epi-, iso-, endo-, and automorphisms Linear Algebra, October 23, 22 6

17 Exercise Linear Mappings Is the mapping f : R 3 R 3, x 5 x + ( 2 3 ) linear? Linear Algebra, October 23, 22 7

18 Exercise Linear Mappings Solution Is the mapping f : R 3 R 3, x 5 x + ( 2 3 ) linear? f is not linear, since f (α x) αf ( x) Linear Algebra, October 23, 22 8

19 Exercise Linear Mappings What s( the linear ) mapping f : R 2 R 2 corresponding to the 4 matrix? 3 2 Linear Algebra, October 23, 22 9

20 Exercise Linear Mappings Solution What s( the linear ) mapping f : R 2 R 2 corresponding to the 4 matrix? 3 2 (( x f y )) ( = 4x 3x + 2y ) Linear Algebra, October 23, 22 2

21 Exercise Linear Mappings Give the rank of the matrix Is the corresponding linear mapping injective, surjective, bijective? Linear Algebra, October 23, 22 2

22 Exercise Linear Mappings Solution Give the rank of the matrix Is the corresponding linear mapping injective, surjective, bijective? The rank is three Thus, the corresponding linear mapping is neither injective, nor surjective or bijective Linear Algebra, October 23, 22 22

23 Examples Linear Mappings Monomorphism: Epimorphism: Iso-/Automorphism: Endomorphism: ( ( ) ) ( ( 2 2 ) ) Linear Algebra, October 23, 22 23

24 Determinants definition properties meaning occurrences Cramer s rule Linear Algebra, October 23, 22 24

25 Determinants Definition det(a) = a, a,2 a,n a 2, a 2,2 a 2,N a N, a N,N = a, a 2,2 a 2,N a N,2 a N,N a,2 a 2, a 2,3 a 2,N a 3, a 3,N a N, a N,3 a N,N + Linear Algebra, October 23, 22 25

26 Exercise Determinants det(a) = A defines a morphism det(a) A defines a morphism Linear Algebra, October 23, 22 26

27 Exercise Determinants Solution det(a) = A defines an Endomorphism det(a) A defines an Automorphism Linear Algebra, October 23, 22 27

28 Exercise Determinants det(a B) =? det ( A ) =? det ( A T ) =? Linear Algebra, October 23, 22 28

29 Exercise Determinants Solution det(a B) = det(a) det(b) det ( A ) = det(a) det ( A T ) = det(a) Linear Algebra, October 23, 22 29

30 Exercise Determinants Determine the solution of the linear system 2x + x 2 = 4 2x 2 + x 3 = x + x 2 + x 3 = 3 with the help of determinants Linear Algebra, October 23, 22 3

31 Exercise Determinants Solution Determine the solution of the linear system 2x + x 2 = 4 2x 2 + x 3 = x + x 2 + x 3 = 3 with the help of determinants x = = 7 3 ; x 2 = = 7 3 ; x 3 = = 4 3 Linear Algebra, October 23, 22 3

32 Eigenvalues notions of eigenvalue, eigenvector, and spectrum similar matrices A, B: S : B = SAS (ie: A and B as two basis representations of the same endomorphism) resulting objective: look for the best / cheapest representation (diagonal form) important: matrix A is diagonalizable iff there is a basis consisting of eigenvectors only characteristic polynomial, its roots are the eigenvalues Jordan normal form important: spectrum characterizes a matrix many situations / applications where eigenvalues are crucial Linear Algebra, October 23, 22 32

33 Exercise Eigenvalues ( ) 3 2 Diagonalize the matrix Give both eigenvalues and 2 3 eigenvectors and the basis transformation matrix transforming the given matrix in diagonal form Linear Algebra, October 23, 22 33

34 Exercise Eigenvalues Solution ( ) 3 2 Diagonalize the matrix Give both eigenvalues and 2 3 eigenvectors and the basis transformation matrix transforming the given matrix in diagonal form Eigenvalues: 3 λ λ = 9 6λ + λ2 4 = 5 6λ + λ 2 λ,2 = 6± = 3 ± 2 λ = 5, λ 2 = Linear Algebra, October 23, 22 34

35 Exercise Eigenvalues Solution ( ) 3 2 Diagonalize the matrix Give both eigenvalues and 2 3 eigenvectors and the basis transformation matrix transforming the given matrix in diagonal form ( ) ( 2 2 x Eigenvector for λ = 5: 2 2 y ( ) ( 2 2 x Eigenvector for λ 2 = : 2 2 y ) ( = ) ( = ) ( x = y x = ) ( x = y x 2 = ) ) Linear Algebra, October 23, 22 35

36 Exercise Eigenvalues Solution ( ) 3 2 Diagonalize the matrix Give both eigenvalues and 2 3 eigenvectors and the basis transformation matrix transforming the given matrix in diagonal form ( The basis transformation matrix thus is ( 5 ) ) and results in the diagonal matrix Linear Algebra, October 23, 22 36

37 Scalar Products and Vector Norms notions of a linear form and a bilinear form scalar product: a positive-definite symmetric bilinear form examples of vector spaces and scalar products vector norms: definition: positivity, homogeneity, triangle inequality meaning of triangle inequality examples: Euclidean, maximum, and sum norm normed vector spaces Cauchy-Schwarz inequality notions of orthogonality and orthonormality turning a basis into an orthonormal one: Gram-Schmidt orthogonalization Linear Algebra, October 23, 22 37

38 Exercise Scalar Products and Vector Norms Are the following operators scalar products in the vector space of continuous functions on the interval [a; b]? f, g := b a f (x) g(x)dx f, g 2 := b a f (x) g(x)2 dx f, g 3 := b a f + (x) g(x)dx Linear Algebra, October 23, 22 38

39 Exercise Scalar Products and Vector Norms Solution Are the following operators scalar products in the vector space of continuous functions on the interval [a; b]? f, g := b a f (x) g(x)dx Yes! f, g 2 := b a f (x) g(x)2 dx No! (not linear in g) f, g 3 := b a f + (x) g(x)dx No! (not positive definite) Linear Algebra, October 23, 22 39

40 Exercise Scalar Products and Vector Norms Proof that a set { x, x 2,, x N } of non-zero orthogonal vectors in a vector space with scalar product (, ) always is a basis of its span Linear Algebra, October 23, 22 4

41 Exercise Scalar Products and Vector Norms Solution Proof that a set { x, x 2,, x N } of non-zero orthogonal vectors in a vector space with scalar product (, ) always is a basis of its span Proof by contradiction: Assume that the set is not linearly independent Then, there is a element x i taht can be written as a linear combination x i = k I α k x k of other elements, where the index set I {, 2,, N} does not contain i With this, we get ( x i, x i ) = ( x i, k I α k x ) k = k I α k ( x i, x k ) = Contradiction Thus, the vector set is linearly independent and, thus, is a basis of its span Linear Algebra, October 23, 22 4

42 Exercise Scalar Products and Vector Norms {( Transform of R 3 ), ( ), ( )} into an orthogonal basis Linear Algebra, October 23, 22 42

43 Exercise Scalar Products and Vector Norms Solution {( Transform of R 3 ), ( ), ( )} into an orthogonal basis Gram-Schmidt orthogonalization: x = x 3 =, x 2 = x, ( x, x ) x x, ( x, x ) x 2, ( x 2, x 2) x = x 2 = 2 3 x = , Linear Algebra, October 23, 22 43

44 Matrix Norms definition: properties corresponding to those of vector norms plus sub-multiplicativity: AB A B plus consistency Ax A x matrix norms can be induced from corresponding vector norms: Euclidean, maximum, sum A := max Ax x = alternative: completely new definition, for example Frobenius norm (consider matrix as a vector, then take Euclidean norm) Linear Algebra, October 23, 22 44

45 Classes of Matrices symmetric: A = A T skew-symmetric: A = A T Hermitian: A = A H = ĀT spd (symmetric positive definite): x T Ax > x orthogonal: A = A T (the whole spectrum has modulus ) unitary: A = A H (the whole spectrum has modulus ) normal: AA T = A T A or AA H = A H A, resp (for those and only those matrices there exists an orthonormal basis of eigenvectors) Linear Algebra, October 23, 22 45