Linear System Theory - PDF Free Download

Linear System Theory Wonhee Kim Lecture 3 Mar. 21, 2017 1 / 38

Overview Recap Nonlinear systems: existence and uniqueness of a solution of differential equations Preliminaries Fields and Vector Spaces 2 / 38

Recap classical vs. modern control theory linearization: local analysis continuous and discrete-time LTI Systems system modeling: from ODE to state-space system system transformation: from time domain to frequency domain In this lecture, we will see why we study linear system theory not in the textbook, see Nonlinear Systems by H. Khalil 3 / 38

Recap: Frequency and Time Domain Analysis Differential equation Transfer function ẍ + 3ẋ + 2x = u(t) (s 2 + 3s + 2)X (s) = U(s) G(s) = X (s) U(s) = 1 s 2 + 3 2 + 2 = 1 (s + 1)(s + 2) State space representation x 1 = x and x 2 = ẋ ) ( ) (ẋ1 0 1 ẋ = = x + ẋ 2 2 3 ( 0 1) u 4 / 38

Recap: Frequency and Time Domain Analysis Transfer function State space representation G(s) = X (s) U(s) = 1 s 2 + 3 2 + 2 = 1 (s + 1)(s + 2) ẋ = ) (ẋ1 = ẋ 2 ( 0 ) 1 2 3 ( 0 x + u 1) What is the pole of the transfer function? What is the eigenvalue of matrix A? det(si A) = 0 ( ) s 1 det = 0 2 s + 3 s 2 + 3s + 2 = (s + 1)(s + 2) = 0 stable!!! Inverse Laplace transform of G(s): g(t) = e t e 2t 5 / 38

Recap:Frequency and Time Domain Analysis Transfer function G(z) = Y (z) U(z) = z (z 0.1)(z 2) = z z 2 2.1z + 0.2 Inverse z-transformation: g(t) = 10 19 2t 10 19 0.1t Pole: 0.1 and 2 Eigenvalue: : 0.1 and 2 6 / 38

One-dimensional LTI System dx(t) dt Is there a solution? What is the solution? = Ax(t), x(0) = x 0,, t [0, T ], A R If there is a solution, does it exist for all t [0, T ]? Is there a unique solution? Does existence and uniqueness of the solution depend on the initial condition x 0? These properties also hold when A is a matrix, which will be studied later in this course 7 / 38

Nonlinear Systems dx(t) dt Is there a solution? Yes = x 1/3 (t), x(0) = 0, t [0, 1] What is the solution? x(t) = 0 and x(t) = (2t/3) 3/2 If there is a solution, does it exist for all t [0, T ]? Yes Is there a unique solution? No 8 / 38

Nonlinear Systems dx(t) dt Is there a solution? Yes What is the solution? = x 2 (t), x(0) = 1, t [0, 1] x(t) = 1 t 1 If there is a solution, does it exist for all t [0, 1]? No finite escape time: the phenomenon that x(t) escapes to infinity at a finite time (also called conjugate point ) In this case, x(t) has a finite escape time at t = 1, since lim t 1 x(t) = 9 / 38

Nonlinear Systems Question: What is the condition under which there exists a unique solution to the nonlinear system? Definition A function f (x) : R n R n is Lipschitz if there exists a constant L 0 such that for all x, y R n, (note that for any x R n, x 2 = x T x) f (x) f (y) L x y f (x) f (y) x y L A function that has infinite slope at some point is not Lipschitz Question: Is f (x) continuous? Is f (x) = Ax Lipschitz? Yes (A: matrix or scalar) Is f (x) = x 1/3 Lipschitz? f (x) = 1 3 x 2 3 as x 0 10 / 38

Nonlinear Systems Theorem Suppose that f (x) is Lipschitz. Then, the state equation ẋ(t) = f (x(t)) with x(0) = x 0 has a unique solution over [0, T ]. LTI system: Lipschitz!! ẋ(t) = x 1/3 (t) dost not have a unique solution f (x) = x 1/3 is not Lipschitz This is one of the main properties of LTI systems Do you want to see the proof? We will not cover this in this class, since it requires heavy mathematics I will introduce the main idea of the proof 11 / 38

Nonlinear Systems Main idea of the proof t ẋ(t) = f (x(t)), x(0) = x 0 x(t) = x 0 + f (x(s))ds 0 Let (T x)(t) = x 0 + t 0 f (x(s))ds If x(t) satisfies the above relation, then x(t) is a solution of the ODE Equivalently, if there exists x(t) such that x(t) = (T x)(t), then we are done x(t) = (T x)(t) fixed point!!! We can show that there exist a unique fixed point of (T x)(t) under the Lipschitz condition The detailed proof will be provided in an advanced control course (e.g. nonlinear system control) 12 / 38

One-dimensional LTV System ẋ(t) = A(t)x(t), x(0) = x 0, A(t) : [0, T ] R A(t) is a function that is defined on [0, T ] The solution does not exist if A(t) is not continuous A(t) is continuous on [0, T ] there exists a constant c 0 such that A(t) c for all t [0, T ] (why???), which implies It is Lipschitz!!! A(t)x A(t)y A(t) x y c x y There exists a unique solution 13 / 38

Matrix, Function, Linear Algebra Linear vector space state space, input space, output space Linear operators reachability (controllability), observability operator Normed spaces stability Inner product controllability and observability grammians 14 / 38

Preliminaries Sets: Z-integer, Q-rational numbers, N-natural numbers, R-real numbers, R +, C-complex numbers, C + Cartesian product: two sets X and Y, the cartesian product is X Y A B = {(a, b) a A and b B} Function: f : X Y for any x X, f assigns a unique f (x) = y Y X : domain, Y : codomain {f (x) : x X }: range f : function, operator, map, transformation 15 / 38

Preliminaries Injective (one to one): f is injective if and only if f (x 1 ) = f (x 2 ) x 1 = x 2, conversely, x 1 x 2 implies f (x 1 ) f (x 2 ) Surjective (onto): f is surjective if and only if for all y Y, there exists x X such that y = f (x) Bijective (one to one correspondnce): f is bijective if and only if for all y Y, there exists a unique x X such that y = f (x) equivalently, f is bijective if and only if f is injective and surjective 16 / 38

Fields A field F is an object that consisting of a set of elements, and two binary operations: addition (+), multiplication ( ) such that Addition (i) associative: (α + β) + γ = α + (β + γ) for all α, β, γ F (ii) commutative: α + β = β + α for all α, β F (iii) there exists an identity element 0: α + 0 = α for all α F (iv) for all α F, there exists an inverse α such that α + ( α) = 0 Multiplication (i) associative: α (β γ) = (α β) γ for all α, β, γ F (ii) commutative: α β = β α for all α, β F (iii) there exists an identity element 1: α 1 = α for all α F (iv) for all α F, α 0, there exists an inverse α 1 such that α α 1 = α 1 α = 1 17 / 38

Fields Examples: R, C, Q Example: Consider the set of all 2 2 matrices for the form Matrix addition and multiplication ( ) ( ) ( ) x y 0 0 1 0, x, y R, 0, 1 y x 0 0 0 1 The inverse always exists How about the set of all 2 2 matrices? 18 / 38

Vector Space vector space = linear space = linear vector space A linear space over a field F, (V, F), consists of a set V of vectors, a field F, and two operations, vector addition and scalar multiplication The two operations satisfy Addition (a) addition: for all x, y V, x + y V (b) associative: for all x, y, z V, (x + y) + z = x + (y + z) (c) commutative: for all x, y V, x + y = y + x (d) there exists a unique zero vector 0 V such that x + 0 = 0 + x = x for all x V (e) there exists a unique inverse x V such that x + ( x) = 0 for all x V 19 / 38

Vector Space vector space = linear space = linear vector space A linear space over a field F, (V, F), consists of a set V of vectors, a field F, and two operations, vector addition and scalar multiplication The two operations satisfy Multiplication (a) multiplication: for any α F and x V, αx V (b) associative: for any α, β F and x V, α(βx) = (αβ)x (c) distributive w.r.t. scalar addition: for any α F and x, y V, α(x + y) = αx + αy (d) distributive w.r.t. scalar multiplication for any α, β F and x V, (α + β)x = αx + βx (e) there exists a unique 1 F such that for any x V, 1x = x (f) there exists a unique 0 F such that for any x V, 0x = 0 20 / 38

Vector Space vector space = linear space = linear vector space A linear space over a field F, (V, F), consists of a set V of vectors, a field F, and two operations, vector addition and scalar multiplication Example: (F n, F) where F n = F F Example: (R n, R), (C n, C), (C n, R) Example: (R, C) is not a vector space! (why?) (1 + i)1 = 1 + i / R Example: a continuous function f : [t 0, t 1 ] R n, the set of such functions, (C([t 0, t 1 ], R n ), R), is a linear space Example: You will see other examples in HW 21 / 38

Vector Subspaces (Linear Subspaces) Let (V, F) be a linear space, and W V, i.e., W is a subset of V. Then (W, F) is called a subspace of (V, F) if (W, F) is itself a vector space (W, F) is a subspace if W V for any w 1, w 1 W and α, β F, αw 1 + βw 2 W 22 / 38

Vector Subspaces Example: In (R 2, R), every straight line passing through the origin is a subspace of (R 2, R). That is, ( x1 cx 2 ) for any fixed c R is a subspace of (R 2, R) Example: The vector space (R n, R) is a subspace of (C n, R) Question: Prove that if W 1 and W 2 are subspaces of V, then W 1 W 2 is a subspace W 1 W 2 is not necessarily a subspace Proof of (ii): We provide a counter example { ( ) 0 } W 1 = span, W 1 2 = span ( ( ( 1 0 1 = + / W 1) 1) 0) 1 W 2 { ( 1 0 ) } 23 / 38

Linear Independence and Dependence Suppose (V, F) is a linear space (vector space). The set of vectors {v 1,..., v p }, v i V is said to be linearly independent if and only if where α i F α 1 v 1 + α 2 v 2 + + α p v p = 0 α 1 = α 2 = = α p = 0 The set of vectors is said to be linearly dependent if and only if it is not linearly independent If the set of vectors is linearly dependent, then at least one (not everyone!!!) of them can be written as a linear combination of others 24 / 38

Example: x 1,..., x n R n and x 1 = 0. This is linearly dependent, since α 1 = 1, and α 2,..., α n = 0 Example: α 1 ( 1 1 ) ( ) 3 + α 2 2 ( 1 0 16/5 0 1 2/5 ( ) ( 2 0 + α 3 = 4 0) ) α ( 1 α 2 0 = 0) α 3 Example: F = R, k {0, 1,...}, and define ( α1 α 1 ) f k : [ 1, 1] R such that f k (t) = t k ( ) 16/5 = α 3 2/5 The set {f 0,..., f n } is linearly independent in (F([ 1, 1], R), R) We need to show α 0 + α 1 tα 2 t 2 + + α n t n = 0, where t [ 1, 1] and α i R, implies α i = 0 for all i 25 / 38

Dimension and Basis Dimension: The maximal number of linear independent vectors in a linear space (V, F) is called the dimension of the linear space (V, F) Example: (F n, F) Basis: A set of linearly independent vectors of a linear space (V, F) is said to be a basis of V if every vector in V can be expressed as a unique linear combination of these vectors Basis (equivalent definition): For a linear space (V, F), a set of vectors {v 1,..., v n } is called basis of V if {v 1,..., v n } spans V, V = { n i=1 α iv i : α i F, i} {v 1,..., v n } is linearly independent Example: Suppose that {v 1,.., v n } is a basis of V. Then for any v V, there exists a unique α 1,..., α n F such that v = α 1 v 1 + α 2 v 2 + + α n v n = ( ) v 1 v n α 1. α n 26 / 38

Dimension and Basis Example: (R n, R), (C, C), (C n, C), (C, R), (C n, R) Example: (R 2 2, R), (R n n, R) Example: F = R, k {0, 1,...}, and define f k : (, ) R such that f k (t) = t k The set {f 0, f 1...} is linearly independent in (F((, ), R), R) Equivalently, there exist no real constants α i s not all zero, such that α i t k = 0 k=1 The dimension of (F((, ), R), R) is not finite (infinite-dimensional space) 27 / 38

Dimension and Basis Question: What is the relationship between dimension and basis in a vector space? Theorem In an n-dimensional vector space, (V, F), any set of n linearly independent vectors forms a basis By definition of the n-dimensional vector space, there exists a set of n linearly independent vectors, {v 1,..., v n } (note that n is the maximum number of linearly independent vectors that we can have). Let x V, and consider {x, v 1,..., v n } (note that they are linearly dependent!). There exist α 0, α 1,.., α n in F not all zero such that α 0 x + α 1 v 1 + + α n v n = 0 Claim: α 0 0 (why??) by contradiction, suppose that α 0 = 0. Then α 1 v 1 + + α n v n = 0 implies α 1,..., α n = 0 contradicts the fact that α i 0 for all i 28 / 38

Dimension and Basis Theorem In an n-dimensional vector space, (V, F), any set of n linearly independent vectors forms a basis Hence, x = ( α 1 /α 0 )v 1 + + ( α n /α 0 )v n = β 1 v 1 + + β n v n This implies that every vector x V can be expressed as a linear combination of {v 1,..., v n } x = ( α 1 /α 0 )v 1 + + ( α n /α 0 )v n = β 1 v 1 + + β n v n What is the next? uniqueness Suppose that there is another linear combination of x, namely x = β 1 v 1 + + β n v n = γ 1 v 1 + + γ n v n We have show that β i = γ i for all i. Consider, 0 = (β 1 γ 1 )v 1 + + (β n γ n )v n 29 / 38

Dimension and Basis Dimension and Basis: If a linear space (V, F) has a basis of n elements, then (V, F) is said to be of dimension n. In other words, (V, F) is a n-dimensional linear space, and we write dim(v) = n Basis is not unique!! Given a basis, every vector x V can be uniquely represented by a set of scalars in F. Namely, for any v V, there exists a unique α i s in F such that v = α 1 v 1 + α 2 v 2 + + α n v n α = (α 1,..., α n ) F n is called the representation of v or the coordination vector of v α i is called the coordinate of v w.r.t. v i 30 / 38

Change of Basis Basis is not unique For each basis, there are different representations of a vector x Suppose that {v 1,..., v n } and {e 1,..., e n } are bases (plural of basis) of the linear space (V, F). Then x = n α i v i = i=1 n β i e i, (α i β i, i) We need to figure out the relationship between the different representations of the same vector i=1 32 / 38

Change of Basis Suppose that {v 1,..., v n } and {e 1,..., e n } are bases (plural of basis) of the linear space (V, F). Then for any x V, we have a unique representation of x for each basis: x = n α i v i = i=1 n β i e i, (α i β i, i) i=1 α 1 = ( v 1 ) v n. = ( e 1 ) e n α n β 1. β n Now, for each e i, there is a unique representation w.r.t. v e i = ( ) v 1 v n γ 1i. γ ni = [v]α = [e]β 33 / 38

Change of Basis For each e i, there is a unique representation w.r.t. v e i = ( ) v 1 v n γ 1i. γ ni Hence, γ 11 γ 12 γ 1n ( ) ( ) γ 21 γ 22 γ 2n e1 e 2 e n = v1 v 2 v n...... γ n1 γ n2 γ nn Now, [e] = [v][γ] x = [v]α = [e]β = [v][γ]β Since for each x V there is a unique representation of x w.r.t. (v 1,..., v n ), we have α = [γ]β 34 / 38

Change of Basis Now, x = [v]α = [e]β = [v][γ]β Since for each x V there is a unique representation of x w.r.t. (v 1,..., v n ), we have α 1 γ 11 γ 12 γ 1n β 1 α 2 γ 21 γ 22 γ 2n β 2 or α = [γ]β. α n =...... γ n1 γ n2 γ nn β 1 γ 11 γ 12 γ 1n β 2. = γ 21 γ 22 γ 2n...... β n γ n1 γ n2 γ nn 1 α 1 α 2. α n. β n 35 / 38

Change of Basis Similarly, Hence, We have v i = ( ) e 1 e n p 1i. p ni p 11 p 12 p 1n p 21 p 22 p 2n [v] = [e]...... = [e][p] p n1 p n2 p nn x = [v]α = [e]β = [e][p]α β 1 p 11 p 12 p 1n α 1 β 2. = p 21 p 22 p 2n α 2......., β n p n1 p n2 p nn α n α = [p] 1 β 36 / 38

Change of Basis Now, we can see that x = [v]α = [e]β = [e][p]α = [v]α = [e]β = [v][γ]β This implies β = [p]α, α = [γ]β α = [γ][p]α [γ][p] = I Hence, [γ] = [p] 1 or [p] = [γ] 1 This is known as Change of Basis!!! For x, suppose that the two bases (plural of basis) {v 1,..., v n } and {e 1,..., e n } are given, and you know that x = n i=1 α iv i = [v]α. Then you can represent x w.r.t. {e 1,..., e n } with x = [e][p]α = [e]β 37 / 38

Change of Basis Example v = Hence, {( 1 0), ( {( 0 1, e =, 1)} 1) [e] = [v][γ], [γ] = [e], [p] = [γ] 1 = [e] 1 x = ( ) ( ) 1 0 1 0 1 2 = [e][p]α = [e][e] 1 α = ( )} ( ( 1 1 1, x =, α = 1 2) 2) ( ) ( ) 1 1 1.5 1 1 0.5 38 / 38