Optimiztion Lecture 1 Review of Differentil Clculus for Functions of Single Vrible http://users.encs.concordi.c/~luisrod, Jnury 14
Outline Optimiztion Problems Rel Numbers nd Rel Vectors Open, Closed nd Bounded sets Rel Functions Limit nd Continuity Mimum nd Minimum of Functions Derivtive, L Hopitl s Rule nd Sttionry Points
Optimiztion Problems An optimiztion problem (OP) is problem of the form min f () s.t Ω This is minimiztion (we cn consider mimiztion of f s minimiztion of f), f is function to be minimized, s.t mens subject to, is the set of fesible vlues of vrible The obvious question is wht kinds of functions, wht kinds of sets nd wht kinds of vribles cn we hndle Ω The vrible cn be single rel vrible or vector, the function f will be ssumed to be of clss C (twice differentible), the set will be n open set or closed set Ω The net slides will eplin wht these concepts men 3
Integer, Rtionl, Irrtionl Numbers We ssume fmilirity with ddition +, subtrction -,division /, multipliction, less <, greter >, less or equl, greter or equl, (pproimtely ) equl =,different,equivlent, implies,or, nd, union, intersection, (not) member ( ), for ll, there eists one, b = = b = The integer numbers re the non-negtive integers,1,, union with the negtive integers -1,-,-3, A rtionl number is the rtio of two integers (e.g, ½). There re numbers tht cnnot be represented s the rtio of two integers nd re clled irrtionl numbers. An emple is the length of the circumference divided by its dimeter tht yields the number π = 3.14159... Rel numbers cn be either rtionl or irrtionl Emple: The re of rectngle with sides mesuring cm nd 1cm is A = 1= centimeters squred b = cm =1cm cm 4
Rel Numbers The set of rel numbers is denoted R. In R we cn define two closed opertions (ddition nd multipliction) tht hve inverses (subtrction nd division, respectively) This mens tht two rel numbers nd cn be dded to yield rel nd multiplied to yield rel or + b Associtive Property: Commuttive Property: Eistence of Identity: Eistence of Inverse: Distributive Property: (b + c) = ( b)+ ( c) A set with two opertions verifying these properties is clled field nd is represented by the trid ( R,+, ) All numbers < < b form n open intervl denoted (,b) Nottion for lrge products, sums: ( + b) + c = + ( b + c), (. b). c =.( b. c) + b = b +,. b = b. b + =, 1= + ( ) =, b b 1 =1, b N =, n = 1 + ++ N N times n=1 5 b N b
Vectors nd Vector Coordintes A vector is geometric object represented by letter with n rrow, e.g., with mgnitude nd direction A vector hs coordintes in given frme. A vector is not the sme s its coordintes. Coordintes of the sme vector cn be different in different frmes. E The coordintes of vector in frme E will be denoted by nd will be represented s column rry of rel numbers. The trnspose of the coordintes of vector will be row rry denoted by E [ ] T nd the derivtive of vector with respect to t will be denoted by = lim (more lter) h [( ( t + h) ( t)) h] Emple: For D frme E with unit vectors ( e 1,e ) = te! 1 + t e, E $ [ ] 1 = # & "# %& =! t $ # & "# t %&, E [ ] T! = t t "# $ %&, E! " $ % =! 1 # " t = e 1 + e 1 6 [ ] $ & %
Open, Closed, Compct Sets R Let be the set of rel numbers nd the set of rel vectors with n coordintes An open bll in with center nd rdius is the set of vectors verifying B r ( ) = : An open set in is set for which round ech point one cn find n open bll tht is fully contined in the set. For emple n open bll is n open set A closed set is set for which its complementry R n \ Ω which contins ll elements in R n tht re not in Ω is open. For emple { : r} A bounded set is set tht cn be contined in n open bll with finite rdius. For emple { : 1} A compct set is set tht is closed nd bounded 7 R n R n r > < r { } R n r
Rel Functions A rel function is mpping of rel vector into rel vlue f ( ). If is sclr, f is function of rel vrible For ech vlue of there cn only be vlue of f but there could be more thn one vlue of tht hve the sme vlue of f. Emple: f () = = ± f () When different vlues of correspond to different vlues of f the function is injective or 1-to-1. Emple: f ( ) = + 1 The set of vlues of for which f eists is clled the domin of f. The set of vlues f is clled the imge of f When the imge of f is equl to R then f is clled surjective or onto. Emple: f ( ) = A function tht is both injective nd surjective is clled bijective. These functions cn be inverted ( eists) A function f is liner if f (α 1 + β ) = α f ( f 1 1 )+ β f ( ), α, β R Emple: f ( ) = 8
Rel Vrible Functions: Trigonometric Stright lines cn be used to form tringles between three points A, B, C Pythgoren Theorem: sin A =, c b sin B = = c cos A = cos A, b c, cos B = tn A = c c = + b = sin A, b tn B = b = B C cot A Right Tringle c A A b B Oblique Tringle B Lw of the sines: sin A = b sin B = c sin C c Lw of the cosines: c = + b b cosc C C b A 9 A
Useful Trigonometric Reltions sin θ + cos θ =1 sin θ =.5.5cos(θ) sin(θ) = sin(θ)cos(θ) cos(θ) = cos (θ) sin (θ) tn(θ) = tn(θ) / (1 tn (θ)) sin(α + β) = sinα cosβ + cosα sin β sin(α β) = sinα cosβ cosα sin β cos(α + β) = cosα cosβ sinα sin β cos(α β) = cosα cosβ + sinα sin β sinα + sin β = sin α + β cosα β sinα sin β = cos α + β sin α β cosα + cosβ = cos α + β cosα β cosα cosβ = sin α + β sin α β 1
Polynomils of Rel Vrible A polynomil function of order n is function of the form f () = α +α 1 +α ++α n n, α i R If is rel vrible then we sy tht When n=1 then f is polynomil of first order We sy tht f is ffine if α nd liner if α = If n= then f is second order. We sy tht f is qudrtic Emples of polynomils of first nd second order: f () = +1 f () = f () = ( +1) = + +1 f R[] 11
Eponentil & Logrithm Functions The Euler s number e is pproimtely equl to.7188 nd is represented by the infinite sum e =1+ 1 1 + 1 1 + 1 1 3 + f () = e The eponentil function is defined by The logrithm function is the inverse of the eponentil function, is defined by f () = ln() nd verifies the following importnt properties Inverse Function Property: ln(e ) =, e ln() = ln(y) = ln()+ ln(y) ln( ) = ln(), R 1
f () f ( 1 ) f ( ) f ( ) Limit of Function nd Continuity Suppose we hve sequence of numbers n, n =1,, getting closer nd closer to some vlue. For emple 1 =.9, =.99, 3 =.999, The vlues get closer nd closer to 1 s n increses. We sy tht converges nd its limit is 1,i.e lim =1 n n n The difference n+p n lso gets smller for ny p s n increses nd we sy tht is Cuchy sequence If f is function of we sometimes wnt to determine if f lso converges to vlue when n converges to limit This vlue, if it eists, is clled lim f ( ). If lim f ( ) = f ( ) then we sy tht f is continuous t. If f is continuous 1 t ll points we sy tht f is continuous function Emple: f ( ) = sin, = π lim n f ( ) = f ( π ) = π 13
The Dirc Delt Function As n emple of function defined by limit we will now tlk bout the impulse function An impulse function, lso clled Dirc delt function δ (t), is the mthemticl model of n impulsive force like, for emple, the one eerted in billrd bll: strong force in short period of time δ (t) The function cn be defined s the result of limiting process s follows: δ ( t) = limδ Δ( t), δ Δ Δ ( t) 1, = Δ, if t otherwise Δ δ Δ (t) 1 Δ Δ Are=1 14 t
Mimum nd Minimum of Function A vlue f ( ) is clled locl minimum (mimum) if δ > : f ( ) f ( ( f ( ) f ( ) ) ), B δ ( ) D If the result is true in ll the domin D of f then it is globl minimum (mimum) Weierstrss Theorem: A continuous function defined on compct set hs mimum nd minimum f () min m 15
f Derivtive of Sclr Function For function f() represented by its grph one cn imgine connecting two points P, Q on the grph by line. This line is clled secnt. If we now mke converge to vlue such tht Q converges to P then the secnt converges to the tngent line t P. The derivtive f of f t is the slope of the tngent nd is defined by the limit in the bo. A function tht hs derivtive t ll points is clled differentible. The second derivtive f is the derivtive of the derivtive Q Q () df d = f ( ) f ( ) f ( + h) f ( ) f '( ) = lim = lim h h P Emple: f ( ) = df d 1 1 (1 + h) = lim h + 1, = 1 + 1 ((1) h 16 + 1) =
Useful Derivtive Reltions f () = α, f '() = α α 1 f () = sin(), f '() = cos() f () = cos(), f '() = sin() f () = tn(), f '() =1/ cos () f () = cos()sin(), f '() = cos() f () = e, f '() = e f () = ln(), f '() =1/ ( f ()+ g())' = f '()+ g'() ( f ()g())' = f '()g()+ f ()g'() ( f () / g())' = ( f '()g() f ()g'()) / g () Theorem (Chin Rule): If the derivtives g () nd f (g()) eist nd F()=f(g()) then Emple: f ( ) = F( ) = f + 1, g( ) = sin ( g( )) = sin F '( ) = f '( g( )) g'( ) + 1, F'( ) = sin( )cos( ) = sin() 17
L Hopitl s Rule As we sw derivtive is the limit of rtio Sometimes when computing the limit of rtio of two functions one cn hve n indeterminte result / For these cses there eists result clled L Hopitl s rule stted in the following theorem Theorem: Let f nd g be two functions tht hve derivtive f '( ), g'( ) t ech point of n open intervl (,b). For point in this intervl let lim f ( ) = lim g( ) = f '( ) If lim g'( ) Emple: f ( ) = sin, df d = eists then lim sin( + h) sin() sin( h) = lim = lim = h h h h f ( ) g( ) = lim f '( ) g'( ), cos( h) lim h 1 = 1 df d = 1 18
Sttionry Points of Functions The derivtive of function t point is the slope of the tngent to the grph of the function t tht point. If the derivtive is zero then the slope of the tngent is zero Points for which the derivtive is zero re thus clled sttionry points of function. They cn be mimum or minimum (see figure below) f () Necessry Condition f '( min ) = f '( m ) = min m 19