Multidisciplinary System Design Optimization (MSDO)

Transcription

1 Multiiscilinary System Design Otimization (MSDO) Graient Calculation an Sensitivity Analysis Lecture 9 Olivier e Weck Karen Willco Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

2 Toay s Toics Graient calculation methos Analytic an Symbolic Finite ifference Comle ste Aoint metho Automatic ifferentiation Post-Processing Sensitivity Analysis effect of changing esign variables effect of changing arameters effect of changing constraints Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

3 Definition of the Graient How oes the function value change locally as we change elements of the esign vector? Comute artial erivatives of with resect to i i n 3 Graient vector oints normal to the tangent hyerlane of () 3 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

4 Geometry of Graient vector (D) Eamle function:, Contour lot Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco Graient normal to contours

5 Eamle Geometry of Graient vector (3D) 3 3 increasing values of o Tangent lane T 3 =3 o T Graient vector oints to larger values of 5 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

6 Other Graient-Relate Quantities acobian: Matri of erivatives of multile functions w.r.t. vector of variables z n n n z n z Hessian: Matri of secon-orer erivatives H n 6 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco n n n n n n z z z

7 Why Calculate Graients Require by graient-base otimization algorithms Normally nee graient of obective function an each constraint w.r.t. esign variables at each iteration Newton methos require Hessians as well Isoerformance/goal rogramming Robust esign Post-rocessing sensitivity analysis etermine if result is otimal sensitivity to arameters, constraint values 7 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

8 Analytical Sensitivities If the obective function is known in close form, we can often comute the graient vector(s) in close form (analytically):, Eamle: Analytical Graient: For comle systems analytical graients are rarely available Eamle = = (,)=3 (,) 0 0 Minimum 8 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

9 Symbolic Differentiation Use symbolic mathematics rograms e.g. MATLAB, Male, Mathematica construct a symbolic obect» syms» =++/(*);» =iff(,) =-/^/» =iff(,) = -//^ ifference oerator 9 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

10 Finite Differences (I) Function of a single variable f() First-orer finite ifference aroimation of graient: ' f o f o f o O f Forwar ifference aroimation to the erivative Truncation Error Secon-orer finite ifference aroimation of graient: o - o o + f f f O ' o o o Central ifference aroimation to the erivative Truncation Error 0 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

11 Finite Differences (II) Aroimations are erive from Taylor Series eansion: f f f f O 3 ' '' o o o o Neglect secon orer an higher orer terms; solve for graient vector: ' f o f o f o O Forwar Difference '' O f Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco o Truncation Error o

12 Finite Differences (III) Take Taylor eansion backwars at ' '' f o f o f o f o O () ' '' f o f o f o f o O () - () an solve again for erivative o () f f f O ' o o o Central Difference Truncation Error O f 6 o ''' o Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

13 Finite Differences (IV) o o o o () finite ifference aroimation o true, analytical sensitivity 3 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco o o -

14 Finite Differences (V) Secon-orer finite ifference aroimation of secon erivative: f ''( ) o f f f o o o f o - o o + Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

15 Graient Error Errors of Finite Differencing Caution: - Finite ifferencing always has errors - Very eenent on erturbation size, = = (,)=3 (,) Choice of is critical 0-5 Truncation Errors ~ Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco Rouning Errors ~ / Perturbation Ste Size

16 Error Analysis Perturbation Size Choice / A f - Forwar ifference /3 f - Central ifference A Machine Precision Ste size at k-th iteration k (Gill et al. 98) Trial an Error tyical value ~ 0.-% k 0 q theoretical function o ~ q-# of igits of machine A comute values Precision for real numbers 6 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

17 Comutational Eense of FD F i n F i z n F i Cost of a single obective function evaluation of i Cost of graient vector one-sie finite ifference aroimation for i for a esign vector of length n Cost of acobian finite ifference aroimation with z obective functions Eamle: 6 obectives 30 esign variables sec er function evaluation 3 min of CPU time for a single acobian estimate - eensive! 7 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

18 Comle Ste Derivative Similar to finite ifferences, but uses an imaginary ste f [ f ( + iδ) ] Im 0 '( 0) Δ Secon orer accurate Can use very small ste sizes e.g. Δ 0 0 Doesn t have rouning error, since it oesn t erform subtraction Limite alication areas Coe must be able to hanle comle ste values.r.r.a. Martins, I.M. Kroo an.. Alonso, An automate metho for sensitivity analysis using comle variables, AIAA Paer , an Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

19 Automatic Differentiation Mathematical formulae are built from a finite set of basic functions, e.g. aitions, sin, e, etc. Using chain rule, ifferentiate analysis coe: a statements that generate erivatives of the basic functions Tracks numerical values of erivatives, oes not track symbolically as iscusse before Oututs moifie rogram = original + erivative caability e.g., ADIFOR (FORTRAN), TAPENADE (C, FORTRAN), TOMLAB (MATLAB), many more Resources at htt:// 9 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

20 Aoint Methos Consier the following roblem: Minimize s.t. (, u) R (, u) 0 where are the esign variables an u are the state variables. The constraints reresent the state equation. e.g. wing esign: are shae variables, u are flow variables, R(,u)=0 reresents the Navier Stokes equations. We nee to comute the graients of wrt : u u Tyically the imension of u is very high (thousans/millions).

21 Aoint Methos To comute u/, ifferentiate the state equation: Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco u u 0 u u R R R R u u R R u R u

22 Aoint Methos We have Now efine Then to etermine the graient: First solve (aoint equation) Then comute R u R u u u T u R u λ T T u λ u R R λ T T λ

23 Aoint Methos Solving aoint equation R u T λ u T about same cost as solving forwar roblem (function evaluation) Aoints wiely use in aeroynamic shae otimization, otimal flow control, geohysics alications, etc. Some automatic ifferentiation tools have reverse moe for comuting aoints 3 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

24 Post-Processing Sensitivity Analysis A sensitivity analysis is an imortant comonent of ost-rocessing Key to unerstaning which esign variables, constraints, an arameters are imortant rivers for the otimum solution How sensitive is the otimal solution * to changes or erturbations of the esign variables *? How sensitive is the otimal solution * to changes in the constraints g(), h() an fie arameters? 4 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

25 Sensitivity Analysis: Aircraft Questions for aircraft esign: How oes my solution change if I change the cruise altitue? change the cruise see? change the range? change material roerties? rela the constraint on ayloa?... 5 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

26 Sensitivity Analysis: Sacecraft Questions for sacecraft esign: How oes my solution change if I change the orbital altitue? change the transmission frequency? change the secific imulse of the roellant? change launch vehicle? Change esire mission lifetime?... 6 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

27 Sensitivity Analysis How oes the otimal solution change as we change the roblem arameters? effect on esign variables effect on obective function effect on constraints Want to answer this question without having to solve the otimization roblem again. 7 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

28 Normalization In orer to comare sensitivities from ifferent esign variables in terms of their relative sensitivity it may be necessary to normalize: i o io, = ( o ) i i i raw - unnormalize sensitivity = artial erivative evaluate at oint i,o o Normalize sensitivity catures relative sensitivity ~ % change in obective er % change in esign variable Imortant for comaring effect between esign variables 8 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

29 Eamle: Dairy Farm Problem Dairy Farm samle roblem L Length = 00 [m] N - # of cows = 0 cow cow R N cow fence L Parameters: f=00$/m n=000$/cow m=$/liter A LR R F L R M 00 A/ N R Raius = 50 [m] o With resect to which esign variable is the obective most sensitive? C f F n N I N M m P I C 9 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

30 Design Variable Dairy Farm Sensitivity Comute obective at o Then comute raw sensitivities Normalize o 0 o ( ) Show grahically with tornao chart o ( ) Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco P L P N P R Dairy Farm Normalize Sensitivities R N L

31 Realistic Eamle: Sacecraft NASA Neus Sacecraft Concet 60 = Centroi itter on Focal Plane [RSS LOS] 40 T=5 sec iel X Sacecraft CAD moel Z Y 0 meters Finite Element Moel Image by MIT OenCourseWare omain -omain What are the esign variables that are rivers of system erformance? 3 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco Centroi Y [ m] m Simulation Requirement:,req =5 m Centroi X [ m]

32 control variables otics variables Design Variables structural variables isturbance variables : Norm Sensitivities: RMMS WFE Ru Us U fc Qc Tst Srg Sst Tgs m_sm K_yPM K_rISO m_bus K_zet t_s I_ss I_rot zeta lamba Ro QE Mgs fca Kc Kcf analytical finite ifference / * / o,o Grahical Reresentation Ru Us U fc Qc Tst Srg Sst Tgs m_sm K_yPM K_rISO m_bus K_zet t_s I_ss I_rot zeta lamba Ro QE Mgs fca Kc Kcf : Norm Sensitivities: RSS LOS o /,o * / Grahical Reresentation of acobian evaluate at esign o, normalize for comarison. : RMMS WFE most sensitive to: Ru - uer wheel see limit [RPM] Sst - star tracker noise [asec] K_rISO - isolator oint stiffness [Nm/ra] K_zet - eloy etal stiffness [N/m] : RSS LOS most sensitive to: U - ynamic wheel imbalance [gcm ] K_rISO - isolator oint stiffness [Nm/ra] zeta - roortional aming ratio [-] Mgs - guie star magnitue [mag] Kcf - FSM controller gain [-] 3 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco 0 o R K u u cf R K cf

33 Parameters R Parameters are the fie assumtions. How sensitive is the otimal solution * with resect to fie arameters? Eamle: Dairy Farm samle roblem cow N cow cow Otimal solution: * =[ R=06.m, L=0m, N=7 cows] T Fie arameters: Parameters: f=00$/m - Cost of fence n=000$/cow - Cost of a single cow m=$/liter - Market rice of milk fence Maimize Profit L How oes * change as arameters change? 33 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

34 KKT conitions: Sensitivity Analysis * ( *) g( ) 0 M gˆ ( *) 0, M 0, For a small change in a arameter,, we require that the KKT conitions remain vali: Rewrite first equation: i 34 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco M (KKT conitions) gˆ * * ( ) ( ) 0, i,..., M i ˆ 0 n Set of active constraints

35 Sensitivity Analysis Sensitivity Analysis Recall chain rule. If: then n Y Y Y ( ) ) (, Y Y = i k i Y Y Y = = + A l i t fi t ti f KKT iti Alying to first equation of KKT conitions: ), ( ˆ ) ( ), ( + g λ 0 ˆ ˆ ) ( + k n M i i g g g λ λ λ λ 0 = = = M i k k M k i k i M i i g g g λ λ n λ 0 = + + = i M i n k k ik c B A λ 35 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

36 Sensitivity Analysis Perform same roceure on equation: ( *, ) 0 gˆ n k B k gˆ n k k k k 0 0 g 36 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

37 c In matri form we can write: n M i A B ik i n M Sensitivity Analysis A B c B T g ˆ i k M i k g ˆ i gˆ i M i gˆ 0 λ 37 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco n λ 0 M

38 Sensitivity Analysis We solve the system to fin an, then the sensitivity of the obective function with resect to can be foun: T (first-orer aroimation) To assess the effect of changing a ifferent arameter, we only nee to calculate a new RHS in the matri system. 38 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

39 Sensitivity Analysis - Constraints We also nee to assess when an active constraint will become inactive an vice versa An active constraint will become inactive when its Lagrange multilier goes to zero: Fin the that makes zero: 0 M This is the amount by which we can change before the th constraint becomes inactive (to a first orer aroimation) 39 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

40 Sensitivity Analysis - Constraints An inactive constraint will become active when g () goes to zero: T g ( ) g ( *) g ( *) 0 Fin the that makes g zero: g g ( *) ( *) T for all not active at * This is the amount by which we can change before the th constraint becomes active (to a first orer aroimation) If we want to change by a larger amount, then the roblem must be solve again incluing the new constraint Only vali close to the otimum 40 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

41 Lagrange Multilier Interretation Consier the roblem: minimize () s.t. h()=0 with otimal solution * What haens if we change constraint k by a small amount? hk ( * ) h ( * ) 0, k 4 Differentiating w.r.t h k * * h 0, Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco k

42 Lagrange Multilier Interretation How oes the obective function change? Using KKT conitions: * h * h * k Lagrange multilier is negative of sensitivity of cost function to constraint value. Also calle shaow rice. 4 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

43 Lecture Summary Graient calculation aroaches Analytical an Symbolic Finite ifference Automatic Differentiation Aoint methos Sensitivity analysis Yiels imortant information about the esign sace, both as the otimization is roceeing an once the otimal solution has been reache. Reaing Paalambros Section 8. Comuting Derivatives 43 Massachusetts Institute of Technology - Prof. e Weck an Prof. Willco

44 MIT OenCourseWare htt://ocw.mit.eu ESD.77 / Multiiscilinary System Design Otimization Sring 00 For information about citing these materials or our Terms of Use, visit: htt://ocw.mit.eu/terms.