G-L-4: Evaluation. Prof. Dr. Christos Spitas Dr. Y. Song (Wolf) Faculty of Industrial Design Engineering Delft University of Technology

Transcription

1 G-L-4: Evaluation Prof. Dr. Christos Spitas Dr. Y. Song (Wolf) Faculty of Industrial Design Engineering Delft University of Technology 1

2 Contents 1 Why evaluation? 2 Define the evaluation criteria 3 Sensitivity analysis of the simulation of the system 4 What did we learn today? 2

3 Evaluation in the modelling cycle what are you studying? Case brief choices touch your thoughts what exactly...? System Build abstract model what do you foresee? choices Thought simulation choices what does your model foresee? Solve/ simulate choices Experience everything you did/ saw/ felt/ remember to be true... did you both agree? Compare Revisit choices Finish! Acceptable? do you need more insights/ data? Experiment Courtesy of centech.com.pl and 3

4 Evaluation Identify, choose Modelling Learn System Solve Evaluation is systematic determination of merit, worth, and significance of something or someone using metrics against a set of quantified expectations Evaluation 4

5 A case study: Heat the house 1 C in 3 min Design brief Specify the areas of radiators and the hot water temperature for the boiler in a house in order to raise the room temperature 1 degree in 3 minutes Courtesy of 5

6 Components Boiler House Water Radiator Air Courtesy of 6

7 Cause - effect Boiler Water Radiator The house is heated Air 7

8 Choice Neglect the heat capacity of radiator Heat transfer from radiator to air is uniform Adiabatic house Model And many more 8

9 Modelling Air temperature Heat capacity of air Mass of air m air c air d dt T air ( t) Heat transfer coefficient T T 1 ha water air radiator Hot water temperature from the heater ( t) The surface area of radiator We choose The initial temperature of room is 10 C The specific heat capacity of air is 1012 J/(kg*K) The volume of the house is 350 m 3 The density of air kg/m 3 The heat transfer coefficient is 10 W/m 2 9

10 Define the evaluation criteria 10

11 Define the evaluation criteria The metric Parameter: design parameter, chosen by designer Input: any intended or unintended excitation of the system Input 1 Input 2 Parameter 1 Parameter 1 System Metric function Usually from design brief Output 1 Output 2 Output: system response (state) 11

12 What is the metric in this system? An output (against expectations) Part of an output (against expectations) Combined (parts of) outputs Combined (parts of) inputs / parameters / outputs Courtesy of 12

13 The metric and the criteria Criterion is the metric for judging (satisfaction) t M T 1C M 1ºC Time to raise the temperature by 1ºC Part of the criterion The metric M air t f T A T ( t t ) T ( t) T air where (, ) M water M radiator Part of the criterion 180 seconds 13

14 Sensitivity analysis 14

15 We cannot always explore every possible solution An example - Drop an cup on the floor Simulate from 0 to second Only one step Computation time: 48 hours on Pentium GHZ CPU 15

16 Sensitivity analysis one parameter vs one metric Input /parameter1 System Metric Derivative of the metric (f) w.r.t. input / parameter(p) d dp f ( p) 16

17 Sensitivity analysis multiple parameters vs one metric Input /parameter1 Input /parameter 2 Input /parameter System Metric Gradient of the metric (f) w.r.t. inputs / parameters (p 1,p 2...,p n ) f ( f ( p1, p2,... pn), f ( p1, p2,... pn),, f ( p1, p2 p p p 1 2 n,... p n )) 17

18 Sensitivity analysis: An Example x y f ( x, y) 2 2 x y xe f ( x, y) f(x, y) 18

19 Sensitivity analysis: An Example x y f ( x, y) 2 2 x y f ( x, y) xe Sensitive Insensitive ( f ( xy, ) ( f( xy, ) f( x, y) (, ) x y 19

20 20 Numerical approach of partial differences p p p p f p p p p f p p p f p n n n ),..., ( ),..., ( ),..., ( ), ( y x xe y x f ) (0 0 0 ) (0 ), ( y x y y x x e x e e x y x f x Example:

21 Sensitivity analysis of the simulation 21

22 Sensitivity analysis in simulation Suppose the temperature of house is 10, the water is 55 and the surface area of radiators is 5m 2, the model is d dt T ( t) T ( t) Using Euler method to solve differential equation over the interval t= seconds. Euler method: d yt ( ) yt ( t) yt ( ) t d t d yt ( ) yt ( t) yt ( ) lim d t t 0 t 22

23 Euler method in detail Euler method: d yt ( ) yt ( t) yt ( ) t d t 23

24 Sensitivity analysis in simulation 1 step Errors 6 steps 12 steps 3 steps Steps in Euler method 24

25 Sensitivity analysis of the system 25

26 Evaluations the metric w.r.t initial temperature Time to raise the temperature by 1 C 180 seconds Choice Initial room temperature 26

27 Evaluations the metric w.r.t. surface area & water temperature Time needed to raise the temperature by 1 C m air c air d dt T air ( t) T T 1 ha water air radiator ( t) Area of radiator (m 2 ) Water temperature (ºC) 27

28 Evaluations s s s s s Safe zone Area of radiator (m 2 ) Water temperature (ºC) 28

29 Recall: A more complicate case Simulate from 0 to second Only one step Computation time: 48 hours on Pentium GHZ CPU 29

30 Evaluation of the results: Sensitivity analysis 1 How much do we trust our model? (sensitivity to solution parameters) 2 Think smart, not hard: Substitutes calculationintensive blind searches of the solutions space 3 Design for insensitivity / toleranceinsensitive, wide tolerances 30

31 What did we learn today? How to establish metric(s) (functions) Proper adjustment of design parameters leads to better designs Parameters play different roles in both modelling and simulation 31

32 Success! Prof. Dr. Christos Spitas Dr. Y. Song (Wolf) Faculty of Industrial Design Engineering Delft University of Technology 32