How to Plan Eperiments Formulate a hypothesis Choose independent variable values Eperimental planning Scientific method Statistical design of eperiments Setting specifications
Formulating a Hypothesis Must be objectively testable Be careful when stating the hypothesis Are you stating the null hypothesis or the alternative hypothesis? Alternative hypothesis H A : testing whether A > B Null hypothesis H 0 : test if A==B Fine point, but can cause you to miss what s really going on due to your own preconceived notions of the system s behavior Eample: r A (T 1 )>r A (T 2 ) in a packed bed with porous catalytic particles. Better stated, We will eperimentally determine whether r A (T 1 ) is equal to r A (T 2 ) in a packed bed with porous catalytic particles when Q, P, C A,0, and d p are fied. Properly formulated hypothesis demonstrates an understanding of important variables Naturally suggests controls Indicates which factors/conditions/treatments will be tested Sometimes, the the prediction will be wrong Does not mean eperiment is a failure Leaps in learning/understanding often the result of an incorrect prediction
Radius of Curvature Formulating a Hypothesis Eample: The lens flattens due to stretching. Original shape h 0 F R 0 h R F 2a 0 2a 8-arm model Lens Diameter
Elastic Modulus (Pa). Formulating a Hypothesis Eample: Lens stiffness comes from its architecture. 2500 2000 E 1023 0.202 0.594 R 2 0.808 1.1 Data Model Fit 1500 1000 500 0 0 1 2 3 4 5 Distance from Center of Nucleus (mm)
Reaction Rate Independent Variable Value Selection Need to know something about your system Look for dynamic range Concentration Limited Intermediate Regime (Dynamic Range) Saturation Threshold Diffusion Limited Particle Size
Independent Variable Value Selection Linearly spaced points: i+1 = i +1 Log spaced points: i+1 =2 i
One Factor at a Time (OFAT) Vary one independent variable at a time until the value of the dependent variable is optimal across the range of independent variable tested AKA The Scientific Method Eample: Minimize the heat input Q to a distillation column while maintaining a known distillate composition y d Can easily find Q for a given C A,0 Will minimum Q be the same if C A,0 increases/decreases? OFAT gives us no insight into this problem: a change in C A,0 will result in a change in y d
Statistical Eperimental Design (DOE) Test all independent variables at carefully selected values Use math to tell you the relative importance of each factor and their interactions Eample: Minimize the heat input Q to a distillation column while maintaining a known distillate composition y d Determine range of feasible operating conditions (e.g. Q, C A,0 ) and input these values to the software Measure the response y d at each of these points Now can control the system such that y d is maintained despite changing C A,0 by systematically varying Q
Statistical Eperimental Design Scientific Method - OFAT OFAT: One Factor At a Time Manipulate one variable 1 while holding the other variables ( 2 n ) constant until the best response y( 1 ) is found, then move on to the net variable 2. Repeat for all n, then hope that the interactions aren t important. Time consuming and inefficient ( 1% inspiration, 99% perspiration Thomas Edison) Gives no insight into interactions Optimum response y( 1 ) will generally not hold for all values of 2 Better Method - DOE DOE: Design Of Eperiments Choose values of all variables, then compute the optimum response. Efficient Gives eplicit values for interactions Optimum response will be the true optimum response See http://www.statease.com/pubs/doeprimer.pdf.
One Factor at a Time (OFAT) 1. Vary 1 until optimal y( 1 ) is found. Hold 1 constant at this value. 2. Vary 2 until optimal y( 2 ) is found. Hold 1 and 2 constant at these values. 3. 4. Vary n until optimal y( k ) is found. Hold 1 k constant at these value. 5. Hope that y( 1, 2,, k ) is really the optimal Image from http://www.compassdude.com
Statistical Design of Eperiments (DOE) 1. Set limits on each 2. Choose a type of eperimental design 3. Use software to generate candidate points 4. Test each point in the candidate matri 5. Determine effects of each i and interactions i j 6. Determine the true optimal
Conveity and Search Compleity Previous eample was conve Any search would eventually find an optimum What about non-conve surfaces?
Types of DOE Designs Screening Eperiments Used to determine which independent variables significantly impact your response Gives linear estimate of factor effects Different designs for different resolutions Full factorial Fractional factorial Response Surfaces Used to determine mapping between independent and dependent variables Different designs for different systems D-optimal Mitures Bo-Behnken Central Composite
Screening Eperiments Full factorial design 2 k eperiments required Eposes all primary effects and first-order interactions (2FI) 1, 2, 3, 1 2, 1 3, 2 3, 1 2 3 In MATLAB, use ff2n(k) to get X Fractional factorial design Sacrifice resolution for epediency Possible to reduce number of required eperiments to only k+1 for main effects 1, 2,, k Main effects may be confounded with interactions
Confounding Factors Confounding Factors Potential effects which mask each other in DOE analysis in fractional factorial designs Blocking variables Variables which can t be controlled, but accounting for them may elucidate results
Statistical Eperiment Design Rapidly evaluate the dependence of the dependent variable on any number of independent variables and their interactions Solid mathematical foundation Provably minimal number of eperiments Available software tools do most of the work for you Factorial design Eamine k factors interactions 2 k eperiments Fractional factorial design Eamine k factors Some interactions will be confounded with primary factors <2 k eperiments (down to k+1 only looking at main factors) Useful for initial screening for important factors
How Linear DOE Works 1. Select independent variables 2. Select bounds of each independent variable 3. Select the level of interaction to eamine usually look at two factor interactions (2FI) 4. Determine the standard order eperimental plan 5. Randomize the order of eperimental runs The MR5 (minimum resolution 5 main and 2FI) layout for si factors in case study: + means the factor is set to the high bound value; - means it is set to the low bound value.
How Linear DOE Works 6. Perform the eperiments listed in the test matri 7. Apply ANOVA and linear regression model to determine effects and interactions 8. Use the trained regression model to make predictions about your system 9. Investigate nonlinear response to important factors and interactions using response surface methods (if desired/necessary) y( n c c c 1,..., ) 0 4 4 14 1 4 Linear model with all main and 2FI: y( 1,..., n) 6 k 3 c 2k 2 c k 0 6 l4 6 i1 c 3l c 3 i l i 6 6 j2 m5 c c 1 j 4m 1 4 j m c 56 5 6 Linear optimization methods used to find the optimal value(s) of y( 1,, n ) within the bounds of the independent variables.
How Response Surface Methods Work 1. Select important factors and interactions based on pilot study 2. Select bounds on these values 3. Input into software 4. Conduct eperiments 5. Input responses into software software will analyze and report most suitable model(s) 6. Select type of model (e.g. quadratic, cubic) 7. Predict conditions which yield the optimum response Figure 2. This is how OFAT sees the relationship between the response variable and factor A. Figure 3. This is how OFAT sees the relationship between the response variable and factor B. Figure 1. The real contour of the response to factors A and B: interactions are important. Figure 4. Response surface shows a pure interaction of two factors. Eample from http://www.chemicalprocessing.com/articles/2006/166.html
Setting Specifications General Approach Etract dependent variable specifications from stated project goals Render initial feasibility decision based on order of magnitude estimates Relate stated goals to independent (control) variables by identifying suitable models Identify important independent (control) variables based on suitable models Specify independent variable tolerances by back-calculating from dependent using models Identify suitable measurement and control equipment Eample: Chemical Reactor Capital budget ( $100,000) Product purity (3 ± 0.001 M) Production rate ( 100 M/hr) Product temperature (35 ± 0.1 C) Review literature Preliminary design calculations Your eperience and epertise Cost charts CSTR, PFR, laminar flow reactor? Rate equations 1. Reactor size 2. Feed concentration 3. Feed flowrate 4. Temperature 1. ±10 C, ±1 C, or ±0.1 C? 2. ±1M, ±0.1M, or ±0.01M? 3. ±1m 3 /s, ±0.1m 3 /s, or ±0.01m 3 /s? 4. ±1m 3, ±0.1m 3, or ±0.01m 3? 1. Thermostat/on-off switch +heater 2. Refractometer + injector 3. Flowmeter + control valves 4. Drilling/lathe/computer-guided laser Sanity check Can we really do this with proposed constraints? within proposed tolerances? without violating laws of thermodynamics?
Summary With enough time and money, anything is possible Never have enough time or money Don t let perfect be the enemy of good enough Learning to identify the important factors in an eperiment, process, or equipment design is an invaluable skill Learn to identify the limits and constraints of your system System could be a company, process, or component design Accurately assessing and specifying your project can save you and your company/lab time and money
Conclusions DOE maimizes information obtained per eperiment Save effort, time, and money Improve process control and stability More robust than the scientific method Eamples here are for systems with only 2-3 independent variables Benefits of DOE are even more obvious in higher dimensions Even in worst-case scenario, DOE is as good or better than OFAT